legacy methods

Does iPad “talking practice” boost oral fluency? A look at Schenker & Kraemer (2017).


In a 2017 paper, Schenker and Kraemer argue that iPad use helps develop oral fluency. Specifically, they found that iPad app users after “speaking practice” were able to say more in German, and were more fluent– rapid and seamless– in saying it than were controls who had not “practiced” speaking. 
So, prima facie, the authors can claim that focused speaking practice helps develop fluency. 

Q: Does this claim hold up?

A: Not according to their evidence. 

Let’s start with the method. Kraemer and Schenker took English L1 students of second-year German, divided them into two groups, and gave one batch iPads. The iPad group had to use Adobe Voice to record three tasks per week, which had to be posted to a group blog. In addition, each iPad user had to respond verbally to some other students’ posted responses to the tasks. 

The tasks included things such as “describe your room” and “recommend a movie to a friend.”

The control group did nothing outside class other than their usual homework, and the iPad group had their other homework (which the authors do not detail, but describe as work involving “vocabulary and grammar knowledge”) slightly reduced in quantity. 

In terms of results, the iPad group during oral testing on average said more, and was more fluent (using language “seamlessly”) than the control.  The authors thereby claim that “practice speaking” boosted oral competence. 

However, there are a number of atudy design flaws which render the authors’ conclusions problematic.

First, the study compares apples and oranges. The speaking group practised, well, speaking, while the controls did not. The speaking group had more time with German (class, plus speaking, plus doing whatever they did to prepare their recordings, plus listening and responding to others’ posted task responses) than did the controls (class, plus “vocabulary and grammar” hwk). The speaking group had more time doing speaking as well as more total German time than the controls. 

This is akin to studying physical fitness by comparing people who work out with those who are couch potatoes, or by comparing people who do two hours a week of working out with those who do four. 

Second, the study does not compare speaking development-focused methods. One group “practiced speaking,” while the other did “vocabulary and grammar” homework.
 This is like comparing strength gains between a group of people who only run two hours a week with another group that runs two hours a week and lifts weights. Yes, both will get fitter, and both will be able to lift more weights  and run a bit faster (overall fitness provides some strength gains, and vice-versa).  

However, what should have been compared here are different ways of developing oral fluency. (We should note that fluency first requires broad comprehension, because you cannot respond to what you don’t understand). 

We could develop oral fluency by 

• listening to various kinds of target-language input (stories, conversations, news etc). 

• watching target-language, L1-subtitled film. 

• reading (it boosts vocabulary). 

Schenker and Kraemer’s “practice speaking” will help (at least in the short term). One could also in theory mix all of these, as a typical class does.

Schenker and Kraemer, however, compare one approach to developing speaking with an approach that does nothing at all to address speaking. 

A more persuasive study design would have had three groups: a control, and two different “speaking development” groups. The “speaking development” groups could have included those doing Schenker & Kraemer’s “practice talking” with, say, people listening to speech, or reading, or watching subtitled film (or a mix).  One group would spend 60 min per week recording German (and listening to 50-75 second German recordings made by their peers). The other would spend 60 min per week, say, listening to German. At the end, control, speakers and listeners would be tested and compared. 

Third, the study does not control for the role of aural (or other) input. The iPad group for one had to come up with their ideas. Since no relatively novice learner by definition comes up with much on their own, they must have gotten language somewhere (Kraemer and Schenker do not discuss what the students did pre-recording their German). My guess is, the speakers used dictionaries, Google translate, reading, grammar charts, things they heard on Youtube, anything they remembered/wrote down from class, possibly Duolingo etc, to “figure out” what to say and how to say it. If you were recording work, being marked on it, and having it responded to by strangers, you would surely make it sound as good as you could…and that (in a language class) could only mean getting extra input.  So did the speaking group get better at speaking because they “practiced speaking,” because they (probably) got help pre-recording, or both? 

Which leads us to the next problem, namely, that the iPad group got aural input which the control group did not. Recall that the iPad group not only had to post their recordings, they also had to listen and respond to these recordings. So, again, did the iPad group get better because they talked, or because they also listened to others’ recordings of German?

Finally, there was no delayed post-test to see if the results “stuck.”  Even if the design had shown the effectiveness of speaking “practice” (which in my view it did not), no delayed post test = no real results. 

The upshot is this: the iPad group got more input, spent more time listening, spent more total time with German, and spent more time preparing, than did the controls. This looks (to me) like a problematic study design. Ideally, both groups would have had the same input, the same amount of listening, etc, with the only difference being that the iPad group recorded their tasks. 

Anyway, the skill-builders’ quest continues for the Holy Grail of evidence that talking, in and of itself, helps us learn to talk. 

The implications for classroom teachers are (in my view) that this is waaaay too much work for too few results. The teacher has to set the tasks (and the blog, iPad apps, etc) up, then check to make sure students are doing the work, and then test them. Sounds like a lot of work! 

Better practice– if one feels one must assign homework– would be to have students listen to a story, or watch a video in the T.L., and answer some basic questions about that. This way people are focused on processing input, which the research clearly says drives acquisition. 

On a personal note, I’m too lazy to plan and assess this sort of thing. My homework is whatever we don’t get done in class, and always involves reading. 

Advertisements

How Much Does T.P.R.S. Cost?

books pic

Image:  Omaha Public Library

Being poor sucks.  It is well-known that the poorer (and darker-skinned) you (and your school District) are, the worse your educational outcomes are, anywhere in North America or Europe.  In language education, the bias is even more specific: in a traditional “communicative” language program, by 5th year, the few remaining students will be affluent, white, with educated parents, and often female.  Yet everyone has the same innate capacity for language learning, so it’s got to be teaching that separates wealthier and whiter from darker and poorer.

Grant Boulanger has done some exemplary work in Minnesota, showing how good C.I.-based language instruction will enable all learners to do more-or-less equally well.  And the research is clear:  C.I.-based teaching narrows marks ranges and raises all of them.

We tend to argue for C.I.’s effectiveness by saying it works better and showing how amazingly well kids can write Chinese or Spanish, or speak it, etc.  Kids who get C.I.– through free voluntary reading, Movietalk, T.P.R.S. stories and reading, Picturetalk, etc– never do worse than grammar kids, occasionally do as well, and generally do significantly better.  But what if there were an economic argument to be made for adopting a C.I. program?

Our beloved Monsieur Tabernac has 30 kids in his French 1 class.  Every 10 years, his District replaces his French textbook program.  This year, he has options.  He can get the Communi-quête program (traditional “communicative” teaching, with videos, audio listening stuff, cahiers, etc) or he can go in for, say, Blaine Ray’s Look, I Can Talk books (which include readings, and let’s throw in a Movietalk book too) .  Let’s take a look at the costs of these options.

We are assuming that
1.  The resources will serve 30 kids, for 10 years
2. At the end, everything will get thrown out and bought anew
3. Each year, in each program, the kids will buy the workbook.

Communi-quête 

Text: 30 books x $65/book = $1950
Teacher book:                            $350
Audio CDs:                                   $200
Video DVD:                                   $190

TOTAL                                             $2700
Workbooks: $13/student/year

Look, I Can Talk

Textbook:                                  none
Teacher book:                            $30
Movietalk book                         $30
Green Bible how-to kit          $40
props for stories                       $100

TOTAL                                          $200

LICT workbooks (these
include readings)                     $14/student/year

So…the textbook option costs thirteen times as much as the T.P.R.S. optionwhile the per-year cost to the students is $1 higher for T.P.R.S.

So if Monsieur Tabernac was given $3,000 for his language program–use it or lose it; if you don’t buy stuff, the English department gets to order 400 more copies of Lord of the Flies etc– what should he buy?  The answer is obvious: the T.P.R.S. curriculum, and novels!

If he ditches the text, Tabernac has $2,800 to buy novels.  At about $5/novel from Blaine Ray or Carol Gaab, he can buy 18 class sets of 30 novels each.  Or, he could by 36 sets of 15 novels each (so the kids can have more free voluntary reading options).

Given what we know about how much student choice and readings and personalisation matter, the answer is a no-brainer: a C.I. curriculum will be cheaper, more fun, and waaaaay more effective.

This is also a significant issue for poorer Districts.  In wealthier areas, the richer, whiter kids can hire tutors, go to France in summer, etc, if the textbook is useless and they want to get better at French.  Poor kids don’t have those options…and if we want them to have a shot at college or Uni, money shouldn’t be wasted on bad textbooks that aren’t fun, don’t work, and cost too much.

But ssssshhhhh….don’t tell ACTFL or the textbook companies…

Old Myths Debunked

This post comes from Carol Gaab.  She is an author, teacher and San Francisco Giants language coach, as well as a presenter and all-around thinker.  Gaab has one of the most critical minds I have ever run into, and likes to dismantle misconceptions almost as much as she likes to show us interesting and effective ways to teach languages.

So here she is, responding to myths like “we must use authentic documents” and “we must practice speaking,” etc.  A fascinating read, and great if you are having discussions with colleagues who embrace older methods.  Thanks, Carol!