Feed Your Head!

Does iPad “talking practice” boost oral fluency? A look at Schenker & Kraemer (2017).


In a 2017 paper, Schenker and Kraemer argue that iPad use helps develop oral fluency. Specifically, they found that iPad app users after “speaking practice” were able to say more in German, and were more fluent– rapid and seamless– in saying it than were controls who had not “practiced” speaking. 
So, prima facie, the authors can claim that focused speaking practice helps develop fluency. 

Q: Does this claim hold up?

A: Not according to their evidence. 

Let’s start with the method. Kraemer and Schenker took English L1 students of second-year German, divided them into two groups, and gave one batch iPads. The iPad group had to use Adobe Voice to record three tasks per week, which had to be posted to a group blog. In addition, each iPad user had to respond verbally to some other students’ posted responses to the tasks. 

The tasks included things such as “describe your room” and “recommend a movie to a friend.”

The control group did nothing outside class other than their usual homework, and the iPad group had their other homework (which the authors do not detail, but describe as work involving “vocabulary and grammar knowledge”) slightly reduced in quantity. 

In terms of results, the iPad group during oral testing on average said more, and was more fluent (using language “seamlessly”) than the control.  The authors thereby claim that “practice speaking” boosted oral competence. 

However, there are a number of atudy design flaws which render the authors’ conclusions problematic.

First, the study compares apples and oranges. The speaking group practised, well, speaking, while the controls did not. The speaking group had more time with German (class, plus speaking, plus doing whatever they did to prepare their recordings, plus listening and responding to others’ posted task responses) than did the controls (class, plus “vocabulary and grammar” hwk). The speaking group had more time doing speaking as well as more total German time than the controls. 

This is akin to studying physical fitness by comparing people who work out with those who are couch potatoes, or by comparing people who do two hours a week of working out with those who do four. 

Second, the study does not compare speaking development-focused methods. One group “practiced speaking,” while the other did “vocabulary and grammar” homework.
 This is like comparing strength gains between a group of people who only run two hours a week with another group that runs two hours a week and lifts weights. Yes, both will get fitter, and both will be able to lift more weights  and run a bit faster (overall fitness provides some strength gains, and vice-versa).  

However, what should have been compared here are different ways of developing oral fluency. (We should note that fluency first requires broad comprehension, because you cannot respond to what you don’t understand). 

We could develop oral fluency by 

• listening to various kinds of target-language input (stories, conversations, news etc). 

• watching target-language, L1-subtitled film. 

• reading (it boosts vocabulary). 

Schenker and Kraemer’s “practice speaking” will help (at least in the short term). One could also in theory mix all of these, as a typical class does.

Schenker and Kraemer, however, compare one approach to developing speaking with an approach that does nothing at all to address speaking. 

A more persuasive study design would have had three groups: a control, and two different “speaking development” groups. The “speaking development” groups could have included those doing Schenker & Kraemer’s “practice talking” with, say, people listening to speech, or reading, or watching subtitled film (or a mix).  One group would spend 60 min per week recording German (and listening to 50-75 second German recordings made by their peers). The other would spend 60 min per week, say, listening to German. At the end, control, speakers and listeners would be tested and compared. 

Third, the study does not control for the role of aural (or other) input. The iPad group for one had to come up with their ideas. Since no relatively novice learner by definition comes up with much on their own, they must have gotten language somewhere (Kraemer and Schenker do not discuss what the students did pre-recording their German). My guess is, the speakers used dictionaries, Google translate, reading, grammar charts, things they heard on Youtube, anything they remembered/wrote down from class, possibly Duolingo etc, to “figure out” what to say and how to say it. If you were recording work, being marked on it, and having it responded to by strangers, you would surely make it sound as good as you could…and that (in a language class) could only mean getting extra input.  So did the speaking group get better at speaking because they “practiced speaking,” because they (probably) got help pre-recording, or both? 

Which leads us to the next problem, namely, that the iPad group got aural input which the control group did not. Recall that the iPad group not only had to post their recordings, they also had to listen and respond to these recordings. So, again, did the iPad group get better because they talked, or because they also listened to others’ recordings of German?

Finally, there was no delayed post-test to see if the results “stuck.”  Even if the design had shown the effectiveness of speaking “practice” (which in my view it did not), no delayed post test = no real results. 

The upshot is this: the iPad group got more input, spent more time listening, spent more total time with German, and spent more time preparing, than did the controls. This looks (to me) like a problematic study design. Ideally, both groups would have had the same input, the same amount of listening, etc, with the only difference being that the iPad group recorded their tasks. 

Anyway, the skill-builders’ quest continues for the Holy Grail of evidence that talking, in and of itself, helps us learn to talk. 

The implications for classroom teachers are (in my view) that this is waaaay too much work for too few results. The teacher has to set the tasks (and the blog, iPad apps, etc) up, then check to make sure students are doing the work, and then test them. Sounds like a lot of work! 

Better practice– if one feels one must assign homework– would be to have students listen to a story, or watch a video in the T.L., and answer some basic questions about that. This way people are focused on processing input, which the research clearly says drives acquisition. 

On a personal note, I’m too lazy to plan and assess this sort of thing. My homework is whatever we don’t get done in class, and always involves reading. 

Advertisements

How clear should I make it?

I gave an all-day workshop in Victoria last Friday and as usual began with a German demo:  asking a story, PQA, Textivate, Picturetalk, Movietalk, embedded reading.  I’ll briefly mention two things of note:

Here’s a few German words:

hatten = had          eine = a, an          Katze = cat

Can you figure out this sentence?

“John und Candice hatten eine Katze”

Right:  “John and Candice had a cat.”

I had written on the board glücklich = 🙂 .  Glücklich means “happy” and sometimes “lucky” in German.  During storyasking, I had used the word “und” many, many times (but I had not written it on the board, or translated it on the embedded reading), and I had also used the word glücklich a bunch.

Near the end of the demo, a participant asked “what does glücklich mean?”  Another participant then asked “what does und mean?”

I was floored.  What, I thought, could be more obvious than 🙂 = happy?  What could possibly be more obvious than und means “and”?  These were language teachers who wanted to be there, who wanted to acquire some German, and who had the metacognitive skills to know when things weren’t clear and ask for help.  All of them spoke at least two languages, and most had studied more at some point.

Today’s question: How clear and unambiguous should I make my classroom target  language?

Today’s answer: Even clearer.  🙂

 

 

The Zen of Language Teaching

Here are your koans.  

If you want to successfully teach grammar, do not teach grammar.

If you want your students to talk, do not ask them to talk.

If you want your students to write well, do not make them practise writing.

If you want them to acquire more words, teach them fewer words.

If you want to make them fluent, do not try to make them fluent.

If you want your students to acquire a language, do not teach them about the language.

If you want your students to know the meanings of lists of words, do not give them lists of words.

If you want your students to spell properly, do not make them practise spelling.

Just because nothing appears to happening doesn’t mean nothing is actually happening.

Just because something is happening doesn’t mean anything is happening.

Don’t just say something– sit there!  

If you want your students to read, do not teach them how or what to read.

If you want your students to prepare for the unknown, make them comfortable with what they know.

A student without a language dictionary is like a fish without a bicycle (sorry, Gloria).

A language classroom with lists of words is like a phone book with stories.

In order to see exactly how much influence you have over the specific language your students acquire, lie in the grass and stare at the clouds. 

“If you want to build a ship, do not gather the men to collect wood, divide up the work, or give orders.  Teach them instead to yearn for the vast and infinite sea.”– Antoine de St. Exupery

As always, the ideas here are are grounded in research, and this one was inspired by Mandarin and S.L.A. guru Terry Waltz.

The game of Go, computers, and language acquisition

I’ve been playing the ancient game of Go for years now– badly; I am ranked around 6 kyu at my best– and there are some fascinating lessons to be learned from Go play and attempts to program computers to play Go.

Go, like chess, is a strategy game where each player moves in turn, each sees everything the other player sees, each move opens certain possibilities and closes others (it’s deterministic), and there can generally be only one winner (draws happen only very occasionally).  Go is big business and has a long history: the top players earn seven-figure salaries; there are Go-only TV channels in China, Korea and Japan; there is a rich history of game recording and analysis stretching back to the sixteenth century; Go championship matches draw hundreds of millions of viewers; Go was considered one of the essential brain-training tools for samurai, emperors, etc.

Now, in terms of complexity, Go is literally billions of times more complex than chess.  A Go game has 361 possible opening moves; a chess game 20.  In Go, the typical game length is 200-250 moves; in chess it is 40.  There are more potential Go games than there are atoms in the Universe.  Computer games reflect this:  in chess, Deep Blue beat Grand Master Kary Kasparov in 1997; off-the-shelf software now always beats top humans; chess software’s greatest challenger is other chess software.  Go, however, has proven a harder nut for the egg-heads to crack.  If you imagine players ranked from zero (your kid learning to play) to 20 (the best two or three players in the world), Go programs now operate at about a five; chess programs at 25 or so.

In this article, the challenges facing Go programmers are detailed.  And there are a lot of interesting things we languages teachers can learn from research into programming Go and into what makes top players tick.

First , when analysing top players, an interesting pattern emerges.  It turns out that there is a fairly predictable order of acquisition of strategies and a fairly consistent (as a function of time spent playing and analysing) speed of getting better.  Indeed, top players can often tell their lesser opponents’ ranks within ten moves.  But…when good players become great players, they don’t just get more skills…they get entirely different skills.  Move (and response) unpredictability goes way up in a kind of quantum leap.  What look like wild, crazy moves start happening, and these moves end up being the keys to winning.  Go programs don’t do that.  With them, it is a “more of same” and slow-and-steady approach…that doesn’t work.

So it is with languages.  As Chomsky and others have noted, while there are definite orders of acquisition (of sounds, grammar etc) with language, people start being able to do unusual things.  They can generate sentences they havn’t heard, they use grammar rules they havn’t had explained, there are mistakes they could make (but don’t) and they can understand things they have not had explained to them.

A friend of mine did her PhD in linguistics on ambiguous noun classes.  She knew that kids somehow figured out that, for example, the word “school” meant different things in different contexts, and what those meanings were.  E.g. you can say “I like school,” and “my school is close,” and “school is hard.”  In each, “school” has a slightly different meaning.  Kids exposed to nouns that are morphologically identical figure out the differences in meaning, and they do it amazingly quickly, and they do it without help.  The upshot of her experiments with noun acquisition was that the kids could not have figured out (from context) what the different meanings/uses were.  So she concluded, as Chomsky predicted and then showed, that the brain’s “language organ” has powers that cannot be explained by the quality/type/context of input alone.

Second, since people massively beat even the best computers at Go, we know that– somehow– they are making better decisions than the computer.  In other words,  Lee Sedol is somehow calculating– and comparing– quadrillions of decision trees per second in a championship match.  However, we know that the conscious brain processes only 2,000 bits of info per second.  So most of the real processing is not happening consciously.  When these top guys (and they are all guys) are asked “what are you thinking about?” during games, they usually say things like “well I just kinda look” or “that move felt right.”  Indeed, they cannot often explain– especially at the very top level– why they do certain things.  On the way to becoming masters, there is conscious study, reflection, etc– oral analysis of games, moves etc are part of the tradition– but when you’re in the moment, you just…let go and play.

Third, “skill” in Go (as in chess, poker, bridge, etc) comes– I think– as much if not more from observation as it does from play.  Good players spend an immense amount of time replaying old matches from the masters, dating right back to the 16th century.  They also ruthlessly review their own games, and watch other players playing, and now, with computers, can watch other people’s saved games.  (On Go servers, sometimes hundreds of people will tune in to watch the 8-dan players duke it out.)  This is input.  It’s also something like reading: you can “examine” at your own speed, go back, pause, etc.

While with games, playing obviously matters (and is the point), and observation is, as with languages, central.

Conclusions?

(a) Most language acquisition cannot happen consciously.  If you want to have maximum acquisition, you are going to have to let the unconscious do its work.  Indeed, you are going to have to get focus on rules etc out of the way as fast as possible.  Minimising grammar explanations, maximising interest, and making people happy and comfortable will get people “immersed” in the story.  Indeed, if you focus on the conscious brain, you are majorly limiting yourself and your students: 2,000 bits per second of processing, or billions?

(b)  If we provide quality input, we will eventually get “quantum leaps” in skills.  Kids will pick up and say things that you havn’t consciously “taught” them.  Today in Spanish we were playing with a story where Farakh doesn’t serve his chair-stacking detention with Mr Stolz, because, on his way to Mr Stolz’s class, he meets a talking cat, and he sooo wants a talking cat, and when he asks the cat “do you want to be my cat?” and the cat answers “no,  I don’t want to be your cat, you have to stack chairs!” he goes home and eats 3 pizzas in disappointment.  One kid, Wasim, blurted out Sólo Farakh quiere hablar con una gata. (“Only Farakh wants to talk to a cat”).  Perfect, unexpected, unprompted Spanish.  I have never used that sentence before.  These guys– level 2s with only a month or so left– are now starting to blurt things out.

The bottom line seems to be, while practice (via input) is necessary, most of what is happening in the brain– in Go as well as in learning a language– is beyond/below the conscious.