C.I.-taught Students Evaluated by A.C.T.F.L. Writing Standards

How well do C.I.-taught students do in terms of ACTFL writing standards? Well…pretty darned well, I’d say.

Inspired by a Facebook post, I thought I would measure some of my Spanish 1 students’ writing on the ACTFL scale.

Here is their criteria for Novice High

Writers at the Novice High sublevel are able to meet limited basic practical writing needs using lists, short messages, postcards, and simple notes. They are able to express themselves within the context in which the language was learned, relying mainly on practiced material. Their writing is focused on common elements of daily life. Novice High writers are able to recombine learned vocabulary and structures to create simple sentences on very familiar topics, but are not able to sustain sentence-level writing all the time. Due to inadequate vocabulary and/or grammar, writing at this level may only partially communicate the intentions of the writer. Novice High writing is often comprehensible to natives used to the writing of non-natives, but gaps in comprehension may occur.

Here are some writing samples.  This is Bani’s work, after about 60 hours of C.I. (I do mostly TPRS, along with Movietalk, Picturetalk and some Slavic-style Invisible “untargeted” stories.)

Let’s see…Bani uses a load of sentences (actually, she uses only sentences). She fully communicates her intentions. There are no gaps in comprehension, The writing is far beyond the “lists, short messages, postcards, and simple notes” that ACTFL says Novice High writers can produce.  So, where is Bani?

Considering her use of various verb tenses, clarity etc, I would say somewhere between Intermediate Mid and Intermediate Advanced. What do you think?

Next, we have Marcus. This kid has an IEP, and has missed about two weeks (~13 hrs) of class.  He has some behaviour challenges, some of which involve staying focused in class.  Here is his most recent story:

 

 

 This is obviously not even close in quantity or quality to Bani’s. He uses English, has some problems with basic verbs, is occasionally incomprehensible, and the story does not really flow.

So, where does this fit on the ACTFL scale? Well, here is their Novice Mid descriptor set:

Writers at the Novice Mid sublevel can reproduce from memory a modest number of words and phrases in context. They can supply limited information on simple forms and documents, and other basic biographical information, such as names, numbers, and nationality. Novice Mid writers exhibit a high degree of accuracy when writing on well-practiced, familiar topics using limited formulaic language. With less familiar topics, there is a marked decrease in accuracy. Errors in spelling or in the representation of symbols may be frequent. There is little evidence of functional writing skills. At this level, the writing may be difficult to understand even by those accustomed to non-native writers.

Marcus fits most of this.  However, he does use sentences, sometimes properly. So– at about 50 hrs of C.I., plus behaviour and learning challenges– he’s at Novice Mid.

The lessons?

1. C.I. works very well indeed, even for students who are not especially motivated or focused, or who have attendance issues. One of many key C.I. plusses: the vocabulary is constantly recycled in comprehensible but new ways.
2. C.I. does get the “grammar teaching” done, despite traditionalist “those TPRS kids don’t know grammar” complaints. As we have all experienced, the stereotypically successful  language-class kids– wealthier, whiter  and fairly L1-literate females– will pick up and memorise whatever grammar rules etc we throw at them. The rest, not so much. Bani can’t tell you what a verb is, or conjugate one in a chart, or explain the difference between preterite and imperfect verb tenses…but she can use them correctly and meaningfully. Grammar: my kids havn’t been taught it…but they got it.
3. C.I. is going to reach kids who would be dead in the water with a textbook. I have had loads of kids like Marcus over the years.  Most of them failed with the text.  Worse, most were disengaged.  Now, I’m not much of a teacher…so if *I* can get Markus this far, anyone can do well!
4. Anyone who has issues with department members who complain that eg “when I get your TPRS kids in Spanish 2, they can’t write out all the numbers from 555 to 579,” or “they can’t conjugate the verb traer in the pluperfect ablative subjunctive causal declension” can just point at ACTFL guidelines to show where their students are. Verb charts, memorised grammar rules, etc, are not part of ACTFL’s proficiency scales: the ability to write in contextually clear and meaningful ways is.
5. ACTFL broadly suggests that in a regular (ie non-Immersion) classroom, students will need about two years to get to Novice High, another two for Intermediate High, and two more to Advanced. These writing samples suggest that we can go waaaaay faster than ACTFL thinks.

One last thing:  these kids do well not because Mr Stolz is a brilliant teacher, but because C.I. methods allow us to stay in the target language much more than the textbook does.

 

 

 

 

 

 

Spring 2017 Spanish 1 Results

Here are two story writes. Students had 25 minutes to write a story. No notes, no dictionary, no questions: write what you can en español.

These kids have had about 35 hours of input, zero writing/talking practice, very few grammar pop-ups, no workbook etc exercises.  As close to pure comprehensible input as you can get before moving into story listening.

First, Kaye. She is originally from the Philippines, and Spanish is her fourth language. I’m including her story because

A. it’s good
B. Kaye likes Duolingo…you can see her playing around with words I havn’t used.

Next, Bani. She is from the Punjab, and Spanish is her 4th language. What can I say? She rocks!

 

Note: not all students did this well. For the kids who are there for 19/20 classes and who pay attention, the average wordcount was about 175 and grammar mark 2.5/3. The skippers and burnouts do much, much worse.

The Curriculum That Actually Works For Everyone

Longtime C.I. badass Mike Peto recently weighed in on a good question. Peto, who led his entire department from textbook servitude to effective C.I. (all their kids who write A.P. and I.B. exams pass), addressed this:

We are getting rid of the textbook.  What should we teach? What are people’s favorite stories and themes?

This is a good question.  Standard language teaching is organised around cunningly-disguised grammar units: we learn the present tense verbs with school vocabulary, the conditional a year later with shopping, etc.

My answer: agree as a department to use 250 high-frequency words per year.  Peto’s is better:  focus on “the Super Seven verbs” (Terry Waltz’s list of highest-frequency verbs), along with nine more that add up to what Peto calls “the Sweet Sixteen verbs.”

 

Here is Peto’s explanation of how and why his language department organised their language program the way they did.

Short answer: 16 verbs & a commitment to fill every class with compelling CI.

Long answer: As a department chair trying to design a common experience for students in different classes, with a half dozen different teachers on staff, I could have insisted that every teacher follow the same collection of story scripts, movie talks, and novels. That is, “all Spanish 1 students will read X novel and discuss Z movie talk. All Spanish 2 students will acquire this list of target structures so that they will be “ready” for Spanish 3.”

The problem Peto correctly notes here is that we are not guaranteed acquisition of “rule”/word ______ on any teacher’s schedule. Just because you taught it doesn’t mean they got it.

 On the other hand, the Sweet 16 verbs represent a different path towards creating a common experience between classes. We do not simply repeat sixteen words for four years, but we do agree that structures with these verbs are the ones that are recycled and given priority at every step in the journey. The Sweet 16 gives a department the flexibility to allow their teachers and students to pursue different interests in class, to use different language, but guarantees that there will be a common communicative foundation throughout the entire program. For example, the Sweet 16 verbs allow one teacher to develop an independent reading program for her students in which students are all reading different books (and thus developing their own idiosyncratic vocabularies), while another teacher develops his CI skills guiding his students through an authentic telenovela, while another teacher spends a good part of her classes discussing student illustrations and spinning stories from them.

This is how it’s done: you make enough horizontal links– the sweet 16– to ensure that kids in different classes can end up a year later with the same teacher and be able to function.  And you leave it open enough that what kids (and teachers) find interesting can be explored.

We collect everything: novels, Martina Bex units, TPRS textbooks by Carol Gaab as well as the ones by Blaine Ray, our own stories, huge collections of movie talks, telenovelas. We invest in PD like some departments buy workbooks; we brought Ben Slavic to our department to show everyone how to make One Word Images and next summer they are all going to Boston to refresh their skills on classic TPRS techniques. When someone starts using a new item (book, movie, whatever) then that person has the right to claim it for a particular level. Teachers can opt in to use it in their classroom, but nobody is obliged to use any particular item. Therefore, I claimed El Internado for level 1 and Gran Hotel for level 3. Any Spanish 1 teacher could use El Internado. Someone else claimed Fiesta Fatal as a level 1 book… I then included it in my FVR library while my colleague taught it as a whole class novel. A colleague claimed La Vampirata as a Spanish 2 whole class novel, so I did not include that in my FVR library (which is potentially read by Spanish 1 students).

We recognized that in any classroom there will be many different interests, and that when students and teachers are following their own interests then the input is likely to be more compelling. More compelling leads to faster acquisition. That is the funny thing about those studies which try to count how many times a student needs to hear a word to fully acquire it… teachers know that swears might be fully acquired the very first time they are understood, whereas an abstract transition word that the student never uses in their own L1 could be uttered comprehensibly 500 times and not be fully acquired. 

There is another major advantage to running a department this way. When any of my teachers get students at the beginning of the year, we do not have a list of target structures in our minds that we assume our students have acquired. We do not get angry if our level 3 kids do not understand X phrase; instead we are trained to start the conversation assuming nothing and paying close attention to their eyes.

Or, as Blaine Ray so perfectly puts it, “figure out what they don’t know, and start there.”

At all levels, as we think about how to phrase our language so that it will be comprehensible, we all return to the Sweet 16 verbs and posters. It is a common experience in all classes, even though I spend a week talking about whales and my colleague spends weeks talking about football (what would you expect from a football coach!).

This illustrates exactly what high-frequency vocabulary does: it provides the flexible foundation onto which people can “stack” whatever vocabulary interests them. The football coach can play around with “el rodilla de Tom Brady se vuelve mejor” while in another class, students can create an OWI who “se vuelve loco por no haber comido.”

Terry Waltz has also famously made this point. She says, imagine you are learning Mandarin Chinese and you are going shopping, and you want to buy a wrench and some apples. If you had to pick one phrase which mattered most, what would it be?  That’s right: want to buy.  This works everywhere.  And ultra-low frequency words (hammer, apples, most clothing words, etc etc) you can figure out on the spot, or gesture, or look up, etc. Waltz elsewhere makes a similar point in response to a teacher who asks shoudl they not know food vocabulary? Waltz, who is translator-fluent in Mandarin (we are talking 10,000 characters plus real-time translation) says, despite having lived for years in Taiwan, that she doesn’t know most of the food words.  Is this a problem?  No, she says, most of the time when I (and Taiwanese) who don’t know the name of something we want to eat just say “I’ll have that” and point. Brilliant. The same is true in, say, Mexico: every region has basic staples– frijoles, tortillas, cerveza– but the local dishes are ultra-low frequency so you just ask about it when you are there. There is no point in teaching Spanish students chapulines, tlayudas, de lechon, etc.

This is also necessary because students move into our district at every level, and we cannot just leave them behind because they did not start with us. We need to provide a comprehensible experience at all levels, even if students missed the first 3 years of our CI  CI program because they were learning thematic vocab in another district.

This is brilliant:  C.I. accommodates everyone, while the textbook does not. The kid from Greg the Grammarian’s class will show up in Peto’s room and go, OMG wut? and probably spend quite a while simply listening.  However, whether or not he has spent four years or four weeks in Grammar Grinder Greg’s class, he will be able to catch up in Peto’s class.

Peto leaves out only one thing: when kids leave his school for Grammarian Greg’s class, high-frequency words will serve more of them much better than anything else.  A teacher (or department) who obsesses about themes and topics is rolling the dice on whether their alumni will end up in a class that uses the same vocabulary. Those who teach high-frequency vocabulary are preparing students for anything.

 

 

Some Notes on Level 1 Results (Fall 2017)

Meaningful discussion of language teaching is kind of like language learning itself: you need a thing to discuss. Language acquisition starts with stuff: language to be processed.  Language teaching discussion starts with student results.  This semester I have two English 9s, Social Justice 12 and intro Spanish, so I have too little time to “curate” results regularly.  Feb 2017 I have both more 1s and time so I will publish more results then.

Bott…ehhhmm, as they say in Ireland, here are two interesting recent story writes.  30 min, no notes or dictionaries, paper-and-pen only.  These students have had about 50 hrs of input.  Instruction is almost pure C.I.: no metacognition, grammar practice, talking practice, writing practice, K/W/A activities, peer group work, bla bla.  Just a lot of input.

This is the fourth story they have written.  Word target was 400.  Most kids are below average this year.  This is possibly because I am doing mostly Slavic-style “untargeted” input, where we have much less repetition of specific vocabulary targets in the short term.

First, here is Angela’s story.  Ang is Filipina and still has a bit of an accent.  She has been in Canada for two years, and reads a lot.  She is also a Duolingo user.  She tells me she is putting in about four hours/week.  Check it:

There is a bunch of stuff I havn’t used in there.  She is generally getting it right in terms of meaning but there are still grammatical holes.  I’m including this because, yes, Duolingo– which frankly bores the crap out of me– does deliver decent C.I.  Krashen noted in a paper that Duolingo works about as well as traditional college Spanish instruction, if the user can manage to stay interested, which most do not.  Angela says she likes the new story feature on Duolingo.

Next, we have Nisha.  She is Punjabi, and has had zero previous Spanish. Lotsa words, a good story, but some obvious issues (eg noun gender).  What is tough for the Punjabi kids (in writing) is that adjective and noun endings in Spanish– -a indicates feminine– indicate masculine in Punjabi (e.g. bacha = boy, bachi = girl, bache = kids) so we get confusion. Also in Punjabi (as nearly as I can tell) there is a lot less verb conjugation (or maybe I just can’t hear it when I ask them about it). Nisha is not a Duolingo user.  Her only Spanish is at school.

But anyway, props to Nisha for doing so well after only 50 hours.

 

To plan, or not to plan?

How do we know what a student can really do with an additional language, in, say, writing?  Suppose we wanted to be deadly boring for our students.  Hmmm, how about, we make them write about their daily routine, that should put both students and teacher nicely to sleep.

If we are a traditional teacher, we want to know what they have learned: what they can consciously do with language.  We could give them a writing prompt–describe your daily routine– time to plan/look at vocab/check out Google translate/”reflect on our learning”/plan out our metacognitive strategies/whatever, then give them a bunch of time to write, and then mark it.

If we are C.I. teachers, we want to see what students have acquired— what they can do without any planning, immediately. So we give them the same topic, zero prep time, a bit of time to write, then mark.

A teacher recently shared two writing samples from Spanish 1.  One sample is from a CI-taught kid in his class, the other from a kid taught by his grammarian colleague.

EXAMPLE A.  This is from the textbook/grammar teacher’s top student

• the teacher spent three weeks doing a “unit” on reflexive verbs
• the teacher’s students had just finished their “reflexive verb unit”
• they had time to “prepare” their writing
• they had 20 min. to write
• about 120 words
• quality: excellent, with minor errors

EXAMPLE B. from one of the C.I. teacher’s middle-of-the-road students

• the C.I. teacher spent about one week doing a story involving this vocab
• the story using this vocabulary was asked one month prior to this being written
• zero planning time
• 10 min. writing time
• 104 words
• Quality: excellent, with very minor errors

What I noticed:

• The CI student’s output is twice the speed of the traditionally-taught kid
• the CI kid does as well as the grammar kid with no planning time and half the writing time
• The CI kid has “been away” from the vocab for three weeks and uses it as well as the kid who just finished a “unit” on it
• The CI kid spent one week with this vocab while the grammar kid spent three weeks

Anyway…faster writing with zero prep time and less instructional time: C.I. is looking a lot better than the text.

Stories 1 Textbook 0

We are three weeks into Spanish 1. I have 30 students: 29 total beginners, and one kid, Marc, who has taken Spanish 1 and 2 at another school with a traditional teacher (where he got a high B both years). Marc is a solid student who pays attention in class. 

Another beginner, Mariam, arrived one week late because of holidays, and she has never taken Spanish. 

Here are their first story writes. They had 20 min (no notes or dictionaries) to rewrite our story with new names, places and details. 

Who would you expect to do better– the kid who’s taken Spanish for two years, or the kid who’s taken it for 11 days?  Check it: 

This is Marc’s story. He’s taken two years of Spanish

Now, here is Mariam’s. She has had eleven days of Spanish. 

Now let us be clear: not all the 1s wrote this much or well. Some wrote 30 words (goal is 100 for the story) with awful grammar. I’ll post stats when I have everything entered.  But it is interesting that a beginner can do this well after 11 days. 

Level Two Spanish Results: First Picture Description

Spanish 2 has been running for three weeks.  We have read a couple of easy novels, and have done one story cycle: el restaurante, which included reading, storyasking, Movietalk (Mr Bean videos RULE!) and Picturetalk.

Today’s first writing assessment: describe this photo.

  
Here are four writing samples.  The kids had 5 minutes, no notes or dictionaries.

First, Janelle, the top student. Amazing how she mixes past and present appropriately.

 

Next, we can compare two Level Two students who are not top performers. Hassan went to another school last year, where he had traditional grammar-and-textbook-based teaching.  This is garbled and nonsensical.  Hasan has some learning challenges and struggles in other classes.

Next, Abbas, who like Hassan has some challenges and struggles with school.  This is not awesome…but we understand, and he has built in a bit of a backstory.  Abbas had only TPRS in Level One.

Based on results, C.I. clearly helps the challenged kids more than does a traditional text.

   
Finally, Amneet.  This is not very good writing.  What is interesting here is that Amneet is probably the best speaker in the class.  I have found this kind of thing typical:  while most of the kids can undertsand everything (the scores for reading and listening quizzes are all between 80-100%), production skills vary dramatically from kid to kid and medium to medium.  Writers are not necessarily speakers, and vice-versa.

Amneet arrived late in Spanish 2, has missed a bunch of classes, but did well last year (over 80%) so I am expecting her written output will pick up.

 

T.P.R.S. or…whatever? More Evidence for the Effectiveness of Comprehensible Input

I have documented TPRS kids’ success in the past (see this) but today we are in for a different kind of treat: we are going to look first at what top students can do with traditional methods (forced output, grammar practice, word lists, memorisation, etc) and then with comprehensible input.

Today, totally by accident, I found my old Spanish 2 binder from when I was a traditional methods teacher using the ¡Juntos Dos! program.  One of my old Level 2 final projects was to create a children’s book.  The kids generally used themselves as characters.   This story was written by Nuvjit S.

Nuvjit was a keen language learner in high school, and has since then acquired Japanese. She was the top student in Spanish in her year.   For this project, the kids got editing help from me, they could use dictionaries, etc. Here is Nuvjit’s children’s book. This was the best project of its kind that I got that year.  So take a look at what I was able to get done with traditional methods.  This is second year Spanish.

  
  
  

Now, let’s take a look at what a kid taught with only comprehensible input methods can do.

This is Neha D.’s story. She is one of the top five or six students from this year.  This was done today, in 50 minutes, with no notes or dictionary.  First draft.  No editing.  Neha is Nuvjit, ten years later, with  Spanish teaching based on what we know the brain needs to acquire language: tons of compelling comprehensible input, in aural and written form.

Neha has never seen a grammar worksheet, a verb conjugation table or an explanation of how the pretérito  differs from the imperfecto.  She has never had her work corrected, and she has never “reflected on her learning,” or fiddled with a portfolio.  She probably can’t even tell you what a verb is and she has never heard the word “conjugate.”

This is first year Spanish.

  
  
  
  
  

So…it’s pretty obvious which method works better…for me, and for these students.  Your mileage may vary.

Now let me also be clear here:  I was a pretty bad communicative teacher.  I didn’t get good results (well, I couldn’t get my kids to have awesome results).  There were– and are– loads of people better than me in that tradition.  So I am pretty sure that any number of people could have gotten better results.  I’m also at best a slowly-improving T.P.R.S. practitioner, and there are loads of people who get better results than me.

This however is also my post’s silver lining:  if I was a bad “communicative” teacher and I’m a marginal (but improving) T.P.R.S. practitioner, my kids are getting more out of the class with T.P.R.S.

At bottom, I don’t attribute Neha’s success to me being smart or a good teacher, or to how funny I am– err, try to be– etc.  Neha and her classmates’ success ultimately stems from T.P.R.S., Movietalk, etc, allowing us to remain comprehensibly in the target language for huuuuuge amounts of time.

Spanish Results: Adriana Ramírez’s Level 1 Spanish #showumine

My colleague Adriana Ramírez (@veganadri) has published some beginner results.  Here are three of her kids’ writing samples from early October 2015.  This is semi-sheltered grammar (no past tense yet) from classic TPRS: storyasking and reading, with some MovieTalk. They had 30 mintes to write these.  Also note that these kids

• do not “practice” writing
• do not “practice” or “study” grammar
• are not forced into any kind of “communicative output” or “communicative pair” activities
• do not use notes or dictionaries when doing writing assessments.

These are very good results.  Check it!

What results does T.P.R.S. get? Amazing ones…and here’s the proof.

Do T.P.R.S., Movietalk, Look and Discuss, and other comprehensible input methods work?

Yes.  And not only do they work, they work much better than anything else out there.

What began as a friendly Twitter challenge– beat my beginner kids’ output using old-school methods or textbook, and I’ll take you crafty beer-drinking, hashtag #showumine– now has a bunch of T.P.R.S. teachers showing what their kids can do.

The rules are simple: show what your kids can do in writing (or speech) without dictionaries, rehearsal, Internet, notes or advance warning, with limited time and no preparation.  In other words, show what’s wired in, i.e. acquired, and not “learned.”

The proof, as they say, is in the pudding.  So, without any further ado, here are results.  This entry, constantly updated, provides links to various teachers’ kids’ written and oral output.

We need more French samples, and all other languages are welcome.  Know something that needs adding?  Lemme know and I’ll add it.

SPANISH  

Eric Herman‘s oral assessment of beginners is here.  Eric notes that “these are unfamiliar tasks and functions, but I challenge non-c.i. teachers to give the same test and get the same results.”

Chris Stolz has Spring semester 2015 beginner writing samples from 7 weeks in, 8 weeks in, stories from 8 weeks in and 11 weeks in.  This post compares two top students– one taught with legacy methods, one with C.I.

Grant Boulanger has 8th graders doing oral output here.  Here is one of Grant’s beginners– using three verb tenses and other so-called “advanced” grammar– to retell a story.  Grant also showcases his 8th graders (Level 1 Spanish) doing an impromptu story retell here.

Mike Coxon‘s kids are recorded here.

Mike Peto has some writing samples here.

Crsytal Barragan here shows first-day-back-to-school writing samples. Here, the student who was taught with T.P.R.S. writes rings around the student from the legacy-methods class.

Adriana Ramírez’ Level 1 Spanish results are here.

Jim Tripp has some Level 2 examples (with discussion) here.

Darcy Pippins’ AP results are here.  

LATIN

Magister Lance Piantaggini shows what beginner kids can do in Latin.

CHINESE

Terry Waltz‘s site has writing samples plus oral stuff.  Her kids can throw down with characters. Check it.

Hai Yun Lu has a level 1 Mandarin student storytelling here.

GERMAN

Brigitte Kahn‘s kids do 5-min speedwrites here.

FRENCH

Bess Hayles shows first day back from vacation writing samples here.

A traditionalist and Kim A. (comprehensible input) here have writing samples.  The reader can decide if the Level 2 (traditional) or Level 1 (C.I.) Kim A vs Traditojnalist exemplars.