Assessment and Evaluation

C.I.-taught Students Evaluated by A.C.T.F.L. Writing Standards

How well do C.I.-taught students do in terms of ACTFL writing standards? Well…pretty darned well, I’d say.

Inspired by a Facebook post, I thought I would measure some of my Spanish 1 students’ writing on the ACTFL scale.

Here is their criteria for Novice High

Writers at the Novice High sublevel are able to meet limited basic practical writing needs using lists, short messages, postcards, and simple notes. They are able to express themselves within the context in which the language was learned, relying mainly on practiced material. Their writing is focused on common elements of daily life. Novice High writers are able to recombine learned vocabulary and structures to create simple sentences on very familiar topics, but are not able to sustain sentence-level writing all the time. Due to inadequate vocabulary and/or grammar, writing at this level may only partially communicate the intentions of the writer. Novice High writing is often comprehensible to natives used to the writing of non-natives, but gaps in comprehension may occur.

Here are some writing samples.  This is Bani’s work, after about 60 hours of C.I. (I do mostly TPRS, along with Movietalk, Picturetalk and some Slavic-style Invisible “untargeted” stories.)


Let’s see…Bani uses a load of sentences (actually, she uses only sentences). She fully communicates her intentions. There are no gaps in comprehension, The writing is far beyond the “lists, short messages, postcards, and simple notes” that ACTFL says Novice High writers can produce.  So, where is Bani?

Considering her use of various verb tenses, clarity etc, I would say somewhere between Intermediate Mid and Intermediate Advanced. What do you think?

Next, we have Marcus. This kid has an IEP, and has missed about two weeks (~13 hrs) of class.  He has some behaviour challenges, some of which involve staying focused in class.  Here is his most recent story:



 This is obviously not even close in quantity or quality to Bani’s. He uses English, has some problems with basic verbs, is occasionally incomprehensible, and the story does not really flow.

So, where does this fit on the ACTFL scale? Well, here is their Novice Mid descriptor set:

Writers at the Novice Mid sublevel can reproduce from memory a modest number of words and phrases in context. They can supply limited information on simple forms and documents, and other basic biographical information, such as names, numbers, and nationality. Novice Mid writers exhibit a high degree of accuracy when writing on well-practiced, familiar topics using limited formulaic language. With less familiar topics, there is a marked decrease in accuracy. Errors in spelling or in the representation of symbols may be frequent. There is little evidence of functional writing skills. At this level, the writing may be difficult to understand even by those accustomed to non-native writers.

Marcus fits most of this.  However, he does use sentences, sometimes properly. So– at about 50 hrs of C.I., plus behaviour and learning challenges– he’s at Novice Mid.

The lessons?

  1. C.I. works very well indeed, even for students who are not especially motivated or focused, or who have attendance issues. One of many key C.I. plusses: the vocabulary is constantly recycled in comprehensible but new ways.
  2. C.I. does get the “grammar teaching” done, despite traditionalist “those TPRS kids don’t know grammar” complaints. As we have all experienced, the stereotypically successful  language-class kids– wealthier, whiter  and fairly L1-literate females– will pick up and memorise whatever grammar rules etc we throw at them. The rest, not so much. Bani can’t tell you what a verb is, or conjugate one in a chart, or explain the difference between preterite and imperfect verb tenses…but she can use them correctly and meaningfully. Grammar: my kids havn’t been taught it…but they got it.
  3. C.I. is going to reach kids who would be dead in the water with a textbook. I have had loads of kids like Marcus over the years.  Most of them failed with the text.  Worse, most were disengaged.  Now, I’m not much of a teacher…so if *I* can get Markus this far, anyone can do well!
  4. Anyone who has issues with department members who complain that eg “when I get your TPRS kids in Spanish 2, they can’t write out all the numbers from 555 to 579,” or “they can’t conjugate the verb traer in the pluperfect ablative subjunctive causal declension” can just point at ACTFL guidelines to show where their students are. Verb charts, memorised grammar rules, etc, are not part of ACTFL’s proficiency scales: the ability to write in contextually clear and meaningful ways is.
  5. ACTFL broadly suggests that in a regular (ie non-Immersion) classroom, students will need about two years to get to Novice High, another two for Intermediate High, and two more to Advanced. These writing samples suggest that we can go waaaaay faster than ACTFL thinks.

One last thing:  these kids do well not because Mr Stolz is a brilliant teacher, but because C.I. methods allow us to stay in the target language much more than the textbook does.







The Curriculum That Actually Works For Everyone

Longtime C.I. badass Mike Peto recently weighed in on a good question. Peto, who led his entire department from textbook servitude to effective C.I. (all their kids who write A.P. and I.B. exams pass), addressed this:

We are getting rid of the textbook.  What should we teach? What are people’s favorite stories and themes?

This is a good question.  Standard language teaching is organised around cunningly-disguised grammar units: we learn the present tense verbs with school vocabulary, the conditional a year later with shopping, etc.

My answer: agree as a department to use 250 high-frequency words per year.  Peto’s is better:  focus on “the Super Seven verbs” (Terry Waltz’s list of highest-frequency verbs), along with nine more that add up to what Peto calls “the Sweet Sixteen verbs.”


Here is Peto’s explanation of how and why his language department organised their language program the way they did.

Short answer: 16 verbs & a commitment to fill every class with compelling CI.

Long answer: As a department chair trying to design a common experience for students in different classes, with a half dozen different teachers on staff, I could have insisted that every teacher follow the same collection of story scripts, movie talks, and novels. That is, “all Spanish 1 students will read X novel and discuss Z movie talk. All Spanish 2 students will acquire this list of target structures so that they will be “ready” for Spanish 3.”

The problem Peto correctly notes here is that we are not guaranteed acquisition of “rule”/word ______ on any teacher’s schedule. Just because you taught it doesn’t mean they got it.

 On the other hand, the Sweet 16 verbs represent a different path towards creating a common experience between classes. We do not simply repeat sixteen words for four years, but we do agree that structures with these verbs are the ones that are recycled and given priority at every step in the journey. The Sweet 16 gives a department the flexibility to allow their teachers and students to pursue different interests in class, to use different language, but guarantees that there will be a common communicative foundation throughout the entire program. For example, the Sweet 16 verbs allow one teacher to develop an independent reading program for her students in which students are all reading different books (and thus developing their own idiosyncratic vocabularies), while another teacher develops his CI skills guiding his students through an authentic telenovela, while another teacher spends a good part of her classes discussing student illustrations and spinning stories from them.

This is how it’s done: you make enough horizontal links– the sweet 16– to ensure that kids in different classes can end up a year later with the same teacher and be able to function.  And you leave it open enough that what kids (and teachers) find interesting can be explored.

We collect everything: novels, Martina Bex units, TPRS textbooks by Carol Gaab as well as the ones by Blaine Ray, our own stories, huge collections of movie talks, telenovelas. We invest in PD like some departments buy workbooks; we brought Ben Slavic to our department to show everyone how to make One Word Images and next summer they are all going to Boston to refresh their skills on classic TPRS techniques. When someone starts using a new item (book, movie, whatever) then that person has the right to claim it for a particular level. Teachers can opt in to use it in their classroom, but nobody is obliged to use any particular item. Therefore, I claimed El Internado for level 1 and Gran Hotel for level 3. Any Spanish 1 teacher could use El Internado. Someone else claimed Fiesta Fatal as a level 1 book… I then included it in my FVR library while my colleague taught it as a whole class novel. A colleague claimed La Vampirata as a Spanish 2 whole class novel, so I did not include that in my FVR library (which is potentially read by Spanish 1 students).

We recognized that in any classroom there will be many different interests, and that when students and teachers are following their own interests then the input is likely to be more compelling. More compelling leads to faster acquisition. That is the funny thing about those studies which try to count how many times a student needs to hear a word to fully acquire it… teachers know that swears might be fully acquired the very first time they are understood, whereas an abstract transition word that the student never uses in their own L1 could be uttered comprehensibly 500 times and not be fully acquired. 

There is another major advantage to running a department this way. When any of my teachers get students at the beginning of the year, we do not have a list of target structures in our minds that we assume our students have acquired. We do not get angry if our level 3 kids do not understand X phrase; instead we are trained to start the conversation assuming nothing and paying close attention to their eyes.

Or, as Blaine Ray so perfectly puts it, “figure out what they don’t know, and start there.

At all levels, as we think about how to phrase our language so that it will be comprehensible, we all return to the Sweet 16 verbs and posters. It is a common experience in all classes, even though I spend a week talking about whales and my colleague spends weeks talking about football (what would you expect from a football coach!).

This illustrates exactly what high-frequency vocabulary does: it provides the flexible foundation onto which people can “stack” whatever vocabulary interests them. The football coach can play around with “el rodilla de Tom Brady se vuelve mejor” while in another class, students can create an OWI who “se vuelve loco por no haber comido.”

Terry Waltz has also famously made this point. She says, imagine you are learning Mandarin Chinese and you are going shopping, and you want to buy a wrench and some apples. If you had to pick one phrase which mattered most, what would it be?  That’s right: want to buy.  This works everywhere.  And ultra-low frequency words (hammer, apples, most clothing words, etc etc) you can figure out on the spot, or gesture, or look up, etc. Waltz elsewhere makes a similar point in response to a teacher who asks shoudl they not know food vocabulary? Waltz, who is translator-fluent in Mandarin (we are talking 10,000 characters plus real-time translation) says, despite having lived for years in Taiwan, that she doesn’t know most of the food words.  Is this a problem?  No, she says, most of the time when I (and Taiwanese) who don’t know the name of something we want to eat just say “I’ll have that” and point. Brilliant. The same is true in, say, Mexico: every region has basic staples– frijoles, tortillas, cerveza– but the local dishes are ultra-low frequency so you just ask about it when you are there. There is no point in teaching Spanish students chapulines, tlayudas, de lechon, etc.

This is also necessary because students move into our district at every level, and we cannot just leave them behind because they did not start with us. We need to provide a comprehensible experience at all levels, even if students missed the first 3 years of our CI  CI program because they were learning thematic vocab in another district.

This is brilliant:  C.I. accommodates everyone, while the textbook does not. The kid from Greg the Grammarian’s class will show up in Peto’s room and go, OMG wut? and probably spend quite a while simply listening.  However, whether or not he has spent four years or four weeks in Grammar Grinder Greg’s class, he will be able to catch up in Peto’s class.

Peto leaves out only one thing: when kids leave his school for Grammarian Greg’s class, high-frequency words will serve more of them much better than anything else.  A teacher (or department) who obsesses about themes and topics is rolling the dice on whether their alumni will end up in a class that uses the same vocabulary. Those who teach high-frequency vocabulary are preparing students for anything.



Don’t Do This

One C.I.-using American colleague recently shared this section from a Spanish test which their defartment head gave their Spanish class, viz

idiot task

How dumb is this?  Let us count the ways:

  1. Unclear instructions.  Are we supposed to rearrange the words in the sentences, or the sentences themselves, or both?
  2. Some of these have more than one possible answer (way to rearrange words).  Eg c. could be vivir juntos no es fácil or no vivir juntos es fácil.
  3. What does this have to do with actual Spanish that people actually speak or write?  Nothing.
  4. I have never seen a language curriculum that says students will be able to take scrambled words and turn them into sentences.
  5. I’m not sure what they are assessing here.  It’s not comprehension of actual Spanish, since nobody speaks or writes like that.  It’s not output, since students aren’t generating language.


This reminds me of those high-school math problems that felt like this:  Suzie is twice as old as Baninder.  When Baninder is twice as old as John, John will be three times as old as Suzie.  How old will Suzie’s dog be on Thursday when Baninder is four? 😉

This is basically a gotcha! question for the grammar geeks.  Yes, you could figure it out, but why bother?


Why I (Almost) Never Assess Speaking

So this was asked on a forum recently and, as usual, it got me thinking.

This is a question about “El Internado,” but, really, it applies to anything we do in a language class.  We read/ask a story/do a Movietalk or Picturetalk, etc, and then we want to assess speaking, comprehension, etc.

My response to this question is don’t bother assessing speaking.

But first, a qualifier:  if our Board/school/dept. etc says we absolutely MUST assess speaking, well, then, go for it.  We do what we have to do to keep our job.  But if we don’t have to assess speaking, don’t.  Here is why.

  1. The info we gain from this cannot generally guide instruction, which is the point of any assessment (other than at the very end of the course).  The reason for this is very simple: what will we do if what we learn from assessment varies wildly (which it almost certainly will)? If Samba has problems with the pretérito verb tense, Max doesn’t understand questions with pronouns, and Sky can fluidly ask and answer anything, how are we going to design future instruction around that info?  How are we going to “customise”  reading/stories, etc to give 30 different kids the input they need?  Answer:  we can’t.
  2. This takes forever.  If we have 30 kids in our class, and we can assess them in three minutes each (which is tough) we are spending 90 min alone on speech assessment.  That’s a period and a half!  During this time, we have to design something else for them to do…and good luck having 29 kids– whose teacher is “distracted” by sitting in the corner assessing speech– staying on task for 60 minutes.
  3. We already know how well they speak.  If we are doing regular PQA– personalised questions and answers (basically, asking the class members the same questions we are asking the actors)– we know exactly how well each kid can talk.  So why waste time with a formal assessment?  In my Spanish 1 right now, Ronnie can only do y/n answers to questions, while Emma Watson (aka Kauthr) speaks fluid sentences, and so does Riya, while Sadhna mixes up present and past tense in her output (but understands tense differences in questions) etc.
    Indeed, this is where feedback to the teacher is useful. If—in the PQA moment—I see that Sadhna mixes up past and present in answers, I can guide PQA around that right then and there.
  4. In terms of bang-for-buck, we are going to get way more results from more input than from assessing speech.  We acquire language not by practising talking etc, but by processing input, as Bill VanPatten endlessly reminds us.  I used to do regular “speaking tests” and they did nothing and the info was useless.  Now, I never test speaking until the end of the course, and the kids speak better, mostly because the wasted time now goes into input.
  5. A question that comes up here, regarding assessing speech post-Internado, is, what are we testing the kids on?  Are they expected to remember content— names, events, “facts” etc– from the show?  Or are we assessing speech generally?  In my opinion, “content” should be off-limits: we are building language ability, not recall.In terms of language ability, one of the problems with assessing right after specific content (eg some of El Internado) is that, since this input is generally not very targeted, we don’t have much of a guarantee that the kids are getting enough exposure (in a period or two) to “master” or acquire anything new.  This is to say, while an episode may be 90- or even 100% comprehensible, thanks to the teacher’s guidance etc, it almost does not focus on a specific vocab set.  In a classic T.P.R.S. story, the teacher makes sure to restrict (shelter) vocab used in order to maximise the number of times each word/phrase/etc is used.

    This is whether s/he has a plan, or, as in totally “untargeted” story creation à la Ben Slavic, the kids are totally driving the bus.  As a result, the odds of the kids picking up specific “stuff” from the story—in the short term, which is the focus of the question– are greater (and greater still if the asked story is followed by reading, Movietalk and Picturetalk) than if the input is familiar but untargeted.

  6. What about the kid who missed some of (in this case) El Internado? If the speaking assessment focuses on Internado-specific vocab, it would (in my opinion) be unfair to ask Johnny who was there for all three periods and Maninder, who missed two of three periods, to do the same thing with the “language content” of the episodes.
  7.  Kids hate speaking and tests.  Anything I can do to avoid tests, or putting people on the spot– which a one-on-one test does– I do.  This is what Johnny looks like when you tell him, speaking test tomorrow:Image result for kid being interviewed by teacher
    (image:  Youtube)
  8. “Authentic content” eg El Internado has lots of low-frequency vocabulary. Sure, the teacher can keep things comprehensible, but there is inevitably kids’ mental bandwidth going into processing low-freq vocab…which is exactly what kids don’t need in a speaking assessment, where you want high-freq vocabulary that is easy to recall and applicable to lots of topics.

Anyway…this is why I save speaking assessment until the end of the course: I know how well my kids can speak, I can adjust aural input where it matters– right now–, I don’t want assessment to detract from input, and speaking assessment doesn’t really help me or my kids.




Should I Mark Behavior? The Great JGR Debate, and a Silver Lining for Behaviour Rubrics.

Some years ago, a teacher built a rubric for her C.I. classes, which Ben Slavic named JGR and which was discussed on his blog and then elsewhere.  Here is a version I have played around with: INTERPERSONAL COMMUNICATION rubric.  I initially opposed the use of JGR, then used it, then ditched it, and now I use it (but not for marks). Note: this is a modified version of the original JGR; and I don’t know for how long she used her rubric, or if she still does, or what the original looked like.

JGR was developed because– like all of us, especially me– the creator had some challenges managing her C.I. classes in her initial year with T.P.R.S., which can in (especially my) rookie hands turn into a “woo-hoo no more textbook!” clown show.  JGR basically “marks” classroom behaviour.  JGR specifies that students make eye contact, add story details, ask for help, not blurt, not use cell-phones etc.  Jen used it (and if memory serves Ben also recommended its use) by making part of her class mark a function of behaviour as marked by JGR.  So the kids might get, say, 20% of their mark each for reading, writing, listening, speaking and 20% for their in-class behaviour.  Part of the thinking here was that some behaviours lead to acquisition, while others do not and also wreck the classroom environment, and so “acquisition-rewarding” behaviour should be rewarded.

JGR– for many people, including me– “works.”  Which is why– especially when linked with allegedly “acquisition-promoting” behaviours– lots of people are interested in it.

JGR is a kind of “carrot-and-stick” marking tool:  if the kids engaged in the behaviours JGR specified, their marks went up, partly because (a) they got marks for those behaviours, and partly because (b) the behaviours should– in theory– help them acquire more language.

This can of worms was shaken around a bit on Ben’s blog, and recently, thanks to the always-remarkable Terry Waltz, there have been FB and Yahoo discussions about it.  So, today’s question:

Should we assess in-class behaviour for final marks purposes?

My answer: no, never.  Why?

1. Behaviours typically asked for in JGR– or other such rubrics– are not part of any     curricula of which I am aware.  Every language curriculum says something like, students of the Blablabian language will read, write, speak and understand spoken Blablabian, and maybe say something about Blablabian culture.  Nowhere does any  curriculum say “students should suggest details for stories” or “students will lookthe teacher in the eye.”

If it’s going to get a mark, it has to be part of course outcomes.  Any assessment guru (Wormelli, Harlen, etc) will tell you the same thing: we do not mark attitude, behaviour, homework, etc, as these are not part of final outcomes.

To put it another way, how do we judge the New England Patriots football team?  By how well, often and/or enthusiastically they practice and look Bill Belichick in the eye, or by how many games they win?  How should Tom Brady be paid: by how often he shows up for practice, and how nice he is to Belichick, or by how many yards he successfully throws?  That’s right.

We could– and I often do– end up in situations where a “bad” kid does well, or a “good” kid does poorly.  I have had bright-eyed, bushy-tailed teacher’s pet-type kids who were not especially good at Spanish, and I have had giant pains-in-the-butt who were quite good.

My best-ever student in TPRS, Hamid Hamid, never added story details, never looked up, and always faced away from the board.  Yet he CRUSHED on assessments and got 100% in Spanish 2.  Two years later, his younger brother, Fahim (also a great student) told me that Hamid Hamid was both shy and deaf in his left ear, so always “pointed” his right ear at the board (and so appeared to be looking away).  This kid’s mark would have been lowered by assessing his “in-class behaviour,” which– given his epic Spanish skills– would have been absurd.

2. As Terry Waltz points out, neurodivergent kids can– and do– acquire language without engaging in many behaviours typically required by participation and behaviour rubrics. She also points out that forcing neurodivergent kids into the “normal” mold is at best less than productive. If you are autistic, anxious, suffering from PTSD (as my stepdaughter does) or facing any other neuro challenges, “engagement” rubrics can make your life miserable while not appreciably meaningfully measuring what you can do with the language.

3. The only thing required for language acquisition is reception of comprehensible input.  While the focus of behaviour rubrics is designed to get kids to tune in, it does not follow that many behaviours which do make for a good class– e.g. people adding good details to stories, looking at each other– are necessary to acquire language.

All of us have been there: you have a plan, you did your story warmup or whatever, but the kids aren’t into it.  You bust out a Movietalk but they aren’t into that either.  Dead class. Now, in a C.I. class, we don’t have recourse to worksheets or whatever, and we still have to teach the language. I have a bail-out move here: direct translation, and I always have a novel on the go, so I can read aloud, and Q&A the novel.  If I’m being particularly non-compelling, I’ll throw an exit quiz at them.

The point: if the kids are getting C.I., they are acquiring.  If they are miserable/tired/bored with stories, fine.  They are gonna get C.I. one way or another.

4. Any kind of behaviour rubric plays the awful “rewards” game.  Ask yourself this question:  why do I teach? The answer– other than because I have to make a living— is probably something like, because it’s interesting, I have some measure of control over my work, and I love kids and my subject.  Some will add that teaching, properly done, opens doors for kids.  Teachers do not teach because they want to be evaluated, or because they want to use the latest gizmo, or because they want public praise, etc.  They are, in other words, intrinsically motivated.  They want to work because the work is good and worthy in itself.

When we institute rewards for behaviours, as Alfie Kohn has spent a career arguing, we destroy intrinsic motivation.  We turn something interesting into payment for marks.  The point stops being paying attention to the story– or adding to it cos you actually care about it– and becomes something rote.

5. Using behaviour rubrics can dampen professional self-examination. If my practice is such that I have to use marks as a stick to keep kids in line (the policing metaphor is not an accident), there are two possibilities: tough kids, and/or I am doing a bad job.  The question why are they not tuned in? might be answerable with any of the following:

— I am not being sufficiently comprehensible

— I am ignoring the top or the bottom end of the class– too fast/slow or simple/complex

— my activities are not interesting, varied or meaningful enough

— the kids see no purpose

— accountability: they don’t see tuning in as something that results in real gains

— I lack basic skills (smart circling, control of vocab, etc etc)

— my story sucks 😉

I had better be able to look in the mirror, consider and then deal with these possibilities, rather than merely acting like a cop and demanding obedience.

Now, behaviour does matter.  You cannot run a T.P.R.S. class without rules etc.  My basic rules:

  • no phones or other distractions (including side-talk, blurting etc)
  • no insults of anyone other than oneself or of rich entitled people
  • listen, watch and read with the intent to understand; ask when you don’t
  • do not create or engage in distractions

The tools that we have for dealing with distracting behaviour include

  • warning the offender, standing by their desk, calling Mom and Dad, etc
  • pointing, with a smile, to classroom rules every time there is a problem
  • sending them to admin if necessary
  • taking their phone until 3:15 (most kids would rather die)
  • detention, where we discuss behaviour
  • assigning read & translate (quiet seatwork)
  • taking the kids outside for a walk, or doing some other kind of physical brain-break
  • changing activities
  • doing a quiz
  • talking to kids one on one and asking what do I need to do to get you focused?


The upshot?  We should not, and need not, mark “behaviour” or “participation.”


Addendum:  is there ever a use for classroom behaviour rubrics?

Yes.  I get my kids to self-evaluate using JGR every 2-3 weeks.  My version generates a mark out of 20.

Nineteen out of twenty kids will very honestly self-evaluate their behaviour, provided they understand exactly what is expected.  One kid in twenty will heap false praise on him/herself.  For the false praisers (“I never blurt in class!”), I sit them down and explain what I think, then we agree on a more realistic mark.

I save these JGR “marks” and once in a blue moon, when a helicopter parent or an Admin wants to know, how is Baninder doing in Spanish, I can point to both the spreadsheet with Numberz and JGR.  This frames the inevitable discussion about marks in terms any parent can understand.  Any parent, from any culture, understands that if Johnny screws around and/or does not pay attention in class, his mark will drop.

JGR– in my experience– accurately “predicts” the marks of about 80% of kids.  When I can show a kid (or a parent or admin), look, here are Johnny’s marks AND Johnny’s own description of how he behaves in class, we can have an honest discussion about marks, Spanish, etc.  Win-win.

How Do I Assess and Evaluate Speaking?

A teacher asked on the Facebook group “How do you assess speaking?”  Responses were basically, “try using one of various apps” (i.e. Google Classrooms, KaBlaBla, etc).  Lots of people want to use tech to do it.

Contrarian here:  save yourself time, and don’t bother…you can accurately, quickly and easily assess speaking with zero tech.

But let’s revisit the basics before we go on:

  • speaking does not improve language acquisition.  The act of talking is not like practicing music or baseball.  The real driver of speech is aural (and written) input.
  • Teachers need a life.  I for one refuse to spend an hour per class listening to students’ prepared recordings of prepared questions.  The kids have better things to do, and so do we.
  • The only speech we should assess if we want to see what the kids have acquired is spontaneous and in-the-moment.  If you want people to learn a language, then by all means let them plan, rehearse, etc…but don’t confuse this with acquisition, where we see what is “wired in” and gut-level, below– and beyond– the conscious mind.  Most of the ed apps I’ve seen are similar:  teacher records their voice asking question or saying prompts; kid listens, decodes and responds and records their answer for teacher to mark.  This kind of “planned” or “reflected-on” communication doesn’t really assess what they have acquired.
  • Feedback doesn’t work.  You can explain, correct, suggest, etc till the cows come home and it won’t make a difference in how well the kids speak.  Only input can really change that.

So how do I assess speaking?

First, every time a kid opens their mouth and uses the target language in class– to answer a question, to add to a story, etc– you are getting perfect feedback about how well they speak.

So with my 2s…Aashir can say– and understand– a word at a time max.  Simrowdy can answer any question and talk at length about anything.  Sadjad extemporaneously comes up with good entire sentences when adding to a story.  Janelle is like Simrowdy.  Daniel will– and does– say anything but has some verb etc issues.  Kevin never talks, but when he does, it’s perfect.  I could go on.

Second, the point– to me– of assessing speaking (as with anything else) in class is to see what the kids do not understand and where they need more input.  This is why we track barometer kids and choral responses.

Third, I don’t play “gotcha.”  I test what I teach.  I use vocab they know, and when using objects, pictures or people, I make sure the kids have the vocab to describe them.

Fourth, I don’t assess speaking for Level One students.  It makes them anxious, and it is time taken away from input.  I assess– i.e. attach a number to– kids once, at the end of Level Two.  I do only what you would do speaking in real life:

  • ask them questions and have them answer
  • have them ask me questions (and I answer).
  • describe something– a photo, an object, another kid in the class

No presentations, storytelling, memorisation, etc.

Here’s my rubric:

For a mark of 3:

  • I can in detail discuss myself, my social and family circle and my activities and interests, and i can describe things.
  • I make minor mistakes that do not affect meaning, and I can speak fluidly.
  • I understand all questions and I come up with my own.
  • I can fix conversational problems or I don’t have any conversational problems.

For a mark of 2

  • I can discuss myself, my social and family circle and my activities and interests, and I can describe things.  There are some gaps in what I can say, and sometimes I provide little detail.
  • I make enough mistakes that meaning occasionallyu breaks down, and I can speak but not quickly nd fluently
  • I understand most questions and I come up with some of my own.
  • I sometimes fix conversational problems.

For a mark of 1:

  • I can  discuss myself, my social and family circle and my activities and interests, and i can describe things, but I can’t do so with much or any relevant detail.
  • My mistakes affect meaning, and I generally don’t use sentences.
  • I don’t understand all questions and I have trouble coming up with my own.
  • I either don’t know when there are conversational problems, or I don’t bother fixing them.

“Conversational problems” means not understanding, and “fixing them” means asking for a repeat, etc (i.e. not just bobble-heading along).

This rubric will generate numbers from four to twelve out of twelve.    I’ll generally show them a pic on the iPad at some point, and have them describe that, or have them describe a kid in class.  it takes about 4 minutes per kid to do this.

If the kid bombs, they can come back in after school and re-do it.

Anyway, there’s my thoughts.

How Do I Do Dictation?

Dictation is old as the language-teaching hills.  I remember doing this in my various français classes and also in Cherman viss Frau Satcher, ja.

Now, dictée is like running, weightlifting or learning music: if you do it wrong, the results are frustrating, painful or non-existent.  Like everything else in language teaching that we still do from 60 years ago, dictado has been tweaked.


  1. teacher reads 5-10 inter-related and meaningful sentences aloud
  2. these sentences should use the most-recently-taught vocab
  3. students write down what they hear.
  4. teacher can repeat each 2-3 x
  5. Ben Slavic suggests then projecting the sentences on the overhead and having kids correct their spelling.
  6. I always finish with translate into L1.


What Diktation should not be:

  • a way of introducing new vocabulary and/or grammar
  • a way for students to “practise” spelling
  • isolated meaningless sentences (e.g. sentence #1 is “the boy is tall” and #2 is “it is raining”)
  • to any degree incomprehensible
  • graded for spelling to any significant extent

I use dictation as assessment, not as a vocab-acquiring activity, and I follow the 80/80 rule: 80% of class needs to get 80% or more for me to move on. Scores are usually quite high, as I do dictation after asking a story and doing readings of the story, or other stories using the same vocab.  The kids don’t complain, it is a zero-prep activity, and it is 90% C.I. as the kids know the vocab.  This is mostly how I get listening marks.  Although it looks like output, mostly it’s input– listening– and the output is in L1.  As James Hosler says, dictation while being assessment is another way to deliver C.I.

Here is what I am going to do for dictation on Monday in Spanish 2.

  1. There was a grandmother who wanted to give her grandson money.
  2. She told him “you must win a a dance contest”
  3. He said “I don’t like to dance, because I am lazy.”
  4. The grandmother wanted to give her money to Donald Trump.
  5. Donald Trump did not want her money.

You could also

  • project a picture and describe it
  • describe a character from your novel or story
  • project bits of a clip from a video, narrate a sentence, and have them write that

Level Two Spanish Results: First Picture Description

Spanish 2 has been running for three weeks.  We have read a couple of easy novels, and have done one story cycle: el restaurante, which included reading, storyasking, Movietalk (Mr Bean videos RULE!) and Picturetalk.

Today’s first writing assessment: describe this photo.

Here are four writing samples.  The kids had 5 minutes, no notes or dictionaries.

First, Janelle, the top student. Amazing how she mixes past and present appropriately.


Next, we can compare two Level Two students who are not top performers. Hassan went to another school last year, where he had traditional grammar-and-textbook-based teaching.  This is garbled and nonsensical.  Hasan has some learning challenges and struggles in other classes.

Next, Abbas, who like Hassan has some challenges and struggles with school.  This is not awesome…but we understand, and he has built in a bit of a backstory.  Abbas had only TPRS in Level One.

Based on results, C.I. clearly helps the challenged kids more than does a traditional text.

Finally, Amneet.  This is not very good writing.  What is interesting here is that Amneet is probably the best speaker in the class.  I have found this kind of thing typical:  while most of the kids can undertsand everything (the scores for reading and listening quizzes are all between 80-100%), production skills vary dramatically from kid to kid and medium to medium.  Writers are not necessarily speakers, and vice-versa.

Amneet arrived late in Spanish 2, has missed a bunch of classes, but did well last year (over 80%) so I am expecting her written output will pick up.


Baby Steps: B.C.’s Proposed French Curriculum

The B.C. Ministry of Education is busily rewriting curricula, including French. Brief summary: the curriculum has added a few things which research and classroom experience have shown us are part of best practices, retains recommendations which are simply not supported by research, and raises a bunch of good questions. I looked at the French 8-12 curriculum.  Here are my notes in five sections:

  • the good
  • the “oughta-have-been-chucked-but-wasn’t”
  • the bad
  • challenges
  • the inexplicably missing
  • recommendations

If you have not yet familiarised yourself with current research about second language

1.  The Good Stuff with Practical and Empirical Support

  • The curriculum finally recognises that storytelling is probably the single-easiest way to soak up a bunch of vocab and grammar, and so asks that students master basic storytelling.  They are expecting a fifth-year (e.g. French 12 student) to be able to tell/retell a complex story.  (Note: any teacher who uses T.P.R.S. or A.I.M. will see their students master this in their first year of language class.)
  • Students should now acquire fewer verbs, but more verb tenses (and other language features), at the same time, which aligns with research (again, T.P.R.S. and A.I.M. practitioners are miles ahead here).  This is called “using unsheltered grammar.” Ideally, the “non-Englishy” parts of French– e.g. the imparfait and passé-composé, the subjunctive, pronoun orders– should be introduced immediately so that people have a lot of exposure to them and can pick them up when their brains are ready. As VanPatten & Wong (2003) note, “acquisition of the tense and aspectual systems (e.g. the use of the preterit/passé composé and the imperfect) is piecemeal and unaffected by instructional intervention.”
  • The curriculum also suggests modifying authentic documents for learner needs.  This is the opposite of the standard bad advice (which is “modify the task, not the text”).  It also suggests increasing repetition (and saliency) of vocab to be acquired.  Great work, Ministry– these are proven strategies.
  • Emphasis is placed on meaningful communication, and not drills etc.


2.  Problematic Legacy Method Recommendations

a.  The curriculum supports “[T]he use of authentic documents and tasks to support the development of communication skills.”  Bad idea.  Why? Well, authentic documents– things by and for native speakers– tend to have the following characteristics:

  • they use low-frequency vocabulary and often slang
  • they often use complex language (e.g. metaphors, irony, etc)

If we really want to build proficiency, the last thing we should be using are “authentic documents.”  You want to use high-frequency (i.e. oft-used) vocab, you want it comprehensible, and you want it recycled zillions of times.  Teaching decoding strategies– the classic “answer” to the straw-man question of “how do we get learners deal with authentic documents?”– serves only to establish meaning and does little for acquisition.  But– fair enough– we’re told to modify where necessary.  And we should be modifying authentic French (or any other) texts most of the time.

“Authentic tasks” is another minefield.  This usually means doing “real world” stuff like learning to give/follow directions, order food etc.  The problem with this, as Bill VanPatten and I have noted, is that you cannot “train” people for future language scenarios the way you can train a doctor to, say, give stitches.  One new word or phrase, and that carefully-practiced restaurant dialogue is useless. Better: teach a ton of comprehension so that people understand.  VanPatten has described language pedagogy as the development of “coping skills”: we want learners to be able to manage in the target language and culture, because we cannot train them for every eventuality.

b.  The “notion that acquiring French includes learning about Francophone culture.”  Really? I must learn about baguettes and Proust and poutine to learn French?  Sure, learning about French culture is a great idea, and it’s important.  And maybe two birds, one stone, etc.  But we  are best off doing French in French, and French– or whatever– culture mostly in L1, so that you can avoid making French culture banal and simplistic.  Of course, this is going to be up to teachers to decide how to do.  Can the Ministry provide any evidence that it is necessary to learn about Francophone culture to acquire French?


3.  The “still needs work” side

First, the curriculum says Grade 8s should knowvocabulary to describe elements of cultural communities, their practices, and their traditions.”  Why this is important is problematic.

Most “cultural” vocabulary is low-frequency (and therefore not very useful), and much of it is specifically tied to regions (e.g. nobody in France cares what a bonhomme is) and so not especially portable.  If we want  people to talk about their own culture etc, great…but that’s not super-French.

You have options when it comes to culture, and they are simple: you can develop acquisition, or understanding of culture, but not both meaningfully.  Why?  Because “cultural” vocab is low-frequency (so learning it will take time away from the useful– i.e. high-frequency vocab people actually need to communicate), and because the kind of cultural stuff you can talk about (esp in the first 3-4 years of a language) in the target language is simplistic and banal.

A classic example– which I’ve seen in every language program I’ve seen in North America– is, say, food. My district’s French text, Communi-quête, Level 3 (French 10), has a food unit.  Shopping for and ordering food, etc.  Necessary vocab:  commander (to order), and prendre (roughly, “to have something to eat/drink”).  According to Wikipedia, commander is not in the top 1000 most-used French words (the 1173rd most-used word is commandes) and neither is prendre. So, here French teachers are being asked by this textbook to teach “culture” vocab which is banal, and low-frequency. In other words, boring and useless.  This is the classic “culture problem” of language teaching: if you want to do the “culture” of L2, the only way to make it not trite and stupid is to do it in L1, and if you do it in L2, it will be silly and low-frequency.

Second, students should “[l]ocate and explore a variety of online media in French.” This is generally a waste of time.  Again, most French media will present

  • too much vocabulary
  • too much low frequency vocabulary
  • not nearly enough repeated vocabulary
  • French which is spoken too quickly to understand, etc, for kids to learn much from.  We know that in a second language (as audiologist Ray Hull notes) adolescents process at about 125-130 words per minute.  Adult native-speaker speech is around 170-180 wpm, de fact rendering most aural online media incomprehensible.

We know from research that the most “bang for the buck” in terms of acquiring useful language is lots of repetitions (160-200 times each) of high-frequency vocab (the 1200-1500 most-used words).  Recommending a variety of online media which is not specifically made for students is a bad idea.

Third, we have some edubabble that appears to want kids to use French way above their heads, and to “reflect on their learning,” neither of which are useful.  For example, in Core Competencies, we have this: Students “[c]ollaborate to plan, carry out, and review constructions and activities.  Students work together to accomplish goals, either face to face, or through digital media. Examples include planning a construction, inquiry or performance, solving a problem, conducting an inquiry, and working together on community projects.”

This should happen in French?  Really?  Good luck with that.  If not, great, makes sense.  I can see this maybe happening in an upper-level Immersion classroom.  Outside that?  Let me know how that works.  Let’s see how much French gets spoken by kids planning something in a group.

If my experience is a guide, zero is pretty close. Yesterday during my planning block, for example, a colleague did a French scavenger hunt.  The kids had to make a set of French directions (turn left, go straight, open…, etc) that took you through the school, and at the end of these directions there was some kind of object to be retrieved.  Groups of kids make one “hunt” and then “do” the hunt of another group.  BTW, I used to do exactly the same thing in my Spanish 2 class.  This is to “practice” giving and following directions and command forms of verbs. So, what actually happened? I dunno but when I used to do this with Spanish, this is what happened:

  • the kids spoke entirely in English
  • the Spanish was poor
  • the amount of (bad Spanish) input in a 30 min. activity was probably maybe 6 sentences of Spanish
  • the focus was on sprinting around the halls, checking phones, and hanging out with buddies, not Spanish

Now, this was a good “communicative” activity in the sense that kids were engaged and actually wanted to do it.  But the amount of good input was minimal, and both directions and school places (hall, stairs etc) are low-frequency vocab.  (But the French teachers  are more experienced than me, so they probably figured out a way to do this  activity better than I could.) I’ll bet that this is typical of peer-and-peer activities.  So why is the Ministry recommending these?

It also says students should “[e]xplain/recount and reflect on experiences and accomplishments, tell about their experiences—especially their learning experiences—and reflect, and share what they learned. Examples include presentations of learning, self-assessment, and receiving/offering feedback.” What useful self-assessment of language use a learner can make is beyond me.  I would love to see two things from the Ministry:  evidence that self-assessment makes any kind of difference in the language classroom, and some examples of meaningful self-assessment becoming acquisition (i.e. a “how to” and some data supporting this). The “I” statements here include

  • I give, receive, and act on feedback.
  • I can recount simple experiences and activities and tell something I learned.
  • I can represent my learning, and tell how it connects to my experiences and efforts

Well, the first is a total dud.  We know from research (and experience) that feedback about language does not transfer into the implicit system where language is processed and stored.  You can give feedback– do X, do not do Y, try Z– till the cows come home and the kids will still say “j’ai allée à l’ecole” or whatever (did I make that mistake correctly?).  Feedback other than “tell me more” or “please pay attention and ask for help” is useless, period.  If I’m misreading this, please Ministry clarify what you mean by “feedback.”

One also wonders what kind of feedback a learner of French is supposed to give another learner of French.  A really egg-headed kid might say “tell me more” or “explain; I don’t get it.”  Beyond that?  Curious to see.

“Recount activities and experiences”?  AWESOME!  STORIES! PERSONALISATION! DO IT!

“Representing learning”?  If they mean “describe all the language you learned this year”, total waste of time.  Also boring.  If they mean, do something cool– like tell and illustrate a story or personal anecdote, or use the language you learned to get something useful described or experienced or done– awesome.

Fourth, there are a lot of recommendations for “communicative” activities like this one which look really frikkin’ cool…until you try to wrap your head around

a) the amount of French needed to actually do this which would overwhelm anybody except senior Immersion kids.

b) the problem is that the activity is so cool that anybody would want to rush into it, and how can learners possibly know– and want to stay in— French to get the task done?  I used to do this kind of thing in Spanish…and if the activity was interesting enough, English inevitably got used.

c) the low-frequency vocab.  For example, a totally necessary phrase for this activity would be gilet de sauvetage (lifejacket).  According to Wiktionary’s frequency lists, this phrase is not in the top 10,000 most-used French words.  So…why teach it?

The biggest problem with any kind of “communicative” activity is that– even if kids wanted to use all the necessary vocab— they inevitably produce junky output which in turn becomes junky input for  other learners.  As researcher Terry Waltz has noted, “peer to peer communication is the McDonalds of language teaching,” a sentiment echoed by Stephen Krashen.

Bill VanPatten has noted that since U.G. develops mental representation of language only from input, and if anything in the input is “off,” mental representation suffers.  VanPatten has also stated that it is the teacher’s– and not the students’– responsibility to provide input in the language classroom. Ministry, can you please explain why you are recommending peer-to-peer communication when all the research suggests it provides very poor input?

Fifth, students should be able to “Identify examples of regional idiomatic expressions in texts.”  Why?  Idioms are by definition local-use and low-frequency.  So they have limited use.  Why bother?  Also, more generally, who cares if an expression is Quebecois or French or Malian?  How will this help students communicate?  Who in the world– other than a scholar– would care whether or not an expression was from France, Mali or Quebec?

Sixth, there are a lot of sample activities emphasising output and editing.  Output, as Krashen and VanPatten note, does not develop either mental representation of language or fluency (until we are at very advanced levels, and even there its role is dwarfed by input).  There should be much more emphasis on input-focused activities.  Editing is basically impossible for most younger people, especially second language learners.


First, if teachers should teach unsheltered grammar (e.g. all verb tenses at once) and storytelling, what are we going to use for textbooks and reading material?  There is not one published French textbook that foregrounds narratives and/or focuses on stories as both the content and method of teaching language.  Every French textbook program I’ve ever seen is poorly designed from the point of view of research about how people acquire languages.  All textbook programs

  • emphasise peer-to-peer communication, which is at best marginally effective
  • don’t use stories, which is boring
  • foreground grammar (even when pretending to be “communicative”), which is ineffective
  • are full of sequenced grammar instruction, which is contrary to research & useless
  • use too much vocabulary and not nearly enough reading (ditto)
  • come with expensive boring grammar cahiers, which are useless

In Spanish we have Cuéntame but in French?  Rien.  For reading, the Blaine Ray and Carol Gaab novels (most of which are available in French) are great.  What we don’t have: Canadian, French and other Francophonie-representative novels, comics, etc.  I’m a culture skeptic, but if there is one place where you can meaningfully “do” culture, it’s in novels. Ministry, can you please explain how you expect teachers to use stories when all textbooks we currently have totally ignore storytelling?  Are you going to provide some funding for books?

Second, we are going to need a massive overhaul of second-languages methods instruction at the University and District level.  Our new teacher graduates are going to need to learn story-based methods like T.P.R.S. and narrative paraphrase (Movietalk), or maybe A.I.M. for little kids.  Baby steps have been taken in this direction (e.g. I have been seeing S.F.U. student teachers and introducing them to T.P.R.S. for three years now; U.B.C.’s Wendy Carr has backed V.S.B. efforts to teach C.I. to teachers, etc) but we have a ways to go.  (How far?  Well, one S.F.U. languages methods prof is still recommending discredited legacy methods such as sequential grammar instruction, grammar practice, forced output for beginners, error correction, etc!  And we’re in the twenty-first century!)  At the District level, more helping teachers need to be trained in both research basics and research-based comprehensible input methods.  As Bill VanPatten notes, less than 1% of University language education teachers have any knowledge whatsoever of linguistics and second language acquisition research.  I have had lots of conversations with University people who do not know the basics of S.L.A. research. This to be fair is not the Ministry’s job, though– but it is a challenge.

Third, we are going to need to have a substantial conversation about assessment.  If we are going to upgrade methods, we need to ditch most of what our textbook programs want.  Asking kids to talk to one another in the target language, discrete grammar/vocab item testing, and testing by asking kids to listen to native-speaker-speed output are all not really representative of what kids can do.



At a Ministry inservice, the Ministry presenter on the new French curriculum said that they had consulted with languages teachers when writing the new curriculum.  Cool. Then they said four things I found astonishing:

  1. They said they were not, and had never been, a language teacher.
  2. They had not consulted with a single linguist– a second-languages acquisition researcher– for advice in redesigning the curriculum.
  3. They had not asked any students what they wanted, liked, disliked etc.
  4. They had not asked parents what they thought of French education.

The first, whatever, but that is pretty bad P.R. by the Ministry.

The second….really?  Frank Smith is at UVic, and Steve Krashen, Bill VanPatten, Wynne Wong and many others consult.  Why has the Ministry not consulted with any S.L.A. researchers about the new curriculum?

The third, not asking students…well, as Canadian Parents for French noted about Core French, “[t]he lack of satisfaction on the part of the core French students is reflected in high program dropout rates, low enrolments in the optional years, and a general feeling among anglophones that they “can’t learn French” (Netten and Germaine, 2012, P.87).   This is not because teachers don’t work hard or speak decent French, but because we use outdated methods.

You would think the minimum the Ministry would do would be to actually ask kids what they think of a program.  I’m a classroom teacher at a school where the admin and the language department head have decided to not allow students to take Spanish before Grade 10 (and where they must take French in Grade 8).  I get a lot of students who had been taking French (and other languages) either where I work or elsewhere.  I get some of the French “refugees,” and the Punjabi teachers get a bunch in Grade 11. Every year, I ask my beginners, why did you opt into Spanish? and I always get these three responses:

  1. ______ was confusing because they kept adding grammar rules
  2. ______ was boring because it was all memorising, grammar tests, etc.
  3. In _____ class, they made us talk though we couldn’t really talk, and that was stressful/hard.

We also know that while Canadian parents want their kids in French Immersion, there is never enough room.  There are well-documented social-class and prejudice issues at play here, and teacher supply (here’s an easy start to the discussion).  But the elephant in the room is, why do so many kids leave/not enter Core French?  The Ministry might have asked students (or parents) what they thought of Core French, and what they wanted to see…but they didn’t.  Hmm…



a)  Ditch the legacy-methods recommendations to use things like peer-to-peer communication and target-language group projects. For example, peer-to-peer communication is unnecessary, at best marginally effective, fake-feeling and often stressful for students; group work in most languages classrooms will mostly happen in L1.

b)  Get rid of the idea that “culture” should be taught only or primarily in the target language.  This idea is old, wrong and counterproductive.

c) Include brief, research-based pointers about what works and doesn’t to guide teachers.  No need to language-geek out here, just brief pointers with links to research.

d) Advocate for MUCH more free voluntary pleasure reading in the target language.  The evidence is overwhelming: free voluntary reading (in L1 or L2+) does more for language acquisition than anything else.

e) Ask students and parents what they want from a language program.  This could be done electronically, and it would be relatively simple.


Anyway, there’s my thoughts.  The Ministry would like your feedback, which you can provide here, or in the comments.

Direct translation: lame, not as boring as you’d think…and effective

This is a bail-out move when stories fall flat, or it’s the day before a test and you don’t have another video to Movietalk, and you are out of readings.  I think the idea originally came from Ben Slavic’s dictation suggestion.  I just thought, why not have them read and write instead of listening and writing?

I have a basic “script” version of each story.  It is the skeleton of what I ask.  Last year, I found that I was unable to keep the stories 100% comprehensible during asking, unless I circled everything sooooo muuuuch that the stories got boring.  (This year, I am better:  using multiple characters = waaaaay more reps).  So, one day when the kids and I were equally grumpy, I went to the photocopy room, made 30 copies of the basic script of the story, and got the kids to translate.  Surprisingly, the kids were quite happy to sit and work quietly.  My guess is that this is because a C.I. class has a lot of listening and talking, and reading & writing is a break from this.

The next day, when we did extended embedded readings, the kids seemed a lot more focused and I realised that the extra repetition– read and translate– had upped comprehension.  So this has become a regular move.

It’s very simple, it gives the teacher a break, it lets the kids slow down with reading, and it’s easy to mark.

a)  Hand out a printed version of the story.  You want 100% comprehensibility.

b) Get the kids to copy the story out in the target language.  They should leave TWO blank lines under each line of the story.

c) Under the T.L. writing– in different-coloured pen– have them translate into L1.

d) Under the L1 translation, leave a blank line.  This is to keep things looking neat.

e)  Keep going, and remember to indent paragraphs and dialogue, etc.

For marking, I’m a big believer in random sampling.    Pick three sentences at random, and see how accurate the translation is.  The kids get a mark out of three.  I am fairly strict with meaning on these, because they have vocab sheets and we have been through asked story and retells, and I am in the room where they can ask for help, etc.  I can mark a class set of these in under five minutes.

I usually give them 40 min with a story.  75% of kids can get it done in class; for the rest, it’s homework.  While this sounds boring, the kids are fine with it, I know I am getting them to really read, and it’s low-tech.  My inner rebel also likes it:  zero tech, no prep, non-“communicative,” super-high levels of comprehensible input, older than old-school, etc.  Above all, it works.