Ben Slavic

Why I (Almost) Never Assess Speaking

So this was asked on a forum recently and, as usual, it got me thinking.


This is a question about “El Internado,” but, really, it applies to anything we do in a language class.  We read/ask a story/do a Movietalk or Picturetalk, etc, and then we want to assess speaking, comprehension, etc.

My response to this question is don’t bother assessing speaking.

But first, a qualifier:  if our Board/school/dept. etc says we absolutely MUST assess speaking, well, then, go for it.  We do what we have to do to keep our job.  But if we don’t have to assess speaking, don’t.  Here is why.

  1. The info we gain from this cannot generally guide instruction, which is the point of any assessment (other than at the very end of the course).  The reason for this is very simple: what will we do if what we learn from assessment varies wildly (which it almost certainly will)? If Samba has problems with the pretérito verb tense, Max doesn’t understand questions with pronouns, and Sky can fluidly ask and answer anything, how are we going to design future instruction around that info?  How are we going to “customise”  reading/stories, etc to give 30 different kids the input they need?  Answer:  we can’t.
  2. This takes forever.  If we have 30 kids in our class, and we can assess them in three minutes each (which is tough) we are spending 90 min alone on speech assessment.  That’s a period and a half!  During this time, we have to design something else for them to do…and good luck having 29 kids– whose teacher is “distracted” by sitting in the corner assessing speech– staying on task for 60 minutes.
  3. We already know how well they speak.  If we are doing regular PQA– personalised questions and answers (basically, asking the class members the same questions we are asking the actors)– we know exactly how well each kid can talk.  So why waste time with a formal assessment?  In my Spanish 1 right now, Ronnie can only do y/n answers to questions, while Emma Watson (aka Kauthr) speaks fluid sentences, and so does Riya, while Sadhna mixes up present and past tense in her output (but understands tense differences in questions) etc.
    Indeed, this is where feedback to the teacher is useful. If—in the PQA moment—I see that Sadhna mixes up past and present in answers, I can guide PQA around that right then and there.
  4. In terms of bang-for-buck, we are going to get way more results from more input than from assessing speech.  We acquire language not by practising talking etc, but by processing input, as Bill VanPatten endlessly reminds us.  I used to do regular “speaking tests” and they did nothing and the info was useless.  Now, I never test speaking until the end of the course, and the kids speak better, mostly because the wasted time now goes into input.
  5. A question that comes up here, regarding assessing speech post-Internado, is, what are we testing the kids on?  Are they expected to remember content— names, events, “facts” etc– from the show?  Or are we assessing speech generally?  In my opinion, “content” should be off-limits: we are building language ability, not recall.In terms of language ability, one of the problems with assessing right after specific content (eg some of El Internado) is that, since this input is generally not very targeted, we don’t have much of a guarantee that the kids are getting enough exposure (in a period or two) to “master” or acquire anything new.  This is to say, while an episode may be 90- or even 100% comprehensible, thanks to the teacher’s guidance etc, it almost does not focus on a specific vocab set.  In a classic T.P.R.S. story, the teacher makes sure to restrict (shelter) vocab used in order to maximise the number of times each word/phrase/etc is used.

    This is whether s/he has a plan, or, as in totally “untargeted” story creation à la Ben Slavic, the kids are totally driving the bus.  As a result, the odds of the kids picking up specific “stuff” from the story—in the short term, which is the focus of the question– are greater (and greater still if the asked story is followed by reading, Movietalk and Picturetalk) than if the input is familiar but untargeted.

  6. What about the kid who missed some of (in this case) El Internado? If the speaking assessment focuses on Internado-specific vocab, it would (in my opinion) be unfair to ask Johnny who was there for all three periods and Maninder, who missed two of three periods, to do the same thing with the “language content” of the episodes.
  7.  Kids hate speaking and tests.  Anything I can do to avoid tests, or putting people on the spot– which a one-on-one test does– I do.  This is what Johnny looks like when you tell him, speaking test tomorrow:Image result for kid being interviewed by teacher
    (image:  Youtube)
  8. “Authentic content” eg El Internado has lots of low-frequency vocabulary. Sure, the teacher can keep things comprehensible, but there is inevitably kids’ mental bandwidth going into processing low-freq vocab…which is exactly what kids don’t need in a speaking assessment, where you want high-freq vocabulary that is easy to recall and applicable to lots of topics.

Anyway…this is why I save speaking assessment until the end of the course: I know how well my kids can speak, I can adjust aural input where it matters– right now–, I don’t want assessment to detract from input, and speaking assessment doesn’t really help me or my kids.

 

 

 

Should I Mark Behavior? The Great JGR Debate, and a Silver Lining for Behaviour Rubrics.

Some years ago, a teacher built a rubric for her C.I. classes, which Ben Slavic named JGR and which was discussed on his blog and then elsewhere.  Here is a version I have played around with: INTERPERSONAL COMMUNICATION rubric.  I initially opposed the use of JGR, then used it, then ditched it, and now I use it (but not for marks). Note: this is a modified version of the original JGR; and I don’t know for how long she used her rubric, or if she still does, or what the original looked like.

JGR was developed because– like all of us, especially me– the creator had some challenges managing her C.I. classes in her initial year with T.P.R.S., which can in (especially my) rookie hands turn into a “woo-hoo no more textbook!” clown show.  JGR basically “marks” classroom behaviour.  JGR specifies that students make eye contact, add story details, ask for help, not blurt, not use cell-phones etc.  Jen used it (and if memory serves Ben also recommended its use) by making part of her class mark a function of behaviour as marked by JGR.  So the kids might get, say, 20% of their mark each for reading, writing, listening, speaking and 20% for their in-class behaviour.  Part of the thinking here was that some behaviours lead to acquisition, while others do not and also wreck the classroom environment, and so “acquisition-rewarding” behaviour should be rewarded.

JGR– for many people, including me– “works.”  Which is why– especially when linked with allegedly “acquisition-promoting” behaviours– lots of people are interested in it.

JGR is a kind of “carrot-and-stick” marking tool:  if the kids engaged in the behaviours JGR specified, their marks went up, partly because (a) they got marks for those behaviours, and partly because (b) the behaviours should– in theory– help them acquire more language.

This can of worms was shaken around a bit on Ben’s blog, and recently, thanks to the always-remarkable Terry Waltz, there have been FB and Yahoo discussions about it.  So, today’s question:

Should we assess in-class behaviour for final marks purposes?

My answer: no, never.  Why?

1. Behaviours typically asked for in JGR– or other such rubrics– are not part of any     curricula of which I am aware.  Every language curriculum says something like, students of the Blablabian language will read, write, speak and understand spoken Blablabian, and maybe say something about Blablabian culture.  Nowhere does any  curriculum say “students should suggest details for stories” or “students will lookthe teacher in the eye.”

If it’s going to get a mark, it has to be part of course outcomes.  Any assessment guru (Wormelli, Harlen, etc) will tell you the same thing: we do not mark attitude, behaviour, homework, etc, as these are not part of final outcomes.

To put it another way, how do we judge the New England Patriots football team?  By how well, often and/or enthusiastically they practice and look Bill Belichick in the eye, or by how many games they win?  How should Tom Brady be paid: by how often he shows up for practice, and how nice he is to Belichick, or by how many yards he successfully throws?  That’s right.

We could– and I often do– end up in situations where a “bad” kid does well, or a “good” kid does poorly.  I have had bright-eyed, bushy-tailed teacher’s pet-type kids who were not especially good at Spanish, and I have had giant pains-in-the-butt who were quite good.

My best-ever student in TPRS, Hamid Hamid, never added story details, never looked up, and always faced away from the board.  Yet he CRUSHED on assessments and got 100% in Spanish 2.  Two years later, his younger brother, Fahim (also a great student) told me that Hamid Hamid was both shy and deaf in his left ear, so always “pointed” his right ear at the board (and so appeared to be looking away).  This kid’s mark would have been lowered by assessing his “in-class behaviour,” which– given his epic Spanish skills– would have been absurd.

2. As Terry Waltz points out, neurodivergent kids can– and do– acquire language without engaging in many behaviours typically required by participation and behaviour rubrics. She also points out that forcing neurodivergent kids into the “normal” mold is at best less than productive. If you are autistic, anxious, suffering from PTSD (as my stepdaughter does) or facing any other neuro challenges, “engagement” rubrics can make your life miserable while not appreciably meaningfully measuring what you can do with the language.

3. The only thing required for language acquisition is reception of comprehensible input.  While the focus of behaviour rubrics is designed to get kids to tune in, it does not follow that many behaviours which do make for a good class– e.g. people adding good details to stories, looking at each other– are necessary to acquire language.

All of us have been there: you have a plan, you did your story warmup or whatever, but the kids aren’t into it.  You bust out a Movietalk but they aren’t into that either.  Dead class. Now, in a C.I. class, we don’t have recourse to worksheets or whatever, and we still have to teach the language. I have a bail-out move here: direct translation, and I always have a novel on the go, so I can read aloud, and Q&A the novel.  If I’m being particularly non-compelling, I’ll throw an exit quiz at them.

The point: if the kids are getting C.I., they are acquiring.  If they are miserable/tired/bored with stories, fine.  They are gonna get C.I. one way or another.

4. Any kind of behaviour rubric plays the awful “rewards” game.  Ask yourself this question:  why do I teach? The answer– other than because I have to make a living— is probably something like, because it’s interesting, I have some measure of control over my work, and I love kids and my subject.  Some will add that teaching, properly done, opens doors for kids.  Teachers do not teach because they want to be evaluated, or because they want to use the latest gizmo, or because they want public praise, etc.  They are, in other words, intrinsically motivated.  They want to work because the work is good and worthy in itself.

When we institute rewards for behaviours, as Alfie Kohn has spent a career arguing, we destroy intrinsic motivation.  We turn something interesting into payment for marks.  The point stops being paying attention to the story– or adding to it cos you actually care about it– and becomes something rote.

5. Using behaviour rubrics can dampen professional self-examination. If my practice is such that I have to use marks as a stick to keep kids in line (the policing metaphor is not an accident), there are two possibilities: tough kids, and/or I am doing a bad job.  The question why are they not tuned in? might be answerable with any of the following:

— I am not being sufficiently comprehensible

— I am ignoring the top or the bottom end of the class– too fast/slow or simple/complex

— my activities are not interesting, varied or meaningful enough

— the kids see no purpose

— accountability: they don’t see tuning in as something that results in real gains

— I lack basic skills (smart circling, control of vocab, etc etc)

— my story sucks 😉

I had better be able to look in the mirror, consider and then deal with these possibilities, rather than merely acting like a cop and demanding obedience.

Now, behaviour does matter.  You cannot run a T.P.R.S. class without rules etc.  My basic rules:

  • no phones or other distractions (including side-talk, blurting etc)
  • no insults of anyone other than oneself or of rich entitled people
  • listen, watch and read with the intent to understand; ask when you don’t
  • do not create or engage in distractions

The tools that we have for dealing with distracting behaviour include

  • warning the offender, standing by their desk, calling Mom and Dad, etc
  • pointing, with a smile, to classroom rules every time there is a problem
  • sending them to admin if necessary
  • taking their phone until 3:15 (most kids would rather die)
  • detention, where we discuss behaviour
  • assigning read & translate (quiet seatwork)
  • taking the kids outside for a walk, or doing some other kind of physical brain-break
  • changing activities
  • doing a quiz
  • talking to kids one on one and asking what do I need to do to get you focused?

 

The upshot?  We should not, and need not, mark “behaviour” or “participation.”

 

Addendum:  is there ever a use for classroom behaviour rubrics?

Yes.  I get my kids to self-evaluate using JGR every 2-3 weeks.  My version generates a mark out of 20.

Nineteen out of twenty kids will very honestly self-evaluate their behaviour, provided they understand exactly what is expected.  One kid in twenty will heap false praise on him/herself.  For the false praisers (“I never blurt in class!”), I sit them down and explain what I think, then we agree on a more realistic mark.

I save these JGR “marks” and once in a blue moon, when a helicopter parent or an Admin wants to know, how is Baninder doing in Spanish, I can point to both the spreadsheet with Numberz and JGR.  This frames the inevitable discussion about marks in terms any parent can understand.  Any parent, from any culture, understands that if Johnny screws around and/or does not pay attention in class, his mark will drop.

JGR– in my experience– accurately “predicts” the marks of about 80% of kids.  When I can show a kid (or a parent or admin), look, here are Johnny’s marks AND Johnny’s own description of how he behaves in class, we can have an honest discussion about marks, Spanish, etc.  Win-win.

How Do I Do Dictation?

Dictation is old as the language-teaching hills.  I remember doing this in my various français classes and also in Cherman viss Frau Satcher, ja.

Now, dictée is like running, weightlifting or learning music: if you do it wrong, the results are frustrating, painful or non-existent.  Like everything else in language teaching that we still do from 60 years ago, dictado has been tweaked.

Dictation:

  1. teacher reads 5-10 inter-related and meaningful sentences aloud
  2. these sentences should use the most-recently-taught vocab
  3. students write down what they hear.
  4. teacher can repeat each 2-3 x
  5. Ben Slavic suggests then projecting the sentences on the overhead and having kids correct their spelling.
  6. I always finish with translate into L1.

 

What Diktation should not be:

  • a way of introducing new vocabulary and/or grammar
  • a way for students to “practise” spelling
  • isolated meaningless sentences (e.g. sentence #1 is “the boy is tall” and #2 is “it is raining”)
  • to any degree incomprehensible
  • graded for spelling to any significant extent

I use dictation as assessment, not as a vocab-acquiring activity, and I follow the 80/80 rule: 80% of class needs to get 80% or more for me to move on. Scores are usually quite high, as I do dictation after asking a story and doing readings of the story, or other stories using the same vocab.  The kids don’t complain, it is a zero-prep activity, and it is 90% C.I. as the kids know the vocab.  This is mostly how I get listening marks.  Although it looks like output, mostly it’s input– listening– and the output is in L1.  As James Hosler says, dictation while being assessment is another way to deliver C.I.

Here is what I am going to do for dictation on Monday in Spanish 2.

  1. There was a grandmother who wanted to give her grandson money.
  2. She told him “you must win a a dance contest”
  3. He said “I don’t like to dance, because I am lazy.”
  4. The grandmother wanted to give her money to Donald Trump.
  5. Donald Trump did not want her money.

You could also

  • project a picture and describe it
  • describe a character from your novel or story
  • project bits of a clip from a video, narrate a sentence, and have them write that

The Zen of Language Teaching

Here are your koans.  Think on them.

 

If you want to successfully teach grammar, do not teach grammar.

If you want your students to talk, do not ask them to talk.

If you want your students to write well, do not make them practise writing.

If you want them to acquire more words, teach them fewer words.

If you want to make them fluent, do not try to make them fluent.

If you want your students to acquire a language, do not teach them about the language.

If you want your students to know the meanings of lists of words, do not give them lists of words.

If you want your students to spell properly, do not make them practise spelling.

Just because nothing appears to happening doesn’t mean nothing is happening.

Just because something is happening doesn’t mean anything is happening.

If you want your students to read, do not teach them how or what to read.

If you want your students to prepare for the unknown, make them comfortable with what they know.

A student without a language dictionary is like a fish without a bicycle (sorry Gloria).

A language classroom without lists of words is like a phone book without stories.

“If you want to build a ship, do not gather the men to collect wood, divide up the work, or give orders.  Teach them instead to yearn for the vast and infinite sea.”– Antoine de St. Exupery

As always, the ideas here are are grounded in research, and this one was inspired by Mandarin and S.L.A. guru Terry Waltz.

What is “circling” and how do I do it?

Learners need a LOT of meaningful repetition to acquire something, so years ago Susan Gross developed the “circling” technique to allow teachers to make huuuuuuge numbers of repetitions on vocab.  Here’s how you do it, and no, you don’t have to use T.P.R.S. to benefit.  You are also going to circle sentences you find in reading, and things you say in Movietalk.

1.  Start with a sentence– Rochelle quiere jugar fútbol (R. wants to play soccer) & make sure kids understand it.

2. Ask a yes question– ¿clase, Rochelle quiere jugar fútbol?–and class answers sí.  Restate sentence.

3.  Ask a no question–clase, ¿Rochelle quiere jugar basquetbol ?– and class answers no. Restate sentence.

4. Ask an either/or question– clase, ¿quiere Rochelle jugar basquetbol o fútbol?– and class answers fútbol.  Restate sentence.

5.  Ask an “adding detail” question where kids have input– clase, ¿dónde quiere jugar fútbol Rochelle?— and when they suggest something interesting, add that to the sentence, e.g. Sí, clase, ¡Rochelle quiere jugar basquetbol en Barcelona!

6.  Now, circle the new detail, always restating the sentence s.l.o.w.l.y. Clase, ¿quiere Rochelle jugar fútbol en Barcelona? ¿Quiere jugar fútbol en Los Angeles? etc

If you want to add details, “with whom?” and “where?” questions are best, as these add details without adding new vocab.  In T.P.R.S., we want to recycle a small amount of vocab so people really acquire it, rather than swamping students in an ocean of partly-acquired words.

The most important thing I have learned about circling is, don’t overdo it.  If you have a story with, say, 3 parallel characters, you are going to re-use each sentence for each character, so please for the kids’ sake do not beat the sentences to death. If your structure is quería tener (wanted to have) you can ask a yes question about one character, a no question about another, etc. If you are doing Ben Slavic-style “pre-teaching” where you circle and play around with vocab before asking a story, always start with two sentences (more variety).

Goddess Laurie Clarq also weighed in– read her ideas here— and another suggestion (dunno where this came from) is to circle subject, verb then object (or to mix the order up).

E.g. your sentence is Maninder tiene tres novos guapos (M. has three handsome boyfriends).

So, first you circle Maninder.  Clase, ¿tiene tres novios guapos Maninder? ¿Tiene tres novios guapos Anna? ¿Tiene tres novios Anna o Maninder?  Always repeat the sentence.

You next circle the verb.  Clase, ¿,quiere o tiene tres novios guapos Maninder? Clase, ¿quiere tres novios guapos? Etc

Finally, you circle the subject.  Clase, ¿Maninder tiene tres novios guapos? ¿Tiene tres perros? ¿Tiene tres gatos o tres novios?

The circling keys to success are

  • go s.l.o.w.l.y
  • keep it 100% comprehensible
  • go slow enough to be understood, and fast enough to not be boring
  • use parallel characters (or sentences) so you don’t beat your questions to death
  • DO NOT CIRCLE EVERYTHING!  You only need to (mainly) circle new-ish stuff.

So You Think You Know Grammar?

I have always urged readers to join Ben Slavic’s blog ($5/month well-spent).  Ben’s books are also well worth a read, esp T.P.R.S. In A Year (without which I probably could not have started T.P.R.S.). Today I am gonna share part of a post from Ben’s six years ago.  Latin master Robert Harrell– who has won every award you can name, and who has used  T.P.R.S. to triple his school’s Latin enrollment, plus producing kids who speak fluent Latin and who crush the A.P. Latin exam without doing six years of grammar worksheets — has a response to the grammarians.  If a grammarian blathers on about how one must know grammar rules, show them this.  For Harrell’s commentary and the full entry, see Ben’s blog.

Let me suggest the following “experiment”: I have a ten-question quiz. Without preparation, give it to any “non-language” (i.e. not teaching English or a foreign language) person at the school, including administrators and evaluators, and see if they pass it.  

Remember that these are experienced speakers of English with advanced degrees that have included many English classes, so the proctor is not allowed to explain any of the terms used, give examples or otherwise provide hints.

Please give the correct form for each of the following verbs:
1. to drink – 3rd person neuter singular present perfect active
2. to go – 2nd person plural future perfect active
3. to hang – 1st person singular future perfect passive
4. to speak to – 3rd person plural pluperfect passive
5. to equivocate with the idiom “to go” – 3rd person feminine singular future continuous active
6. to hang – 3rd person neuter singular pluperfect passive
7. to hear – 2nd person singular pluperfect passive
8. to lay – 3rd person masculine singular future perfect progressive active
9. to lie (= be in a horizontal position) – 3rd person feminine singular present perfect active
10. to be – 1st person singular pluperfect active subjunctive
Bonus: Use the verb in #10 in a conditional sentence.

For those who don’t want to think this through, here are the answers:
1. It has drunk
2. You will have gone
3. I will have been hanged
4. They had been spoken to.
5. She is going to be equivocating
6. It had been hung
7. You had been heard
8. He will have been laying
9. She has lain
10. I had been

Do we understand?