Exit Quizzes

Should I Mark Behavior? The Great JGR Debate, and a Silver Lining for Behaviour Rubrics.

Some years ago, Jen S. and her colleagues built a behaviour rubric for her C.I. classes, which Ben Slavic named JGR and which was discussed on his blog and then elsewhere.  Here is a version I have played around with: INTERPERSONAL COMMUNICATION rubric.  I initially opposed the use of JGR, then used it, then ditched it, and now I use it (but not for marks). Note: this is a modified version of the original JGR; and I don’t know for how long she used her rubric, or if she still does, or what the original looked like.

JGR was developed because– like all of us, especially me– the creator had some challenges managing her C.I. classes in her initial year with T.P.R.S., which can in (especially my) rookie hands turn into a “woo-hoo no more textbook!” clown show.  JGR basically “marks” classroom behaviour.  JGR specifies that students make eye contact, add story details, ask for help, not blurt, not use cell-phones etc.  Jen used it (and if memory serves Ben also recommended its use) by making part of her class mark a function of behaviour as marked by JGR.  So the kids might get, say, 20% of their mark each for reading, writing, listening, speaking and 20% for their in-class behaviour.  Part of the thinking here was that some behaviours lead to acquisition, while others do not and also wreck the classroom environment, and so “acquisition-rewarding” behaviour should be rewarded.

JGR– for many people, including me– “works.”  Which is why– especially when linked with allegedly “acquisition-promoting” behaviours– lots of people are interested in it.

JGR is a kind of “carrot-and-stick” marking tool:  if the kids engaged in the behaviours JGR specified, their marks went up, partly because (a) they got marks for those behaviours, and partly because (b) the behaviours should– in theory– help them acquire more language.

This can of worms was shaken around a bit on Ben’s blog, and recently, thanks to the always-remarkable Terry Waltz, there have been FB and Yahoo discussions about it.  So, today’s question:

Should we assess in-class behaviour for final marks purposes?

My answer: no, never.  Why?

1. Behaviours typically asked for in JGR– or other such rubrics– are not part of any     curricula of which I am aware.  Every language curriculum says something like, students of the Blablabian language will read, write, speak and understand spoken Blablabian, and maybe say something about Blablabian culture.  Nowhere does any  curriculum say “students should suggest details for stories” or “students will lookthe teacher in the eye.”

If it’s going to get a mark, it has to be part of course outcomes.  Any assessment guru (Wormelli, Harlen, etc) will tell you the same thing: we do not mark attitude, behaviour, homework, etc, as these are not part of final outcomes.

To put it another way, how do we judge the New England Patriots football team?  By how well, often and/or enthusiastically they practice and look Bill Belichick in the eye, or by how many games they win?  How should Tom Brady be paid: by how often he shows up for practice, and how nice he is to Belichick, or by how many yards he successfully throws?  That’s right.

We could– and I often do– end up in situations where a “bad” kid does well, or a “good” kid does poorly.  I have had bright-eyed, bushy-tailed teacher’s pet-type kids who were not especially good at Spanish, and I have had giant pains-in-the-butt who were quite good.

My best-ever student in TPRS, Hamid Hamid, never added story details, never looked up, and always faced away from the board.  Yet he CRUSHED on assessments and got 100% in Spanish 2.  Two years later, his younger brother, Fahim (also a great student) told me that Hamid Hamid was both shy and deaf in his left ear, so always “pointed” his right ear at the board (and so appeared to be looking away).  This kid’s mark would have been lowered by assessing his “in-class behaviour,” which– given his epic Spanish skills– would have been absurd.

2. As Terry Waltz points out, neurodivergent kids can– and do– acquire language without engaging in many behaviours typically required by participation and behaviour rubrics. She also points out that forcing neurodivergent kids into the “normal” mold is at best less than productive. If you are autistic, anxious, suffering from PTSD (as my stepdaughter does) or facing any other neuro challenges, “engagement” rubrics can make your life miserable while not appreciably meaningfully measuring what you can do with the language.

3. The only thing required for language acquisition is reception of comprehensible input.  While the focus of behaviour rubrics is designed to get kids to tune in, it does not follow that many behaviours which do make for a good class– e.g. people adding good details to stories, looking at each other– are necessary to acquire language.

All of us have been there: you have a plan, you did your story warmup or whatever, but the kids aren’t into it.  You bust out a Movietalk but they aren’t into that either.  Dead class. Now, in a C.I. class, we don’t have recourse to worksheets or whatever, and we still have to teach the language. I have a bail-out move here: direct translation, and I always have a novel on the go, so I can read aloud, and Q&A the novel.  If I’m being particularly non-compelling, I’ll throw an exit quiz at them.

The point: if the kids are getting C.I., they are acquiring.  If they are miserable/tired/bored with stories, fine.  They are gonna get C.I. one way or another.

4. Any kind of behaviour rubric plays the awful “rewards” game.  Ask yourself this question:  why do I teach? The answer– other than because I have to make a living— is probably something like, because it’s interesting, I have some measure of control over my work, and I love kids and my subject.  Some will add that teaching, properly done, opens doors for kids.  Teachers do not teach because they want to be evaluated, or because they want to use the latest gizmo, or because they want public praise, etc.  They are, in other words, intrinsically motivated.  They want to work because the work is good and worthy in itself.

When we institute rewards for behaviours, as Alfie Kohn has spent a career arguing, we destroy intrinsic motivation.  We turn something interesting into payment for marks.  The point stops being paying attention to the story– or adding to it cos you actually care about it– and becomes something rote.

5. Using behaviour rubrics can dampen professional self-examination. If my practice is such that I have to use marks as a stick to keep kids in line (the policing metaphor is not an accident), there are two possibilities: tough kids, and/or I am doing a bad job.  The question why are they not tuned in? might be answerable with any of the following:

— I am not being sufficiently comprehensible

— I am ignoring the top or the bottom end of the class– too fast/slow or simple/complex

— my activities are not interesting, varied or meaningful enough

— the kids see no purpose

— accountability: they don’t see tuning in as something that results in real gains

— I lack basic skills (smart circling, control of vocab, etc etc)

— my story sucks 😉

I had better be able to look in the mirror, consider and then deal with these possibilities, rather than merely acting like a cop and demanding obedience.

Now, behaviour does matter.  You cannot run a T.P.R.S. class without rules etc.  My basic rules:

  • no phones or other distractions (including side-talk, blurting etc)
  • no insults of anyone other than oneself or of rich entitled people
  • listen, watch and read with the intent to understand; ask when you don’t
  • do not create or engage in distractions

The tools that we have for dealing with distracting behaviour include

  • warning the offender, standing by their desk, calling Mom and Dad, etc
  • pointing, with a smile, to classroom rules every time there is a problem
  • sending them to admin if necessary
  • taking their phone until 3:15 (most kids would rather die)
  • detention, where we discuss behaviour
  • assigning read & translate (quiet seatwork)
  • taking the kids outside for a walk, or doing some other kind of physical brain-break
  • changing activities
  • doing a quiz
  • talking to kids one on one and asking what do I need to do to get you focused?

 

The upshot?  We should not, and need not, mark “behaviour” or “participation.”

 

Addendum:  is there ever a use for classroom behaviour rubrics?

Yes.  I get my kids to self-evaluate using JGR every 2-3 weeks.  My version generates a mark out of 20.

Nineteen out of twenty kids will very honestly self-evaluate their behaviour, provided they understand exactly what is expected.  One kid in twenty will heap false praise on him/herself.  For the false praisers (“I never blurt in class!”), I sit them down and explain what I think, then we agree on a more realistic mark.

I save these JGR “marks” and once in a blue moon, when a helicopter parent or an Admin wants to know, how is Baninder doing in Spanish, I can point to both the spreadsheet with Numberz and JGR.  This frames the inevitable discussion about marks in terms any parent can understand.  Any parent, from any culture, understands that if Johnny screws around and/or does not pay attention in class, his mark will drop.

JGR– in my experience– accurately “predicts” the marks of about 80% of kids.  When I can show a kid (or a parent or admin), look, here are Johnny’s marks AND Johnny’s own description of how he behaves in class, we can have an honest discussion about marks, Spanish, etc.  Win-win.

C.O.F.L.T. Conference Reflections

The energy-loaded Tina Hargaden, vice-president of the C.O.F.L.T. in Portland, organised a conference and I got to do the T.P.R.S. part of it– a one-day workshop with German storyasking demo, reading, Movietalk, Picturetalk, method explanation, Q&A, etc.

To say I had a busy weekend would be an understatement:  work Fri, drive 7 hours to Portland (through Seattle traffic, its own special Hell, thank you NPR for making it bearable), have a beer and talk shop with Tina, sleep like a baby at the Kennedy School Hotel (a high school converted to hotel– awesome– “fall asleep in class” is their tag), do presentation, drive back to Canada, time change, it’s now 1 AM, sleep three hours, get on plane to Cuba…where thank God they have mojitos  and overhung limestone rock routes.

Anyway, we had the most people of any workshop at the conference (almost 30) and Tina told me that we were the only room where people were regularly laughing.  There were a bunch of Chinese student teachers doing their degrees in Portland, a few TPRSers who were in for a tune-up, and a whack of curious rookies.

So I got my evaluations back.  You can see the COFLT 2016 Stolz TPRS feedback forms if you want to see how awesome I am 😉 and how much Oregonians appreciate their gluten-free, salad-based, vegan or organic meat, locally-sourced artisanally-cooked dishes, etc 😄.  But mostly what is interesting in the comments are the themes that recur.

1.  A lot of people said they really appreciated the German demo aspect of the presentation (an idea I got from Blaine Ray).  People wrote along the lines of “it was great to experience what it is like to be a student.”  I remain convinced that the only way to make any language-instruction method convincing is to teach people part of a language they don’t know.  It is so easy for us to forget how tough it is– even with good C.I.– to pick up a new language.

2.  Recognising that, and because we had some native Mandarin speakers at the workshop, I asked participant Yuan to teach us some Mandarin (Blaine Ray also does this).  She parallel-circled two sentences:  Chris climbs mountains and Tina drinks beer

  
This put me into the students’ seat and it was enlightening.  I noticed two things:

a) I needed a LOT of reps to remember the Mandarin, and I was glad Yuan went s.l.o.w.l.y.

b) Mandarin does not seem very difficult.  No articles, verb conjugation, etc, though word order seems crucial.

3.  Most people wanted more time with T.P.R.S. (or even me as presenter).  There seems to be a need (in OR and WA) for more C.I.-themed language workshops.  Luckily, Tina Hargaden and C.O.F.L.T. on it and there will be a conference Oct 13-15 which will feature Steve Krashen, Karen Rowan, etc.

4. I talked to another presenter who had a workshop called something like “using authentic docs to design authentic tasks for authentic assessment.” He did some explaining and I wondered two things:

a. What do you actually do with the info from an “end of unit” assessment?  If Max and Sky do well, and Rorie and Arabella terribly, now what? How does that info shape your next “unit”?  I guess if you want a number, awesome, but numbers help neither teachers nor students. 

b. How much energy is a teacher productively using when they design #authres-based activities for assessment? I mean, most #authres don’t use high-freq vocab and are often more of a guessing game for students.  

As I talked to this guy, it struck me that you would get a lot better assessment with exit quizzes for reading & translating, and with comprehension checks along the way– especially with what Ben Slavic has called “barometer kids”– so that, in the moment, you can provide more input for what the kids are misunderstanding. 

5. Laughter matters. Laughing bonds people, lightens any mood, is a brain break, comes from when unexpected ideas are conjoined, etc. So I am glad that we got to laugh at our workshop (yet another practice that Blaine Ray is all about with his dancing monkeys and girls without noses). 

6. There were some experienced C.I. teachers there and I was super-stoked (sorry I can’t remember names). These folks asked good questions, and they often said “well Chris does ____ but I do _____ instead.”  Which teaches us that while there is a basic C.I. recipe– use a story, limit and recycle vocab, have people read the story, add images and short films for more vocab recycling– there are many cooks with a panoply of flavours.  Also,  the experienced people generated great lunchtime discussions over craft organic artisanal salads and quinoa vegan quiche 😉. 

So, thanks COFLT and Tina for a great opportunity for all those language teachers. Their Oct confernce will rock– stay tuned. 

How Do I Do Dictation?

Dictation is old as the language-teaching hills.  I remember doing this in my various français classes and also in Cherman viss Frau Satcher, ja.

Now, dictée is like running, weightlifting or learning music: if you do it wrong, the results are frustrating, painful or non-existent.  Like everything else in language teaching that we still do from 60 years ago, dictado has been tweaked.

Dictation:

  1. teacher reads 5-10 inter-related and meaningful sentences aloud
  2. these sentences should use the most-recently-taught vocab
  3. students write down what they hear.
  4. teacher can repeat each 2-3 x
  5. Ben Slavic suggests then projecting the sentences on the overhead and having kids correct their spelling.
  6. I always finish with translate into L1.

 

What Diktation should not be:

  • a way of introducing new vocabulary and/or grammar
  • a way for students to “practise” spelling
  • isolated meaningless sentences (e.g. sentence #1 is “the boy is tall” and #2 is “it is raining”)
  • to any degree incomprehensible
  • graded for spelling to any significant extent

I use dictation as assessment, not as a vocab-acquiring activity, and I follow the 80/80 rule: 80% of class needs to get 80% or more for me to move on. Scores are usually quite high, as I do dictation after asking a story and doing readings of the story, or other stories using the same vocab.  The kids don’t complain, it is a zero-prep activity, and it is 90% C.I. as the kids know the vocab.  This is mostly how I get listening marks.  Although it looks like output, mostly it’s input– listening– and the output is in L1.  As James Hosler says, dictation while being assessment is another way to deliver C.I.

Here is what I am going to do for dictation on Monday in Spanish 2.

  1. There was a grandmother who wanted to give her grandson money.
  2. She told him “you must win a a dance contest”
  3. He said “I don’t like to dance, because I am lazy.”
  4. The grandmother wanted to give her money to Donald Trump.
  5. Donald Trump did not want her money.

You could also

  • project a picture and describe it
  • describe a character from your novel or story
  • project bits of a clip from a video, narrate a sentence, and have them write that

How do exit quizzes work?

Richmond, B.C. powerhouse teacher Sonya ONeill writes “Exit quizzes…could you post an example?–I’m a bit confused about how to do these well. Do you do translations only? If so, are you starting in Spanish always with all levels? Do you ever use comprehension questions (of the story you just asked) at this time? Are these your main listening assessments?”

OK, today’s question, how can we do exit quizzes?

My system is simple.

  • Based on what we did in class, I read five sentences aloud.  These sentences contain the vocab from the story we are working on, or are sentences directly from that story.  If they are not from the story, they have to be stand-alone meaningful.
  • I tell the kids, “write down what you hear in Spanish, then translate into English.”
  • They write Spanish then translate into English.
  • The kids trade papers and we mark (the Spanish writing doesn’t matter much– it’s comprehension we are after).
  • The kids return the marked papers to each other.
  • I get a show of hands: Put your hand up if you got either 4/5 or 5/5.

If 80% of class got 4 or 5 out of 5, I am happy.  If not, I delivered bad/too little input, or they weren’t listening, and so we need to do more work around those sentences.  Sometimes I collect the marks, sometimes not.

Do I do “comprehension questions”?  By this, Sonya (I think) means, Do I ask the kids comprehension questions based on the story we have read/asked without them looking at/hearing the story at the time of the quiz?  I.e., do they have to remember and then answer?

Never.  Why?  The problems with comprehension questions are as follows

a) especially with beginners, the “mental load” involved in comp questions is super high, because kids have to do three things

  • decode the meaning of the question
  • remember content
  • write answers.

We know output (writing or speaking) does not aid acquisition, so no point with that.  We also know that all we need for acquisition is comprehensible input, so again responses don’t help.  This is a lot of mental work, and Bill VanPatten reminds us that what we might call “mental bandwidth overload” is an inevitable and insurmountable fact.  Basically, the less they have to “do” with a chunk of language, the more processing power they have for each chunk.

b) We also know that comprehension always and massively outpaces production.  Our kids– and we teachers– always  recognise more words in any language than we can produce.  If we ask for output, we may be forcing kids to “do” something they havn’t acquired yet.  

Say I tell my kids five sentences in Spanish, one sentence at a time.  Max (average), Samba (fast processor) and Rorie (insanely fast) all understand the new structures fui and trabajé that were in our story.  But Max hasn’t acquired them (i.e. he can’t say or write them) yet, while Samba and Rorie have.  If we know that acquisition goes at different speeds for different students, does asking for output not penalise Max for something he cannot control?

c) In my view– and I thank James Hosler for this insight– assessment should basically just be another excuse to deliver input to the kids.  I don’t want to play “gotcha” and I want people to succeed, so I’ll focus listening around what they can understand and easily do.

By the way, I think we can totally use questions for exit quizzes, provided we do not ask for output answers.  Just have one sentence be the question and the next the answer.  You say 1. ¿trabajaste anoche? and the kids write it down in Spanish, and then translate it: “did you work last night?”  Your next sentence is 2. “No, no trabajé anoche.  Fui al cine” and the kids write that down, and then they write “No, I didn’t work.  I went to the movies.”  Then you say three more Spanish sentences which they all copy and translate.

Once the quizzes are done and marked, you can also use these for PQA if you have a few minutes at the end of class.  Ask the fast processors some of the questions and have the class listen to what they say.