Assessment and Evaluation

(How) Should I Use Questions to Assess Reading?

Yesterday I found a kid in my English class copying this from her neighbour.  It is post reading assessment– in Q&A form– for the novel Les yeux de Carmen. TPT is full of things like this, as are teachers guides,, workbooks, etc.

The idea here is, read, then show your understanding of the novel by answering various questions about it. It “works” as a way to get learners to re-read, and as what Adminz like to call “the accountability piece,” ie, “the reason to do it is cos it’s for marks.”

Before I get into today’s post, I should note, I (and every teacher I know) uses some kind of post-reading activity.

Q: Should I use questions to assess reading?

A: Probably not. Here’s why.

  1. How do we mark it? What if the answer is right, but the French is poor? Or the reverse? Half a mark each? Do we want complete sentences? What qualifies as acceptable and not for writing purposes? What if there is more than one answer? What’s the rubric we use for marking?
  2. It can (and, basically, should) be copied. This is the kind of thing that a teacher would send home to get kids to re-read the novel. Fine, but…it’s boring, and it takes a long time. It doesn’t use much brain power. If I were a student, I would copy this off my neighbour. If you don’t get caught, you save a bunch of time, and the teacher has no way of noticing.
  3. It would totally suck to mark this. Do you actually want to read 30– or 60!— of these?!? I dunno about you folks, but I have a life. We have to mark, obviously, but these, ugh, I’d fall asleep.
  4. It’s a lot of work for few returns. I asked the kid who’d lent her answers to her friend how long it took (btw, there is one more page I didn’t copy), and she said “about 45 min.” This is a lot of time where very little input is happening.  The activity should either be shorter, or should involve reading another story. As Beniko Mason, Stephen Krashen and Jeff McQuillan (aka The Backseat Linguist) show us, input is more efficient than input plus activities (ie, instead of questions about a story, read another story).  As the great Latinist James Hosler once remarked, “for me, assessment is just another excuse to deliver input.”

So…how should we assess reading? Here are a bunch of ideas, none of them mine, that work.

A. Read the text, and make it into a comic. Easy, fun, useful for your classroom library and requires a bit of creativity.

B. Do some smash doodles. This is basically a comic, but minus any writing. As usual, Martina Bex has killer ideas.

C. Do a discourse scramble activity. For these, take 5-10 sentences from the text, and print them out of order (eg a sentence from the end of the text near the beginning, etc). Students have to sort them into correct order, then translate into L1. This is fairly easy– and even easier if a student has done the reading, heh heh–, as well as not requiring output while requiring re-reading.

Another variant on a discourse scramble is, have students copy the sentences down into order and then illustrate them.

For C, they get one mark per correct translation (or accurate pic), and one mark for each sentence in its proper place. Discourse scramble answers can be copied, so I get kids to do them in class.  They are also due day-of, because if kids take them home others will copy.

D. If you have kids with written output issues, you can always just interview them informally: stop at their desk or have them come to you and ask them some questions (L1, or simple L2) about the text.

Alrighty! Go forth and assess reading mercifully :-).

 

 

 

Oral Assessment Made Easy

If you must assess oral proficiency– which should not be done before the end of Level 2, in my humble opinion– here is the world’s simplest hack. No more interviews, “speaking tasks” and hassles where you sit with one kid and the other 28 are off-ta– err, I mean, doing their Kahoots or whatever.

In a C.I. class, because any feedback other than “pay more attention & ask for help” doesn’t do anything for acquisition, and because testing wastes time by not delivering input, we want to put as little time and effort into Assigning Numberz as possible. We also never want to assign role-plays or “pretend you’re customer and sales associate”-type scenarios, which test memorisation rather than spontaneous communication.

Image result for kids reading from scripts

This is what we do not want.

This is very simple. Starting in the last 1/4 or so of level 2 Spanish, I randomly check on how kids communicate in class.  Do they ask and answer questions?  Do they use complete sentences? Can they initiate and sustain conversation? To what if any extent do their speaking errors interfere with communication?

Every two weeks, I give each kid a score out of three. At the end of the year, I average these and that’s their oral mark. If the kids disagree, they can come in for a formal oral interview. (If they choose this, it is totally unstructured and unplanned ie they cannot prepare and memorise Qs & As. I will ask them to tell me a story including dialogue, and we will basically ask each other questions.) I have to be conscious: some days kids are sick, exhausted etc, so the score has to reflect their overall (ie best possible) proficiency.

This is the first year I have done this and not a single kid complained about their mark.

Here’s the marking rubric. Go ahead and steal it but please acknowledge authorship.  Note that the rubric will generate a mark between four and twelve.

Oral proficiency is evaluated in the last ¼ of Spanish 11. I will note how you communicate in class. If you disagree with your oral mark, please come in for a one-on-one interview. I am available Fridays after 2:30.

When using Spanish, for a score of ___, I
3 (mastery) 
— ask and answer questions in complete sentences.
— initiate and sustain conversations.
— demonstrate understanding of what is being said to me

— have minor errors that do not interfere with meaning

2 (basic proficiency)
— occasionally ask and answer questions in complete sentences
— sometimes initiate and sustain conversations
— generally demonstrate understanding of what is being said to me
— have errors that noticeably interfere with meaning


1 (not yet proficient) 

— use mostly one-word statements
— don’t sustain or initiate conversations
— often don’t clearly understand what is being said to me

— have errors that consistently block meaning

C.I.-taught Students Evaluated by A.C.T.F.L. Writing Standards

How well do C.I.-taught students do in terms of ACTFL writing standards? Well…pretty darned well, I’d say.

Inspired by a Facebook post, I thought I would measure some of my Spanish 1 students’ writing on the ACTFL scale.

Here is their criteria for Novice High

Writers at the Novice High sublevel are able to meet limited basic practical writing needs using lists, short messages, postcards, and simple notes. They are able to express themselves within the context in which the language was learned, relying mainly on practiced material. Their writing is focused on common elements of daily life. Novice High writers are able to recombine learned vocabulary and structures to create simple sentences on very familiar topics, but are not able to sustain sentence-level writing all the time. Due to inadequate vocabulary and/or grammar, writing at this level may only partially communicate the intentions of the writer. Novice High writing is often comprehensible to natives used to the writing of non-natives, but gaps in comprehension may occur.

Here are some writing samples.  This is Bani’s work, after about 60 hours of C.I. (I do mostly TPRS, along with Movietalk, Picturetalk and some Slavic-style Invisible “untargeted” stories.)

img_0627-1img_0628-1img_0629-1

Let’s see…Bani uses a load of sentences (actually, she uses only sentences). She fully communicates her intentions. There are no gaps in comprehension, The writing is far beyond the “lists, short messages, postcards, and simple notes” that ACTFL says Novice High writers can produce.  So, where is Bani?

Considering her use of various verb tenses, clarity etc, I would say somewhere between Intermediate Mid and Intermediate Advanced. What do you think?

Next, we have Marcus. This kid has an IEP, and has missed about two weeks (~13 hrs) of class.  He has some behaviour challenges, some of which involve staying focused in class.  Here is his most recent story:

 

 

 This is obviously not even close in quantity or quality to Bani’s. He uses English, has some problems with basic verbs, is occasionally incomprehensible, and the story does not really flow.

So, where does this fit on the ACTFL scale? Well, here is their Novice Mid descriptor set:

Writers at the Novice Mid sublevel can reproduce from memory a modest number of words and phrases in context. They can supply limited information on simple forms and documents, and other basic biographical information, such as names, numbers, and nationality. Novice Mid writers exhibit a high degree of accuracy when writing on well-practiced, familiar topics using limited formulaic language. With less familiar topics, there is a marked decrease in accuracy. Errors in spelling or in the representation of symbols may be frequent. There is little evidence of functional writing skills. At this level, the writing may be difficult to understand even by those accustomed to non-native writers.

Marcus fits most of this.  However, he does use sentences, sometimes properly. So– at about 50 hrs of C.I., plus behaviour and learning challenges– he’s at Novice Mid.

The lessons?

  1. C.I. works very well indeed, even for students who are not especially motivated or focused, or who have attendance issues. One of many key C.I. plusses: the vocabulary is constantly recycled in comprehensible but new ways.
  2. C.I. does get the “grammar teaching” done, despite traditionalist “those TPRS kids don’t know grammar” complaints. As we have all experienced, the stereotypically successful  language-class kids– wealthier, whiter  and fairly L1-literate females– will pick up and memorise whatever grammar rules etc we throw at them. The rest, not so much. Bani can’t tell you what a verb is, or conjugate one in a chart, or explain the difference between preterite and imperfect verb tenses…but she can use them correctly and meaningfully. Grammar: my kids havn’t been taught it…but they got it.
  3. C.I. is going to reach kids who would be dead in the water with a textbook. I have had loads of kids like Marcus over the years.  Most of them failed with the text.  Worse, most were disengaged.  Now, I’m not much of a teacher…so if *I* can get Markus this far, anyone can do well!
  4. Anyone who has issues with department members who complain that eg “when I get your TPRS kids in Spanish 2, they can’t write out all the numbers from 555 to 579,” or “they can’t conjugate the verb traer in the pluperfect ablative subjunctive causal declension” can just point at ACTFL guidelines to show where their students are. Verb charts, memorised grammar rules, etc, are not part of ACTFL’s proficiency scales: the ability to write in contextually clear and meaningful ways is.
  5. ACTFL broadly suggests that in a regular (ie non-Immersion) classroom, students will need about two years to get to Novice High, another two for Intermediate High, and two more to Advanced. These writing samples suggest that we can go waaaaay faster than ACTFL thinks.

One last thing:  these kids do well not because Mr Stolz is a brilliant teacher, but because C.I. methods allow us to stay in the target language much more than the textbook does.

 

 

 

 

 

 

The Curriculum That Actually Works For Everyone

Longtime C.I. badass Mike Peto recently weighed in on a good question. Peto, who led his entire department from textbook servitude to effective C.I. (all their kids who write A.P. and I.B. exams pass), addressed this:

We are getting rid of the textbook.  What should we teach? What are people’s favorite stories and themes?

This is a good question.  Standard language teaching is organised around cunningly-disguised grammar units: we learn the present tense verbs with school vocabulary, the conditional a year later with shopping, etc.

My answer: agree as a department to use 250 high-frequency words per year.  Peto’s is better:  focus on “the Super Seven verbs” (Terry Waltz’s list of highest-frequency verbs), along with nine more that add up to what Peto calls “the Sweet Sixteen verbs.”

super72 

Here is Peto’s explanation of how and why his language department organised their language program the way they did.

Short answer: 16 verbs & a commitment to fill every class with compelling CI.

Long answer: As a department chair trying to design a common experience for students in different classes, with a half dozen different teachers on staff, I could have insisted that every teacher follow the same collection of story scripts, movie talks, and novels. That is, “all Spanish 1 students will read X novel and discuss Z movie talk. All Spanish 2 students will acquire this list of target structures so that they will be “ready” for Spanish 3.”

The problem Peto correctly notes here is that we are not guaranteed acquisition of “rule”/word ______ on any teacher’s schedule. Just because you taught it doesn’t mean they got it.

 On the other hand, the Sweet 16 verbs represent a different path towards creating a common experience between classes. We do not simply repeat sixteen words for four years, but we do agree that structures with these verbs are the ones that are recycled and given priority at every step in the journey. The Sweet 16 gives a department the flexibility to allow their teachers and students to pursue different interests in class, to use different language, but guarantees that there will be a common communicative foundation throughout the entire program. For example, the Sweet 16 verbs allow one teacher to develop an independent reading program for her students in which students are all reading different books (and thus developing their own idiosyncratic vocabularies), while another teacher develops his CI skills guiding his students through an authentic telenovela, while another teacher spends a good part of her classes discussing student illustrations and spinning stories from them.

This is how it’s done: you make enough horizontal links– the sweet 16– to ensure that kids in different classes can end up a year later with the same teacher and be able to function.  And you leave it open enough that what kids (and teachers) find interesting can be explored.

We collect everything: novels, Martina Bex units, TPRS textbooks by Carol Gaab as well as the ones by Blaine Ray, our own stories, huge collections of movie talks, telenovelas. We invest in PD like some departments buy workbooks; we brought Ben Slavic to our department to show everyone how to make One Word Images and next summer they are all going to Boston to refresh their skills on classic TPRS techniques. When someone starts using a new item (book, movie, whatever) then that person has the right to claim it for a particular level. Teachers can opt in to use it in their classroom, but nobody is obliged to use any particular item. Therefore, I claimed El Internado for level 1 and Gran Hotel for level 3. Any Spanish 1 teacher could use El Internado. Someone else claimed Fiesta Fatal as a level 1 book… I then included it in my FVR library while my colleague taught it as a whole class novel. A colleague claimed La Vampirata as a Spanish 2 whole class novel, so I did not include that in my FVR library (which is potentially read by Spanish 1 students).

We recognized that in any classroom there will be many different interests, and that when students and teachers are following their own interests then the input is likely to be more compelling. More compelling leads to faster acquisition. That is the funny thing about those studies which try to count how many times a student needs to hear a word to fully acquire it… teachers know that swears might be fully acquired the very first time they are understood, whereas an abstract transition word that the student never uses in their own L1 could be uttered comprehensibly 500 times and not be fully acquired. 

There is another major advantage to running a department this way. When any of my teachers get students at the beginning of the year, we do not have a list of target structures in our minds that we assume our students have acquired. We do not get angry if our level 3 kids do not understand X phrase; instead we are trained to start the conversation assuming nothing and paying close attention to their eyes.

Or, as Blaine Ray so perfectly puts it, “figure out what they don’t know, and start there.

At all levels, as we think about how to phrase our language so that it will be comprehensible, we all return to the Sweet 16 verbs and posters. It is a common experience in all classes, even though I spend a week talking about whales and my colleague spends weeks talking about football (what would you expect from a football coach!).

This illustrates exactly what high-frequency vocabulary does: it provides the flexible foundation onto which people can “stack” whatever vocabulary interests them. The football coach can play around with “el rodilla de Tom Brady se vuelve mejor” while in another class, students can create an OWI who “se vuelve loco por no haber comido.”

Terry Waltz has also famously made this point. She says, imagine you are learning Mandarin Chinese and you are going shopping, and you want to buy a wrench and some apples. If you had to pick one phrase which mattered most, what would it be?  That’s right: want to buy.  This works everywhere.  And ultra-low frequency words (hammer, apples, most clothing words, etc etc) you can figure out on the spot, or gesture, or look up, etc. Waltz elsewhere makes a similar point in response to a teacher who asks shoudl they not know food vocabulary? Waltz, who is translator-fluent in Mandarin (we are talking 10,000 characters plus real-time translation) says, despite having lived for years in Taiwan, that she doesn’t know most of the food words.  Is this a problem?  No, she says, most of the time when I (and Taiwanese) who don’t know the name of something we want to eat just say “I’ll have that” and point. Brilliant. The same is true in, say, Mexico: every region has basic staples– frijoles, tortillas, cerveza– but the local dishes are ultra-low frequency so you just ask about it when you are there. There is no point in teaching Spanish students chapulines, tlayudas, de lechon, etc.

This is also necessary because students move into our district at every level, and we cannot just leave them behind because they did not start with us. We need to provide a comprehensible experience at all levels, even if students missed the first 3 years of our CI  CI program because they were learning thematic vocab in another district.

This is brilliant:  C.I. accommodates everyone, while the textbook does not. The kid from Greg the Grammarian’s class will show up in Peto’s room and go, OMG wut? and probably spend quite a while simply listening.  However, whether or not he has spent four years or four weeks in Grammar Grinder Greg’s class, he will be able to catch up in Peto’s class.

Peto leaves out only one thing: when kids leave his school for Grammarian Greg’s class, high-frequency words will serve more of them much better than anything else.  A teacher (or department) who obsesses about themes and topics is rolling the dice on whether their alumni will end up in a class that uses the same vocabulary. Those who teach high-frequency vocabulary are preparing students for anything.

 

 

Don’t Do This

One C.I.-using American colleague recently shared this section from a Spanish test which their defartment head gave their Spanish class, viz

idiot task

How dumb is this?  Let us count the ways:

  1. Unclear instructions.  Are we supposed to rearrange the words in the sentences, or the sentences themselves, or both?
  2. Some of these have more than one possible answer (way to rearrange words).  Eg c. could be vivir juntos no es fácil or no vivir juntos es fácil.
  3. What does this have to do with actual Spanish that people actually speak or write?  Nothing.
  4. I have never seen a language curriculum that says students will be able to take scrambled words and turn them into sentences.
  5. I’m not sure what they are assessing here.  It’s not comprehension of actual Spanish, since nobody speaks or writes like that.  It’s not output, since students aren’t generating language.

 

This reminds me of those high-school math problems that felt like this:  Suzie is twice as old as Baninder.  When Baninder is twice as old as John, John will be three times as old as Suzie.  How old will Suzie’s dog be on Thursday when Baninder is four? 😉

This is basically a gotcha! question for the grammar geeks.  Yes, you could figure it out, but why bother?

 

Why I (Almost) Never Assess Speaking

So this was asked on a forum recently and, as usual, it got me thinking.


This is a question about “El Internado,” but, really, it applies to anything we do in a language class.  We read/ask a story/do a Movietalk or Picturetalk, etc, and then we want to assess speaking, comprehension, etc.

My response to this question is don’t bother assessing speaking.

But first, a qualifier:  if our Board/school/dept. etc says we absolutely MUST assess speaking, well, then, go for it.  We do what we have to do to keep our job.  But if we don’t have to assess speaking, don’t.  Here is why.

  1. The info we gain from this cannot generally guide instruction, which is the point of any assessment (other than at the very end of the course).  The reason for this is very simple: what will we do if what we learn from assessment varies wildly (which it almost certainly will)? If Samba has problems with the pretérito verb tense, Max doesn’t understand questions with pronouns, and Sky can fluidly ask and answer anything, how are we going to design future instruction around that info?  How are we going to “customise”  reading/stories, etc to give 30 different kids the input they need?  Answer:  we can’t.
  2. This takes forever.  If we have 30 kids in our class, and we can assess them in three minutes each (which is tough) we are spending 90 min alone on speech assessment.  That’s a period and a half!  During this time, we have to design something else for them to do…and good luck having 29 kids– whose teacher is “distracted” by sitting in the corner assessing speech– staying on task for 60 minutes.
  3. We already know how well they speak.  If we are doing regular PQA– personalised questions and answers (basically, asking the class members the same questions we are asking the actors)– we know exactly how well each kid can talk.  So why waste time with a formal assessment?  In my Spanish 1 right now, Ronnie can only do y/n answers to questions, while Emma Watson (aka Kauthr) speaks fluid sentences, and so does Riya, while Sadhna mixes up present and past tense in her output (but understands tense differences in questions) etc.
    Indeed, this is where feedback to the teacher is useful. If—in the PQA moment—I see that Sadhna mixes up past and present in answers, I can guide PQA around that right then and there.
  4. In terms of bang-for-buck, we are going to get way more results from more input than from assessing speech.  We acquire language not by practising talking etc, but by processing input, as Bill VanPatten endlessly reminds us.  I used to do regular “speaking tests” and they did nothing and the info was useless.  Now, I never test speaking until the end of the course, and the kids speak better, mostly because the wasted time now goes into input.
  5. A question that comes up here, regarding assessing speech post-Internado, is, what are we testing the kids on?  Are they expected to remember content— names, events, “facts” etc– from the show?  Or are we assessing speech generally?  In my opinion, “content” should be off-limits: we are building language ability, not recall.In terms of language ability, one of the problems with assessing right after specific content (eg some of El Internado) is that, since this input is generally not very targeted, we don’t have much of a guarantee that the kids are getting enough exposure (in a period or two) to “master” or acquire anything new.  This is to say, while an episode may be 90- or even 100% comprehensible, thanks to the teacher’s guidance etc, it almost does not focus on a specific vocab set.  In a classic T.P.R.S. story, the teacher makes sure to restrict (shelter) vocab used in order to maximise the number of times each word/phrase/etc is used.

    This is whether s/he has a plan, or, as in totally “untargeted” story creation à la Ben Slavic, the kids are totally driving the bus.  As a result, the odds of the kids picking up specific “stuff” from the story—in the short term, which is the focus of the question– are greater (and greater still if the asked story is followed by reading, Movietalk and Picturetalk) than if the input is familiar but untargeted.

  6. What about the kid who missed some of (in this case) El Internado? If the speaking assessment focuses on Internado-specific vocab, it would (in my opinion) be unfair to ask Johnny who was there for all three periods and Maninder, who missed two of three periods, to do the same thing with the “language content” of the episodes.
  7.  Kids hate speaking and tests.  Anything I can do to avoid tests, or putting people on the spot– which a one-on-one test does– I do.  This is what Johnny looks like when you tell him, speaking test tomorrow:Image result for kid being interviewed by teacher
    (image:  Youtube)
  8. “Authentic content” eg El Internado has lots of low-frequency vocabulary. Sure, the teacher can keep things comprehensible, but there is inevitably kids’ mental bandwidth going into processing low-freq vocab…which is exactly what kids don’t need in a speaking assessment, where you want high-freq vocabulary that is easy to recall and applicable to lots of topics.

Anyway…this is why I save speaking assessment until the end of the course: I know how well my kids can speak, I can adjust aural input where it matters– right now–, I don’t want assessment to detract from input, and speaking assessment doesn’t really help me or my kids.

 

 

 

Should I Mark Behavior? The Great JGR Debate, and a Silver Lining for Behaviour Rubrics.

Some years ago, a teacher built a rubric for her C.I. classes, which Ben Slavic named JGR and which was discussed on his blog and then elsewhere.  Here is a version I have played around with: INTERPERSONAL COMMUNICATION rubric.  I initially opposed the use of JGR, then used it, then ditched it, and now I use it (but not for marks). Note: this is a modified version of the original JGR; and I don’t know for how long she used her rubric, or if she still does, or what the original looked like.

JGR was developed because– like all of us, especially me– the creator had some challenges managing her C.I. classes in her initial year with T.P.R.S., which can in (especially my) rookie hands turn into a “woo-hoo no more textbook!” clown show.  JGR basically “marks” classroom behaviour.  JGR specifies that students make eye contact, add story details, ask for help, not blurt, not use cell-phones etc.  Jen used it (and if memory serves Ben also recommended its use) by making part of her class mark a function of behaviour as marked by JGR.  So the kids might get, say, 20% of their mark each for reading, writing, listening, speaking and 20% for their in-class behaviour.  Part of the thinking here was that some behaviours lead to acquisition, while others do not and also wreck the classroom environment, and so “acquisition-rewarding” behaviour should be rewarded.

JGR– for many people, including me– “works.”  Which is why– especially when linked with allegedly “acquisition-promoting” behaviours– lots of people are interested in it.

JGR is a kind of “carrot-and-stick” marking tool:  if the kids engaged in the behaviours JGR specified, their marks went up, partly because (a) they got marks for those behaviours, and partly because (b) the behaviours should– in theory– help them acquire more language.

This can of worms was shaken around a bit on Ben’s blog, and recently, thanks to the always-remarkable Terry Waltz, there have been FB and Yahoo discussions about it.  So, today’s question:

Should we assess in-class behaviour for final marks purposes?

My answer: no, never.  Why?

1. Behaviours typically asked for in JGR– or other such rubrics– are not part of any     curricula of which I am aware.  Every language curriculum says something like, students of the Blablabian language will read, write, speak and understand spoken Blablabian, and maybe say something about Blablabian culture.  Nowhere does any  curriculum say “students should suggest details for stories” or “students will lookthe teacher in the eye.”

If it’s going to get a mark, it has to be part of course outcomes.  Any assessment guru (Wormelli, Harlen, etc) will tell you the same thing: we do not mark attitude, behaviour, homework, etc, as these are not part of final outcomes.

To put it another way, how do we judge the New England Patriots football team?  By how well, often and/or enthusiastically they practice and look Bill Belichick in the eye, or by how many games they win?  How should Tom Brady be paid: by how often he shows up for practice, and how nice he is to Belichick, or by how many yards he successfully throws?  That’s right.

We could– and I often do– end up in situations where a “bad” kid does well, or a “good” kid does poorly.  I have had bright-eyed, bushy-tailed teacher’s pet-type kids who were not especially good at Spanish, and I have had giant pains-in-the-butt who were quite good.

My best-ever student in TPRS, Hamid Hamid, never added story details, never looked up, and always faced away from the board.  Yet he CRUSHED on assessments and got 100% in Spanish 2.  Two years later, his younger brother, Fahim (also a great student) told me that Hamid Hamid was both shy and deaf in his left ear, so always “pointed” his right ear at the board (and so appeared to be looking away).  This kid’s mark would have been lowered by assessing his “in-class behaviour,” which– given his epic Spanish skills– would have been absurd.

2. As Terry Waltz points out, neurodivergent kids can– and do– acquire language without engaging in many behaviours typically required by participation and behaviour rubrics. She also points out that forcing neurodivergent kids into the “normal” mold is at best less than productive. If you are autistic, anxious, suffering from PTSD (as my stepdaughter does) or facing any other neuro challenges, “engagement” rubrics can make your life miserable while not appreciably meaningfully measuring what you can do with the language.

3. The only thing required for language acquisition is reception of comprehensible input.  While the focus of behaviour rubrics is designed to get kids to tune in, it does not follow that many behaviours which do make for a good class– e.g. people adding good details to stories, looking at each other– are necessary to acquire language.

All of us have been there: you have a plan, you did your story warmup or whatever, but the kids aren’t into it.  You bust out a Movietalk but they aren’t into that either.  Dead class. Now, in a C.I. class, we don’t have recourse to worksheets or whatever, and we still have to teach the language. I have a bail-out move here: direct translation, and I always have a novel on the go, so I can read aloud, and Q&A the novel.  If I’m being particularly non-compelling, I’ll throw an exit quiz at them.

The point: if the kids are getting C.I., they are acquiring.  If they are miserable/tired/bored with stories, fine.  They are gonna get C.I. one way or another.

4. Any kind of behaviour rubric plays the awful “rewards” game.  Ask yourself this question:  why do I teach? The answer– other than because I have to make a living— is probably something like, because it’s interesting, I have some measure of control over my work, and I love kids and my subject.  Some will add that teaching, properly done, opens doors for kids.  Teachers do not teach because they want to be evaluated, or because they want to use the latest gizmo, or because they want public praise, etc.  They are, in other words, intrinsically motivated.  They want to work because the work is good and worthy in itself.

When we institute rewards for behaviours, as Alfie Kohn has spent a career arguing, we destroy intrinsic motivation.  We turn something interesting into payment for marks.  The point stops being paying attention to the story– or adding to it cos you actually care about it– and becomes something rote.

5. Using behaviour rubrics can dampen professional self-examination. If my practice is such that I have to use marks as a stick to keep kids in line (the policing metaphor is not an accident), there are two possibilities: tough kids, and/or I am doing a bad job.  The question why are they not tuned in? might be answerable with any of the following:

— I am not being sufficiently comprehensible

— I am ignoring the top or the bottom end of the class– too fast/slow or simple/complex

— my activities are not interesting, varied or meaningful enough

— the kids see no purpose

— accountability: they don’t see tuning in as something that results in real gains

— I lack basic skills (smart circling, control of vocab, etc etc)

— my story sucks 😉

I had better be able to look in the mirror, consider and then deal with these possibilities, rather than merely acting like a cop and demanding obedience.

Now, behaviour does matter.  You cannot run a T.P.R.S. class without rules etc.  My basic rules:

  • no phones or other distractions (including side-talk, blurting etc)
  • no insults of anyone other than oneself or of rich entitled people
  • listen, watch and read with the intent to understand; ask when you don’t
  • do not create or engage in distractions

The tools that we have for dealing with distracting behaviour include

  • warning the offender, standing by their desk, calling Mom and Dad, etc
  • pointing, with a smile, to classroom rules every time there is a problem
  • sending them to admin if necessary
  • taking their phone until 3:15 (most kids would rather die)
  • detention, where we discuss behaviour
  • assigning read & translate (quiet seatwork)
  • taking the kids outside for a walk, or doing some other kind of physical brain-break
  • changing activities
  • doing a quiz
  • talking to kids one on one and asking what do I need to do to get you focused?

 

The upshot?  We should not, and need not, mark “behaviour” or “participation.”

 

Addendum:  is there ever a use for classroom behaviour rubrics?

Yes.  I get my kids to self-evaluate using JGR every 2-3 weeks.  My version generates a mark out of 20.

Nineteen out of twenty kids will very honestly self-evaluate their behaviour, provided they understand exactly what is expected.  One kid in twenty will heap false praise on him/herself.  For the false praisers (“I never blurt in class!”), I sit them down and explain what I think, then we agree on a more realistic mark.

I save these JGR “marks” and once in a blue moon, when a helicopter parent or an Admin wants to know, how is Baninder doing in Spanish, I can point to both the spreadsheet with Numberz and JGR.  This frames the inevitable discussion about marks in terms any parent can understand.  Any parent, from any culture, understands that if Johnny screws around and/or does not pay attention in class, his mark will drop.

JGR– in my experience– accurately “predicts” the marks of about 80% of kids.  When I can show a kid (or a parent or admin), look, here are Johnny’s marks AND Johnny’s own description of how he behaves in class, we can have an honest discussion about marks, Spanish, etc.  Win-win.

How Do I Assess and Evaluate Speaking?

A teacher asked on the Facebook group “How do you assess speaking?”  Responses were basically, “try using one of various apps” (i.e. Google Classrooms, KaBlaBla, etc).  Lots of people want to use tech to do it.

Contrarian here:  save yourself time, and don’t bother…you can accurately, quickly and easily assess speaking with zero tech.

But let’s revisit the basics before we go on:

  • speaking does not improve language acquisition.  The act of talking is not like practicing music or baseball.  The real driver of speech is aural (and written) input.
  • Teachers need a life.  I for one refuse to spend an hour per class listening to students’ prepared recordings of prepared questions.  The kids have better things to do, and so do we.
  • The only speech we should assess if we want to see what the kids have acquired is spontaneous and in-the-moment.  If you want people to learn a language, then by all means let them plan, rehearse, etc…but don’t confuse this with acquisition, where we see what is “wired in” and gut-level, below– and beyond– the conscious mind.  Most of the ed apps I’ve seen are similar:  teacher records their voice asking question or saying prompts; kid listens, decodes and responds and records their answer for teacher to mark.  This kind of “planned” or “reflected-on” communication doesn’t really assess what they have acquired.
  • Feedback doesn’t work.  You can explain, correct, suggest, etc till the cows come home and it won’t make a difference in how well the kids speak.  Only input can really change that.

So how do I assess speaking?

First, every time a kid opens their mouth and uses the target language in class– to answer a question, to add to a story, etc– you are getting perfect feedback about how well they speak.

So with my 2s…Aashir can say– and understand– a word at a time max.  Simrowdy can answer any question and talk at length about anything.  Sadjad extemporaneously comes up with good entire sentences when adding to a story.  Janelle is like Simrowdy.  Daniel will– and does– say anything but has some verb etc issues.  Kevin never talks, but when he does, it’s perfect.  I could go on.

Second, the point– to me– of assessing speaking (as with anything else) in class is to see what the kids do not understand and where they need more input.  This is why we track barometer kids and choral responses.

Third, I don’t play “gotcha.”  I test what I teach.  I use vocab they know, and when using objects, pictures or people, I make sure the kids have the vocab to describe them.

Fourth, I don’t assess speaking for Level One students.  It makes them anxious, and it is time taken away from input.  I assess– i.e. attach a number to– kids once, at the end of Level Two.  I do only what you would do speaking in real life:

  • ask them questions and have them answer
  • have them ask me questions (and I answer).
  • describe something– a photo, an object, another kid in the class

No presentations, storytelling, memorisation, etc.

Here’s my rubric:

For a mark of 3:

  • I can in detail discuss myself, my social and family circle and my activities and interests, and i can describe things.
  • I make minor mistakes that do not affect meaning, and I can speak fluidly.
  • I understand all questions and I come up with my own.
  • I can fix conversational problems or I don’t have any conversational problems.

For a mark of 2

  • I can discuss myself, my social and family circle and my activities and interests, and I can describe things.  There are some gaps in what I can say, and sometimes I provide little detail.
  • I make enough mistakes that meaning occasionallyu breaks down, and I can speak but not quickly nd fluently
  • I understand most questions and I come up with some of my own.
  • I sometimes fix conversational problems.

For a mark of 1:

  • I can  discuss myself, my social and family circle and my activities and interests, and i can describe things, but I can’t do so with much or any relevant detail.
  • My mistakes affect meaning, and I generally don’t use sentences.
  • I don’t understand all questions and I have trouble coming up with my own.
  • I either don’t know when there are conversational problems, or I don’t bother fixing them.

“Conversational problems” means not understanding, and “fixing them” means asking for a repeat, etc (i.e. not just bobble-heading along).

This rubric will generate numbers from four to twelve out of twelve.    I’ll generally show them a pic on the iPad at some point, and have them describe that, or have them describe a kid in class.  it takes about 4 minutes per kid to do this.

If the kid bombs, they can come back in after school and re-do it.

Anyway, there’s my thoughts.

How Do I Do Dictation?

Dictation is old as the language-teaching hills.  I remember doing this in my various français classes and also in Cherman viss Frau Satcher, ja.

Now, dictée is like running, weightlifting or learning music: if you do it wrong, the results are frustrating, painful or non-existent.  Like everything else in language teaching that we still do from 60 years ago, dictado has been tweaked.

Dictation:

  1. teacher reads 5-10 inter-related and meaningful sentences aloud
  2. these sentences should use the most-recently-taught vocab
  3. students write down what they hear.
  4. teacher can repeat each 2-3 x
  5. Ben Slavic suggests then projecting the sentences on the overhead and having kids correct their spelling.
  6. I always finish with translate into L1.

 

What Diktation should not be:

  • a way of introducing new vocabulary and/or grammar
  • a way for students to “practise” spelling
  • isolated meaningless sentences (e.g. sentence #1 is “the boy is tall” and #2 is “it is raining”)
  • to any degree incomprehensible
  • graded for spelling to any significant extent

I use dictation as assessment, not as a vocab-acquiring activity, and I follow the 80/80 rule: 80% of class needs to get 80% or more for me to move on. Scores are usually quite high, as I do dictation after asking a story and doing readings of the story, or other stories using the same vocab.  The kids don’t complain, it is a zero-prep activity, and it is 90% C.I. as the kids know the vocab.  This is mostly how I get listening marks.  Although it looks like output, mostly it’s input– listening– and the output is in L1.  As James Hosler says, dictation while being assessment is another way to deliver C.I.

Here is what I am going to do for dictation on Monday in Spanish 2.

  1. There was a grandmother who wanted to give her grandson money.
  2. She told him “you must win a a dance contest”
  3. He said “I don’t like to dance, because I am lazy.”
  4. The grandmother wanted to give her money to Donald Trump.
  5. Donald Trump did not want her money.

You could also

  • project a picture and describe it
  • describe a character from your novel or story
  • project bits of a clip from a video, narrate a sentence, and have them write that

Level Two Spanish Results: First Picture Description

Spanish 2 has been running for three weeks.  We have read a couple of easy novels, and have done one story cycle: el restaurante, which included reading, storyasking, Movietalk (Mr Bean videos RULE!) and Picturetalk.

Today’s first writing assessment: describe this photo.

  
Here are four writing samples.  The kids had 5 minutes, no notes or dictionaries.

First, Janelle, the top student. Amazing how she mixes past and present appropriately.

 

Next, we can compare two Level Two students who are not top performers. Hassan went to another school last year, where he had traditional grammar-and-textbook-based teaching.  This is garbled and nonsensical.  Hasan has some learning challenges and struggles in other classes.

Next, Abbas, who like Hassan has some challenges and struggles with school.  This is not awesome…but we understand, and he has built in a bit of a backstory.  Abbas had only TPRS in Level One.

Based on results, C.I. clearly helps the challenged kids more than does a traditional text.

   
Finally, Amneet.  This is not very good writing.  What is interesting here is that Amneet is probably the best speaker in the class.  I have found this kind of thing typical:  while most of the kids can undertsand everything (the scores for reading and listening quizzes are all between 80-100%), production skills vary dramatically from kid to kid and medium to medium.  Writers are not necessarily speakers, and vice-versa.

Amneet arrived late in Spanish 2, has missed a bunch of classes, but did well last year (over 80%) so I am expecting her written output will pick up.