C.I.-taught Students Evaluated by A.C.T.F.L. Writing Standards

How well do C.I.-taught students do in terms of ACTFL writing standards? Well…pretty darned well, I’d say.

Inspired by a Facebook post, I thought I would measure some of my Spanish 1 students’ writing on the ACTFL scale.

Here is their criteria for Novice High

Writers at the Novice High sublevel are able to meet limited basic practical writing needs using lists, short messages, postcards, and simple notes. They are able to express themselves within the context in which the language was learned, relying mainly on practiced material. Their writing is focused on common elements of daily life. Novice High writers are able to recombine learned vocabulary and structures to create simple sentences on very familiar topics, but are not able to sustain sentence-level writing all the time. Due to inadequate vocabulary and/or grammar, writing at this level may only partially communicate the intentions of the writer. Novice High writing is often comprehensible to natives used to the writing of non-natives, but gaps in comprehension may occur.

Here are some writing samples.  This is Bani’s work, after about 60 hours of C.I. (I do mostly TPRS, along with Movietalk, Picturetalk and some Slavic-style Invisible “untargeted” stories.)

Let’s see…Bani uses a load of sentences (actually, she uses only sentences). She fully communicates her intentions. There are no gaps in comprehension, The writing is far beyond the “lists, short messages, postcards, and simple notes” that ACTFL says Novice High writers can produce.  So, where is Bani?

Considering her use of various verb tenses, clarity etc, I would say somewhere between Intermediate Mid and Intermediate Advanced. What do you think?

Next, we have Marcus. This kid has an IEP, and has missed about two weeks (~13 hrs) of class.  He has some behaviour challenges, some of which involve staying focused in class.  Here is his most recent story:

 

 

 This is obviously not even close in quantity or quality to Bani’s. He uses English, has some problems with basic verbs, is occasionally incomprehensible, and the story does not really flow.

So, where does this fit on the ACTFL scale? Well, here is their Novice Mid descriptor set:

Writers at the Novice Mid sublevel can reproduce from memory a modest number of words and phrases in context. They can supply limited information on simple forms and documents, and other basic biographical information, such as names, numbers, and nationality. Novice Mid writers exhibit a high degree of accuracy when writing on well-practiced, familiar topics using limited formulaic language. With less familiar topics, there is a marked decrease in accuracy. Errors in spelling or in the representation of symbols may be frequent. There is little evidence of functional writing skills. At this level, the writing may be difficult to understand even by those accustomed to non-native writers.

Marcus fits most of this.  However, he does use sentences, sometimes properly. So– at about 50 hrs of C.I., plus behaviour and learning challenges– he’s at Novice Mid.

The lessons?

1. C.I. works very well indeed, even for students who are not especially motivated or focused, or who have attendance issues. One of many key C.I. plusses: the vocabulary is constantly recycled in comprehensible but new ways.
2. C.I. does get the “grammar teaching” done, despite traditionalist “those TPRS kids don’t know grammar” complaints. As we have all experienced, the stereotypically successful  language-class kids– wealthier, whiter  and fairly L1-literate females– will pick up and memorise whatever grammar rules etc we throw at them. The rest, not so much. Bani can’t tell you what a verb is, or conjugate one in a chart, or explain the difference between preterite and imperfect verb tenses…but she can use them correctly and meaningfully. Grammar: my kids havn’t been taught it…but they got it.
3. C.I. is going to reach kids who would be dead in the water with a textbook. I have had loads of kids like Marcus over the years.  Most of them failed with the text.  Worse, most were disengaged.  Now, I’m not much of a teacher…so if *I* can get Markus this far, anyone can do well!
4. Anyone who has issues with department members who complain that eg “when I get your TPRS kids in Spanish 2, they can’t write out all the numbers from 555 to 579,” or “they can’t conjugate the verb traer in the pluperfect ablative subjunctive causal declension” can just point at ACTFL guidelines to show where their students are. Verb charts, memorised grammar rules, etc, are not part of ACTFL’s proficiency scales: the ability to write in contextually clear and meaningful ways is.
5. ACTFL broadly suggests that in a regular (ie non-Immersion) classroom, students will need about two years to get to Novice High, another two for Intermediate High, and two more to Advanced. These writing samples suggest that we can go waaaaay faster than ACTFL thinks.

One last thing:  these kids do well not because Mr Stolz is a brilliant teacher, but because C.I. methods allow us to stay in the target language much more than the textbook does.

 

 

 

 

 

 

Spring 2017 Spanish 1 Results

Here are two story writes. Students had 25 minutes to write a story. No notes, no dictionary, no questions: write what you can en español.

These kids have had about 35 hours of input, zero writing/talking practice, very few grammar pop-ups, no workbook etc exercises.  As close to pure comprehensible input as you can get before moving into story listening.

First, Kaye. She is originally from the Philippines, and Spanish is her fourth language. I’m including her story because

A. it’s good
B. Kaye likes Duolingo…you can see her playing around with words I havn’t used.

Next, Bani. She is from the Punjab, and Spanish is her 4th language. What can I say? She rocks!

 

Note: not all students did this well. For the kids who are there for 19/20 classes and who pay attention, the average wordcount was about 175 and grammar mark 2.5/3. The skippers and burnouts do much, much worse.

Some Notes on Level 1 Results (Fall 2017)

Meaningful discussion of language teaching is kind of like language learning itself: you need a thing to discuss. Language acquisition starts with stuff: language to be processed.  Language teaching discussion starts with student results.  This semester I have two English 9s, Social Justice 12 and intro Spanish, so I have too little time to “curate” results regularly.  Feb 2017 I have both more 1s and time so I will publish more results then.

Bott…ehhhmm, as they say in Ireland, here are two interesting recent story writes.  30 min, no notes or dictionaries, paper-and-pen only.  These students have had about 50 hrs of input.  Instruction is almost pure C.I.: no metacognition, grammar practice, talking practice, writing practice, K/W/A activities, peer group work, bla bla.  Just a lot of input.

This is the fourth story they have written.  Word target was 400.  Most kids are below average this year.  This is possibly because I am doing mostly Slavic-style “untargeted” input, where we have much less repetition of specific vocabulary targets in the short term.

First, here is Angela’s story.  Ang is Filipina and still has a bit of an accent.  She has been in Canada for two years, and reads a lot.  She is also a Duolingo user.  She tells me she is putting in about four hours/week.  Check it:

There is a bunch of stuff I havn’t used in there.  She is generally getting it right in terms of meaning but there are still grammatical holes.  I’m including this because, yes, Duolingo– which frankly bores the crap out of me– does deliver decent C.I.  Krashen noted in a paper that Duolingo works about as well as traditional college Spanish instruction, if the user can manage to stay interested, which most do not.  Angela says she likes the new story feature on Duolingo.

Next, we have Nisha.  She is Punjabi, and has had zero previous Spanish. Lotsa words, a good story, but some obvious issues (eg noun gender).  What is tough for the Punjabi kids (in writing) is that adjective and noun endings in Spanish– -a indicates feminine– indicate masculine in Punjabi (e.g. bacha = boy, bachi = girl, bache = kids) so we get confusion. Also in Punjabi (as nearly as I can tell) there is a lot less verb conjugation (or maybe I just can’t hear it when I ask them about it). Nisha is not a Duolingo user.  Her only Spanish is at school.

But anyway, props to Nisha for doing so well after only 50 hours.

 

More Notes on Feedback

Amy Lenord started a great Twitter discussion about how one encourages language learners to process language.  This eventually led to Martina Bex refering us to her excellent “I am a grammar geek” post, in which she talks about how much she loved– and found effective– the “red ink” from her Spanish profs in Uni. Bex and I very briefly discussed this.  (I will bet that when she has a spare moment– and she is a Mom again, congrats!– she’ll discuss this more.  Ha!)

Now, anyone who knows Bex knows that the basic deal with her is that what she wants done, she gets done.  Bex wants babies? Bex has four (at last count).  Bex wants to acquire Spanish?  Bex signs a months-long “no English” agreement with her room-mate!  Bex wants to master C.I.?  Bex does, in like two years of teaching.

So it is not surprising that she acquired a ton of Spanish in very short order in Uni. 

Again: she wanted, liked & felt she benefited from corrective feedback  in her Spanish classes. 

This raises two questions:  did the feedback she got actually help her, and, if so, why and how?

Well, let’s take Martina’s word for it, and say, sure, corrections and comments helped.  Now, how?

Well, suppose young Bex– or anyone else– wrote this on their Spanish 201 composition:

*  Ayer, yo fue al cine con mis amigos, y vimos una película.

This should be “yo fui,” and say her prof writes that on her paper.  Now, what happens next?

1. Bex notes there is an error.
2. Bex re-reds the sentence: yo fui al cine.

Most of our students will not even do #1.  Most will go straight to the mark, wondering  what did I get?  did I get an A?

Some will note, ok, there was an error.

A very few will re-read the corrected sentence, and maybe linger on it, in which case it is functioning as good comprehensible input (albeit not many repetitions).

So, why is the feedback working for Bex?  In my view, it is because

a. Bex is majorly motivated which means,

b. Bex wants feedback, and when she gets it,

c. the feedback provides comprehensible input.

Suppose the prof had written “ser takes an -i in the first-person singular.”  Would this have done Bex any good?  The research says no.  Maybe for Bex it did.  Maybe she went, hmm, yo fui al cine…

I was also recently talking to Adriana Ramírez and Luce Arsenault about giving corrections in their Sp and Fr classes.  Both maintained that their kids got better as a reuslt of having to do corrections.  They havn’t obviously had time to do a controlled study, but we noted a few things:

1.  Both have very motivated, mostly Asian and wealthy white kids, who have been hearing from their literate parents from Day 1 of school, memorise (for many Asian kids, who have had to learn zillions of Chinese characters before coming to and sometimes while in Canada), and edit (for wealthy white kids, whose parents are uber-literate, professional, etc).
2. My kids– who are generally Hindi, Punjabi and Urdu-speaking, and have less-literate and generally non-English speaking parents, almost none of whom have any formal experience learning additional languages– have not been primed to memorise and relentlessly improve their work.  This is not to say that our parents do not value education– they do, very much– but it is to say that they have not “acquired” some of the academic habits that can sometimes for kids in language classes.

There is a simple lesson here:  unless people want feedback, and get it, and the feedback is comprehensible input, it is not going to do any good.

So the teacher should focus not on marking and correcting, but on relaxing and reading and being happy in their spare time, so when they show up in class, they have the energy and mood to provide good C.I.– in story asking or reading or MovieTalk form– for kids.  And kids should not be forced to correct work (although if they want to, why not?).  Rather, their work should be hearing C.I. in class, and– if they must have homework– reading or viewing comprehensible and interesting target-language stuff.

 

 

 

Level Two Spanish Results: First Picture Description

Spanish 2 has been running for three weeks.  We have read a couple of easy novels, and have done one story cycle: el restaurante, which included reading, storyasking, Movietalk (Mr Bean videos RULE!) and Picturetalk.

Today’s first writing assessment: describe this photo.

  
Here are four writing samples.  The kids had 5 minutes, no notes or dictionaries.

First, Janelle, the top student. Amazing how she mixes past and present appropriately.

 

Next, we can compare two Level Two students who are not top performers. Hassan went to another school last year, where he had traditional grammar-and-textbook-based teaching.  This is garbled and nonsensical.  Hasan has some learning challenges and struggles in other classes.

Next, Abbas, who like Hassan has some challenges and struggles with school.  This is not awesome…but we understand, and he has built in a bit of a backstory.  Abbas had only TPRS in Level One.

Based on results, C.I. clearly helps the challenged kids more than does a traditional text.

   
Finally, Amneet.  This is not very good writing.  What is interesting here is that Amneet is probably the best speaker in the class.  I have found this kind of thing typical:  while most of the kids can undertsand everything (the scores for reading and listening quizzes are all between 80-100%), production skills vary dramatically from kid to kid and medium to medium.  Writers are not necessarily speakers, and vice-versa.

Amneet arrived late in Spanish 2, has missed a bunch of classes, but did well last year (over 80%) so I am expecting her written output will pick up.

 

My Marking System for a Language Class Summarised.

“How do we mark kids in C.I. classes?” asks a C.I. newbie on the Yahoo listserv.  Assessment and evaluation are always hot topics.

First, definitions.  Assessment is observing how people are doing, and based on that observation

1. changing what you the teacher do and
2. giving students feedback in order to
3. improve student performance.

Assessment in a comprehensible input classroom means checking whether individual students (or the class) understand both statements and questions, and– if they don’t– clarifying. Weak choral responses?  No response?  Wrong answer?  Go back and clarify. The same goes for individual kids.  If Johnny can’t answer the question “What does ___ mean?”, go back and re-explain.  BOOM! that’s assessment.

Evaluation is assigning a number to student work based on a student’s performance in relation to standards (criteria).  This we do at the end of instruction (ie when a story cycle is done, or at the end of the year).

Second, principles.  If you havn’t been in-serviced about “assessment for learning,” google it, I can’t explain it all here, but let’s make three things SUPER CLEAR.

A.   We only evaluate “final products,” not practice, homework, behaviour, attitude, etc.   The long and the short of this surprisingly controversial statement is that we expect our kids to do X, Y and Z, and the mark we ultimately assign them at the end of Year ____ of Blablabian Language Class should reflect how well they do X, Y and Z, baseed on a set of criteria.  Whether or not Johnny is nice in class, and whether or not Suzy does her homework are irrelevant: what matters is how well they do in relation to criteria.

B.  We only evaluate what has been taught.  No random new vocab, no activities on tests the kids havn’t done, no gotcha! games etc.  You test what you teach, period.

C. We always let ppl re-do work so they can improve their mark. In my class, this means that their mark is always their most recent writing, reading and speaking mark.

Assessment should be fast, simple and minimal, because people learn from interesting comprehensible input and not from testing and feedback, and because teachers need to see their families & have a life 👍.  

Now, here is what I do.

DURING THE COURSE

My instruction is very loosely built around story cycles. I will ask a TPRS-style story, and add to that Movietalks, Intro Routine, Story Listening, Picturetalk, reading, etc. When we are “done” with one story and its associated vocab, I evaluate. Evaluation for a story cycle includes the following:

1. Two listening quizzes. For this, I read a 5-7 sentence story that I make up on the spot aloud. The kids listen, write down the Spanish, and translate into English. This will be 33% of the story cycle mark. To evaluate these, they basically get a mark for getting the gist of each sentence. I don’t “grade spelling” unless the spelling makes their writing incomprehensible.
2. One reading quiz. I will give the kids something to read, and get them to translate it. Whatever it is, it has to be 98% comprehensible. Usually, I will give kids a version of something we have created in class, or something similar to what we made in class. To evaluate this, I will randomly pick three sentences and see how well they understand. They will get a mark out of three.
3. One Story Listening grade. I do Story Listening on Fridays. I get the kids to summarise a printed version of the story and hand that in.  They almost always get 2.5/3 or 3/3 on this. This goes into the reading category. If a story cycle takes more than a week, sometimes they get two SL grades added to their reading.
4. One 5-min picture description write and one story write. This is how I mark writing.
6. Other stuff: I can also grade comics for a reading mark. If we are reading a novel, I can copy a few sentences from a chapter and get the kids to translate that for a reading mark. You can easily get reading marks from Textivate, and I also use Sr Wooly scores sometimes for reading marks.

So, at the end of a story cycle, I will have two marks for writing, two for listening, and one or two for reading. Each category is worth 33.3% of grade. I print marks out & post them online.

Now, if kids have unexcused absences, or they didn’t do the comic or Q&A for stories, etc., they will get a much lower grade than they expect. If the grade is low, I tell the kid or their parent “Baninder is missing X, Y and Z. If she completes these next story cycle, she will probably get a higher grade.” So the kids know that the posted grade is their grade right now. They also know that they can change their grade by paying attention.

When the next story cycle starts, the gradebook resets.

AT THE END OF THE YEAR

In my course, the final exam is 100% of the mark. I tell kids and parents that their marks along the way are a rough indicator of how well they are doing.  At the end of the year, we do the following:

1. First class: they have a listening test where I read a longer story (15-20 sentences) aloud.  They have to write down the Spanish and translate. Then, we have a reading test: they get a written text they have never seen, and translate it into English. I randomly pick 5 sentences and grade those for each of reading & listening.

2. Second class: they have three writing tasks.  First, they have 5 min. to describe themselves.  Second, I project a picture, and they have 5 min. to describe the picture. Third, they have to write a story.

3.  For speaking, I do this.

After seven years of C.I., I have found this system clear, simple, low-work and low-stress for the kids. To sum up,

• students are always marked on their most recent work
• no oral evaluation for beginners
• this is real “assessment for learning” where only final products are evaluated with reference to criteria
• there is no marking of attitude, homework, participation, etc.  If kids screw around, or don’t work, I deal with it– but not by using marks as carrot or stick

How do I mark writing? Blaine Ray’s ideas, slightly modified

Stephen B.– who after twenty years of traditional grammar teaching jumped headfirst into C.I.; how bad-assed is THAT?; writes

“Anyways, I had a question about grading speed/timed writings. I know Blaine says one point per word and he talked about scaling and not to mark it for accuracy. However, what does one do in this situtation: for example, if after a 5 minute speed write one students writes 85 words but is all over the place and makes several grammatical and spelling errors, and another student only writes 60 words but it is almost perfect, how can I give the former student a higher mark?”

Here is how I do it (with many suggestions from Adriana Ramírez, whose Teaching Spanish Through Comprehensible Input text I am using this year). First, classic TPRS in a story cycle, with movietalk and picturetalk:

A) story cycle: establish meaning, ask story (with a few parallel characters), review, retell, read a couple of versions of story

B) have the kids create a comic of the story. Story must be “complete” but obviously not everything can be put into comic. Story must be coloured, look awesome, etc (clip art fine). Each panel must

• have at least one Spanish sentence
• have perfect alignment between Spanish and pictures
• where there is no dialogue, “thought bubbles” in the first person
• where appropriate, have dialogue

This will make the kids read, choose sentences, and clarify meaning via illustrating.

C) movietalk and picturetalk to support story structures (e.g. if you taught “wants,” movietalk and picturetalk a video where a person wants something)

Then, for assessment, I am doing the following:

1) when the “story cycle” (A-C above) is done, kids will do a five-minute speedwrite and a forty-minute relaxed write.

Their first speedwrite topic will be “describe yourself.”

The speedwrite is evaluated in 2 ways:

First, wordcount. Kids count the # of words in their composition (tell them no lists, or, if they want a list, they must describe all things in the list). End of year goal: 100 good words in 5 min. For their first speedwrite, they get a 40-word bonus. So if they write 30 words, their wordcount score is 70/100.

Second, they get a grammar mark out of 3, thus:
1– it’s full of mistakes and largely incomprehensible
2– it’s mostly comprehensible but has some “whaaat?” moments and “feels” junky
3– it’s fully comprehensible, has no “whaaat?” moments, and “feels” fluid and solid (but not necessarily perfect)

Multiply their grammar mark by 33.3 and they have a grammar mark /100.

Now, average the two marks and they have a spedwrite percentage.

For the 40-min relaxed write, I tell them “either retell the story, or write your own, and you must have 3 dialogues, but put changes into your version of the story.” The goal for the year: write an 800-word story in 40 min. For their first story, I’ll expect 70-150 words. I will assign a wordcount mark out of, say, 200 and give them a 50-word bonus. I will also give them a grammar mark /3 above.  Every time they write a story, the amount of words expected goes up and the curving bonus goes down. 

We average their grammar mark and wordcount: if Johnny gets 2/3 for grammar, and writes 90 words, his score is 66.6% (grammar) + 140/200 (70%) for wordcount = 69%.

After we do the second story of the year (and until the end of the course), we repeat the procedure, with a few changes

A) the speedwrite bonus drops by 5 words each time
B) the relaxed write bonus drops by 5 words each time and the “benchmark” goes up by 75 words. By end of year kids should be able to write 800 words in 45 min.
C) we use another topic for the speedwrite for the second time: describe a picture that you project onto your screen. This picture should support what was in your story. So, if the story had a girl who wants an elephant, your picture could be a boy who has an elephant.

D) for the third speedwrite, use topic #1 (describe yourself). For the fourth, use a picture. Keep alternating.  I use fully unsheltered grammar from Day 1 (all verb tenses, subjunctive, etc) so the picture describing tests evaluate how well they can use present tense. 

The writing will improve during the year. As I write this after having done only two stories, wordcounts are WAY up and grammar is also improving.

A few notes:

— you MUST carefully restrict vocab. This has been my single-greatest problem with TPRS: adding vocab at random. If you don’t restrict vocab, you get fewer reps on each item…and worse/less acquisition.

— initially, the kids will generate pretty crappy stories. Later, word count goes up and grammar will get better. Some kids will re-write the story; most will start to improvise.

— their “mark” at any given time is simply their most recent speedwrite and relaxed write mark, combined.  I also do exit quizzes for listening and reading (1 each/week) so I have a pretty good overall picture of how everyone is doing. 

— PQA is super important. Adriana’s book has a list of “personalised questions.” The Blaine Ray books also do. If you are doing your own stories, you make them up. Personalised questions are super-easy: you basically ask the class the questions you ask the actor.

So if you narrate “the boy liked running,” you ask your actor “do you like running?” and he says “yes, I like running.” You then ask “do you like vomiting?” (something contrastive) and he says “no, I do not like vomiting.” Then, starting with your superstar, you ask the class members “do you like vomiting or running?” etc. Simple.

This is important because the kids need to hear the present-tense forms.

— Adriana’s advice was to make sure all the kids do the comic. This is because the comic writing is “deep reading:” it makes the kids re-read, choose, copy and write, etc. For the non-artists, translation also works: copy story, underneath it translate (diff coloured pen), leave a blank line to keep it clear. Here is a pretty good example of “Los Gatos Azules” turned into a comic (one of Adriana’s kids did this one):

Anyway, this is how I have organised the “units” of TPRS and how I assess. Coments, as always, welcome!

How do we grade speedwrites and freewrites?

Here is another question from Sarah-Beth:

Q:  “I was also wondering how you score the Relaxed Writes? Is it a similar process? I’m not sure what to put the total out of (same with Timed Writes). The marks seem really inflated when I factor in the 40 to top them off and I’m getting over 100% on some students, which are totally skewing their marks. What do you suggest? Merci!”

A:  These are Blaine Ray’s ideas.  First, announce course goals.  For beginners, write 100 coherent words in 5 min; write a 500-word story in 3 verb tenses in 30 min (by end of course).  I use 2 kinds of writing during the year– speedwrites and timed writes.  I start writing after about 20 hours of CI.

A speedwrite is, the kids have 5 minutes to write.  They have to produce as many words as possible during that time.  Then they count their words.  The only “rules” I have are no lists, and focus.  I start grading these with a “handicap” system.  The freewrite is out of 100 and the kids’ scores are the # of words they write, plus the handicap.  First time out, the kids will typically write 20-60 words.  Add 40 to that, and you get a score of 60-100.  Next time, two weeks later, do the same thing…but add only 30.  By the 4th month (semester block system of 6.25 hrs/week) kids should be easily hammering out 100 words in 5 min.

The speedwrites will start out junky and then improve.  EG you’ll get “My am John.  I is tall and me likes girls and videos and my Dad name Mike.”

b) For “freewrites,” they have 1/2 hour to write on a given topic.  I usually use stories.  I’ll say something like “write a story that starts with there was a boy/girl named ____ and include at least one food/clothing item, animal, etc.”  I also tell them, include dialogue, describe characters (and their families etc etc) and include some kind of clear problem that has to be solved.   The kids will begin copying stories you’ve done in class, and as they get better, their stories will diverge from yours more and more.

For marking freewrites, I use the following rubric (from first freewrite right up to advanced students).  The only changes are, I would expect my beginners (after say 20-30 hours of CI) to write say 75-125 words, while end-of-year Level 2 kids should be able to do around 1,000.  In a CI class, the scores– for kids who are always there and who actively listen– shouldn’t be below the 6/9 mark and most will be higher.  I mark /12 if the writing includes graphics etc (e.g. I am doing a “visit Spain” culture project or suchlike).

Writing criteria.doc