Writing

More Notes on Feedback

Amy Lenord started a great Twitter discussion about how one encourages language learners to process language.  This eventually led to Martina Bex refering us to her excellent “I am a grammar geek” post, in which she talks about how much she loved– and found effective– the “red ink” from her Spanish profs in Uni. Bex and I very briefly discussed this.  (I will bet that when she has a spare moment– and she is a Mom again, congrats!– she’ll discuss this more.  Ha!)

Now, anyone who knows Bex knows that the basic deal with her is that what she wants done, she gets done.  Bex wants babies? Bex has four (at last count).  Bex wants to acquire Spanish?  Bex signs a months-long “no English” agreement with her room-mate!  Bex wants to master C.I.?  Bex does, in like two years of teaching.

So it is not surprising that she acquired a ton of Spanish in very short order in Uni. 

Again: she wanted, liked & felt she benefited from corrective feedback  in her Spanish classes. 

This raises two questions:  did the feedback she got actually help her, and, if so, why and how?

Well, let’s take Martina’s word for it, and say, sure, corrections and comments helped.  Now, how?

Well, suppose young Bex– or anyone else– wrote this on their Spanish 201 composition:

*  Ayer, yo fue al cine con mis amigos, y vimos una película.

This should be “yo fui,” and say her prof writes that on her paper.  Now, what happens next?

  1. Bex notes there is an error.
  2. Bex re-reds the sentence: yo fui al cine.

Most of our students will not even do #1.  Most will go straight to the mark, wondering  what did I get?  did I get an A?

Some will note, ok, there was an error.

A very few will re-read the corrected sentence, and maybe linger on it, in which case it is functioning as good comprehensible input (albeit not many repetitions).

So, why is the feedback working for Bex?  In my view, it is because

a. Bex is majorly motivated which means,

b. Bex wants feedback, and when she gets it,

c. the feedback provides comprehensible input.

Suppose the prof had written “ser takes an -i in the first-person singular.”  Would this have done Bex any good?  The research says no.  Maybe for Bex it did.  Maybe she went, hmm, yo fui al cine…

I was also recently talking to Adriana Ramírez and Luce Arsenault about giving corrections in their Sp and Fr classes.  Both maintained that their kids got better as a reuslt of having to do corrections.  They havn’t obviously had time to do a controlled study, but we noted a few things:

  1.  Both have very motivated, mostly Asian and wealthy white kids, who have been hearing from their literate parents from Day 1 of school, memorise (for many Asian kids, who have had to learn zillions of Chinese characters before coming to and sometimes while in Canada), and edit (for wealthy white kids, whose parents are uber-literate, professional, etc).
  2. My kids– who are generally Hindi, Punjabi and Urdu-speaking, and have less-literate and generally non-English speaking parents, almost none of whom have any formal experience learning additional languages– have not been primed to memorise and relentlessly improve their work.  This is not to say that our parents do not value education– they do, very much– but it is to say that they have not “acquired” some of the academic habits that can sometimes for kids in language classes.

There is a simple lesson here:  unless people want feedback, and get it, and the feedback is comprehensible input, it is not going to do any good.

So the teacher should focus not on marking and correcting, but on relaxing and reading and being happy in their spare time, so when they show up in class, they have the energy and mood to provide good C.I.– in story asking or reading or MovieTalk form– for kids.  And kids should not be forced to correct work (although if they want to, why not?).  Rather, their work should be hearing C.I. in class, and– if they must have homework– reading or viewing comprehensible and interesting target-language stuff.

 

 

 

Advertisements

Level Two Spanish Results: First Picture Description

Spanish 2 has been running for three weeks.  We have read a couple of easy novels, and have done one story cycle: el restaurante, which included reading, storyasking, Movietalk (Mr Bean videos RULE!) and Picturetalk.

Today’s first writing assessment: describe this photo.

  
Here are four writing samples.  The kids had 5 minutes, no notes or dictionaries.

First, Janelle, the top student. Amazing how she mixes past and present appropriately.

 

Next, we can compare two Level Two students who are not top performers. Hassan went to another school last year, where he had traditional grammar-and-textbook-based teaching.  This is garbled and nonsensical.  Hasan has some learning challenges and struggles in other classes.

Next, Abbas, who like Hassan has some challenges and struggles with school.  This is not awesome…but we understand, and he has built in a bit of a backstory.  Abbas had only TPRS in Level One.

Based on results, C.I. clearly helps the challenged kids more than does a traditional text.

   
Finally, Amneet.  This is not very good writing.  What is interesting here is that Amneet is probably the best speaker in the class.  I have found this kind of thing typical:  while most of the kids can undertsand everything (the scores for reading and listening quizzes are all between 80-100%), production skills vary dramatically from kid to kid and medium to medium.  Writers are not necessarily speakers, and vice-versa.

Amneet arrived late in Spanish 2, has missed a bunch of classes, but did well last year (over 80%) so I am expecting her written output will pick up.

 

My Marking System for a Language Class Summarised.

“How do we mark kids in T.P.R.S. classes?” asks a C.I. newbie on the Yahoo listserv.  Assessment and evaluation are always hot topics.

First, definitions.  Assessment is observing how people are doing, and based on that observation

1. changing what you the teacher do and

2. giving students feedback in order to

3. improve student performance.

Assessment in a comprehensible input classroom means checking whether individual students (or the class) understand both statements and questions, and– if they don’t– clarifying. Weak choral responses?  No response?  Wrong answer?  Go back and clarify. The same goes for individual kids.  If Johnny can’t answer the question “What does ___ mean?”, go back and re-explain.  BOOM! that’s assessment.

Evaluation is assigning a number to student work based on a student’s performance in relation to standards (criteria).  This we do at the end of instruction.

Second, principles.  If you havn’t been in-serviced about “assessment for learning,” google it, I can’t explain it all here, but let’s make three things SUPER CLEAR.

A.   We only evaluate “final products,” not practice, homework, behaviour, attitude, etc.   The long and the short of this surprisingly controversial statement is that we expect our kids to do X, Y and Z, and the mark we ultimately assign them at the end of Year ____ of Blablabian Language Class should reflect how well they do X, Y and Z.  Whether Johnny is nice (or a total jerk) in class, and whether or not Suzy does her homework are irrelevant: what matters is how well they do in relation to criteria.

B.  We only evaluate what has been taught.  No random new vocab, no activities on tests the kids havn’t done, etc.  You test what you teach, period.

C. We always let ppl re-do work so they can improve their mark. In my class, this means that their mark is always their most recent writing, reading and speaking mark.

Assessment should be fast, simple and minimal, because people learn from interesting comprehensible input and not from testing, and because teachers need to see their families & have a life 👍.  

Now, here is what I do.

OVERALL YEAR MARKS:

  • 10% all work done during year
  • 10% two culture projects (5% each)
  • 80% final listening, reading & writing exam (speaking for Level 2 & up)

First, this is how I mark writing.

Second, for reading and listening marks, the easiest thing is the exit quiz.  One a week for reading and one for listening.  Speak– or write on the board– five sentences.  For listening, the kids write in the target language and translate into English.  For reading, they translate into English.  You can have them trade papers and mark.  You can do an exit quiz in ten minutes, the marks recording is quick, and you’ll know immediately if they understand or not.

Another great idea– thanks, Ben Slavic and also ironically legacy methods– is dictation.  For dictation, read a very short (e.g. 10-sentence) story aloud.  The kids write down what they hear.  Then, they translate.  Finally, you project the story, and they fix their errors.  I have no idea how to mark dictation.  Suggestions?

I also mark my kids’ comics for a reading mark.  This is Adriana’s idea from her special-ed course and it’s simple & excellent:

  • read the extended version of the most recent story (if using embedded readings, read the most complex one)
  • make a 10-16 panel comic (Internet clip art etc fine) of the story
  • each panel MUST have at least one “narrating sentence,” (e.g. “John was hungry and went to the store”)
  • each panel MUST have EITHER dialogue or thought-bubbles (characters either describing themselves, e.g. “I like girls” or thinking, e.g. a girl thinking “I am hungry”)
  • the pictures MUST accurately support the meaning

This is what Adriana calls “deep reading.”  It makes the kids read, extract the main points, and illustrate them.  It’s also fairly easy, and the results– which I say should look decent, be in pen, have a bit of colour and not use lined paper– go on the wall, where the kids read each others’.  If you have Adminz or Headz who get excited about Technologiez, the kids can make them using Computerz!

The comic marks are cumulative: if they don’t do them, they get zero until they are done.  A comic typically takes a kid about 30-40 minutes to do, and is the only homework I assign (other than occasional reading).

Third, I do not assess “speaking skills” in Level 1.  Shocker, I know, but there’s a few reasons.  I know exactly how well each student can talk.  And–unless your modus operandi is hand out worksheets and put your feet up– so do you. Level 1 is where we lay foundations, where we plant seeds, and if the kids are getting a load of input, these seeds, as Ben Slavic puts it, will bloom in Level 2.  And practising speaking does not develop speaking skills.  My kids can talk– a lot— but I don’t pressure them with “prepare for your oral test” nonsense (how do you “prepare” for an oral test anyway?!?).

I assess speaking skills in one three-minute interview at the end of Level Two (and up).  These interviews are 100% random questions.  For Level Two, I am basically looking for their ability to understand and respond (they should be able to answer a question like “What do you like to do?” or “What did you do last weekend?” with 1-3 sentences).  I also get them to tell me a story.  For Level 3 and up, they should be able to provide longer answers and ask me questions as well.

Now, how does it all hang together?

Well, what I have is a “rolling” gradebook.  That is, their mark always reflects their latest performance (the exception:  the comics, which are all added together under the reading mark).  Their mark is always the most recent thing they have done (for writing, listening and reading).  We do assessment during the story-asking process for listening and reading skills, and at the end of a “story cycle” for writing.

This allows me to say “right now, you are getting ___%” and also “if you blow off coming to class, reading etc, your mark can drop.”  With T.P.R.S., most kids do quite well, so I have very few parent or kid complaints.  In the last three years, one parent has complained…and when I told her that her son missed an average of 60% of classes, she said “oh, I see” and that was that.  Also, the kids know that if they have a crappy day on writing assessment day, they can boost their mark next time around.  Generally, the marks are quite consistent.

When I get the dreaded question “how can Maninder improve her mark?” from a parent , I say all you can do is listen in class, read, and try to find easy listening or reading in Spanish outside of class.  This also happens to be the only thing that really helps.  There is no edubabble like “look in your portfolio and revise your ____” or “master your ____ verbs” or whatever (feedback is useless: if it adresses the conscious mind, it will be either forgotten or unavailable in real-time interactions).

Teachers say “well, a kid could do nothing all year and then try to ace the final.”  Yes, they could.  But…for one, as Bill VanPatten reminds us, the “work” in a language class is processing input.  No processing input = no (or very little) acquisition. And sure enough, the kids who blow off listening and reading (and/or skip classes) see their marks drop.  If they keep re-writing the same story, and don’t learn new vocab, the end-of-year writing criteria takes that into account.  They are also expected (in Level 1) to write about 75 more words in each 40-min story-write and 10 more words per 5-min speedwrite (describing themselves or a picture) than they did the previous time.  Kids who skip etc see their marks drop, because their output has less varied (and less well-used) vocab, and lower wordcount.

In 2015 Spring semester, I had two kids who missed about 50% of classes.  Sure enough, at the end of the year exam, their wordcounts for stories were in the 300 range (the rest of the class could do 600, and the top kids almost 900), their 5-min speedwrites were about 60 words (every other kid was over 100), and their reading and listening comprehension was significantly lower.  Most interesting:  these kids did not badly…but would have failed under the grammar grind/communicative system.

So, they have “ongoing” evaluation where their marks are always the last thing they’ve done.

For final evaluation, I do a 45-min story write and a 5-min describe-a-picture write.  I use this Timed Writing Rubric (from Kristin Duncan) at the end of the year.  In Level 1, the story word target is 800 words in 45 min; for Level 2 it is 1,000.  For the picture,t he target is 100 words in 5 min.

For listening, I do dictation:  they listen to a 20-sentence Spanish story, write it in Spanish, then translate into English. For reading, I give them 20 questions relating to the embedded (long) reading versions of the asked stories, and they have to hunt through the stories (re-reading! reps!) for answers.  The answers are 1. copy from the text and 2. translate.  The evaluation here again is of comprehension, not output.

Generally, I use 80% final evaluation (read, write, listen; oral for 2s and up), 10% culture projects and I make the entire year’s work worth 10% for their final Spanish mark.  Interestingly– I kept stats this year– there was very little difference between kids’ final exam marks and their during-the-year marks.

This has never happened, but I tell kids, if you blow it on the final writing or reading exam, you can re-do it.  As Vancouver T.P.R.S. teacher Steve Bruno remarks, one of the great things about T.P.R.S. is low test anxiety.  The kids know what they have to do, they can do it, and there is zero difference between a final exam and a regular test.

So anyway…to sum it up:

  • students are always marked on their most recent work
  • no oral evaluation for beginners
  • this is real “assessment for learning” where only final products are evaluated with reference to criteria
  • there is no marking of attitude, homework, participation, etc.  If kids screw around, or don’t work, I deal with it– but not by using marks as carrot or stick

How do I mark writing? Blaine Ray’s ideas, slightly modified

Stephen B.– who after twenty years of traditional grammar teaching jumped headfirst into C.I.; how bad-assed is THAT?; writes

“Anyways, I had a question about grading speed/timed writings. I know Blaine says one point per word and he talked about scaling and not to mark it for accuracy. However, what does one do in this situtation: for example, if after a 5 minute speed write one students writes 85 words but is all over the place and makes several grammatical and spelling errors, and another student only writes 60 words but it is almost perfect, how can I give the former student a higher mark?”

Here is how I do it (with many suggestions from Adriana Ramírez, whose Teaching Spanish Through Comprehensible Input text I am using this year). First, classic TPRS in a story cycle, with movietalk and picturetalk:

A) story cycle: establish meaning, ask story (with a few parallel characters), review, retell, read a couple of versions of story

B) have the kids create a comic of the story. Story must be “complete” but obviously not everything can be put into comic. Story must be coloured, look awesome, etc (clip art fine). Each panel must

  • have at least one Spanish sentence
  • have perfect alignment between Spanish and pictures
  • where there is no dialogue, “thought bubbles” in the first person
  • where appropriate, have dialogue

This will make the kids read, choose sentences, and clarify meaning via illustrating.

C) movietalk and picturetalk to support story structures (e.g. if you taught “wants,” movietalk and picturetalk a video where a person wants something)

Then, for assessment, I am doing the following:

1) when the “story cycle” (A-C above) is done, kids will do a five-minute speedwrite and a forty-minute relaxed write.

Their first speedwrite topic will be “describe yourself.”

The speedwrite is evaluated in 2 ways:

First, wordcount. Kids count the # of words in their composition (tell them no lists, or, if they want a list, they must describe all things in the list). End of year goal: 100 good words in 5 min. For their first speedwrite, they get a 40-word bonus. So if they write 30 words, their wordcount score is 70/100.

Second, they get a grammar mark out of 3, thus:
1– it’s full of mistakes and largely incomprehensible
2– it’s mostly comprehensible but has some “whaaat?” moments and “feels” junky
3– it’s fully comprehensible, has no “whaaat?” moments, and “feels” fluid and solid (but not necessarily perfect)

Multiply their grammar mark by 33.3 and they have a grammar mark /100.

Now, average the two marks and they have a spedwrite percentage.

For the 40-min relaxed write, I tell them “either retell the story, or write your own, and you must have 3 dialogues, but put changes into your version of the story.” The goal for the year: write an 800-word story in 40 min. For their first story, I’ll expect 70-150 words. I will assign a wordcount mark out of, say, 200 and give them a 50-word bonus. I will also give them a grammar mark /3 above.  Every time they write a story, the amount of words expected goes up and the curving bonus goes down. 

We average their grammar mark and wordcount: if Johnny gets 2/3 for grammar, and writes 90 words, his score is 66.6% (grammar) + 140/200 (70%) for wordcount = 69%.

After we do the second story of the year (and until the end of the course), we repeat the procedure, with a few changes

A) the speedwrite bonus drops by 5 words each time
B) the relaxed write bonus drops by 5 words each time and the “benchmark” goes up by 75 words. By end of year kids should be able to write 800 words in 45 min.
C) we use another topic for the speedwrite for the second time: describe a picture that you project onto your screen. This picture should support what was in your story. So, if the story had a girl who wants an elephant, your picture could be a boy who has an elephant.

D) for the third speedwrite, use topic #1 (describe yourself). For the fourth, use a picture. Keep alternating.  I use fully unsheltered grammar from Day 1 (all verb tenses, subjunctive, etc) so the picture describing tests evaluate how well they can use present tense. 

The writing will improve during the year. As I write this after having done only two stories, wordcounts are WAY up and grammar is also improving.

A few notes:

— you MUST carefully restrict vocab. This has been my single-greatest problem with TPRS: adding vocab at random. If you don’t restrict vocab, you get fewer reps on each item…and worse/less acquisition.

— initially, the kids will generate pretty crappy stories. Later, word count goes up and grammar will get better. Some kids will re-write the story; most will start to improvise.

their “mark” at any given time is simply their most recent speedwrite and relaxed write mark, combined.  I also do exit quizzes for listening and reading (1 each/week) so I have a pretty good overall picture of how everyone is doing. 

PQA is super important. Adriana’s book has a list of “personalised questions.” The Blaine Ray books also do. If you are doing your own stories, you make them up. Personalised questions are super-easy: you basically ask the class the questions you ask the actor.

So if you narrate “the boy liked running,” you ask your actor “do you like running?” and he says “yes, I like running.” You then ask “do you like vomiting?” (something contrastive) and he says “no, I do not like vomiting.” Then, starting with your superstar, you ask the class members “do you like vomiting or running?” etc. Simple.

This is important because the kids need to hear the present-tense forms.

— Adriana’s advice was to make sure all the kids do the comic. This is because the comic writing is “deep reading:” it makes the kids re-read, choose, copy and write, etc. For the non-artists, translation also works: copy story, underneath it translate (diff coloured pen), leave a blank line to keep it clear. Here is a pretty good example of “Los Gatos Azules” turned into a comic (one of Adriana’s kids did this one):

20141031-122945.jpg

20141031-123008.jpg

Anyway, this is how I have organised the “units” of TPRS and how I assess. Coments, as always, welcome!

How do we grade speedwrites and freewrites?

Here is another question from Sarah-Beth:

Q:  “I was also wondering how you score the Relaxed Writes? Is it a similar process? I’m not sure what to put the total out of (same with Timed Writes). The marks seem really inflated when I factor in the 40 to top them off and I’m getting over 100% on some students, which are totally skewing their marks. What do you suggest? Merci!”

A:  These are Blaine Ray’s ideas.  First, announce course goals.  For beginners, write 100 coherent words in 5 min; write a 500-word story in 3 verb tenses in 30 min (by end of course).  I use 2 kinds of writing during the year– speedwrites and timed writes.  I start writing after about 20 hours of CI.

A speedwrite is, the kids have 5 minutes to write.  They have to produce as many words as possible during that time.  Then they count their words.  The only “rules” I have are no lists, and focus.  I start grading these with a “handicap” system.  The freewrite is out of 100 and the kids’ scores are the # of words they write, plus the handicap.  First time out, the kids will typically write 20-60 words.  Add 40 to that, and you get a score of 60-100.  Next time, two weeks later, do the same thing…but add only 30.  By the 4th month (semester block system of 6.25 hrs/week) kids should be easily hammering out 100 words in 5 min.

The speedwrites will start out junky and then improve.  EG you’ll get “My am John.  I is tall and me likes girls and videos and my Dad name Mike.”

b) For “freewrites,” they have 1/2 hour to write on a given topic.  I usually use stories.  I’ll say something like “write a story that starts with there was a boy/girl named ____ and include at least one food/clothing item, animal, etc.”  I also tell them, include dialogue, describe characters (and their families etc etc) and include some kind of clear problem that has to be solved.   The kids will begin copying stories you’ve done in class, and as they get better, their stories will diverge from yours more and more.

For marking freewrites, I use the following rubric (from first freewrite right up to advanced students).  The only changes are, I would expect my beginners (after say 20-30 hours of CI) to write say 75-125 words, while end-of-year Level 2 kids should be able to do around 1,000.  In a CI class, the scores– for kids who are always there and who actively listen– shouldn’t be below the 6/9 mark and most will be higher.  I mark /12 if the writing includes graphics etc (e.g. I am doing a “visit Spain” culture project or suchlike).

Writing criteria.doc