Organising the toolbox in statistics and operations research
17 December 2012
Statistics or Calculus? Do both!
21 January 2013

My northern hemisphere twitter buddies are well into the academic year, and facing the demands of grading, while here in New Zealand we are enjoying the sunshine and trying hard not to think about going back to work. However the teachers of High School statistics in New Zealand are facing (or trying not to) an interesting challenge in the coming year. They are going to have to mark (our word for grade) essays. Eek.
One of the main reasons I majored in operations research, and became a mathematics teacher was that I was required neither to write nor grade essays. This must sound funny coming from someone who can’t stop blogging! I remember exam time as a new high school teacher, happily putting little red ticks against numerically correct answers, and occasionally pausing to decide if the working were adequate. Next to me was the also new English teacher having to give grades for essays, agonising over what the work was telling her and what grade to assign. It was hard not to feel smug.  In the end I felt sorry for her and helped with some of her marking.
Then at university I taught operations research and statistics, both of which I thought could be reasonably assessed with mainly numeric questions. But as time went on and I gained a greater understanding of my discipline and of teaching and learning I realised there was no escape. I dabbled with orals, essays, assignments and on-line assessment. I put on a brave face, and tried to focus on what I was learning from the mistakes they were making. To be sure you do learn a lot from marking student work, but the effect wears off on the 50th script. Or sooner.
There is no escaping it…

Marking/Grading is difficult, often unpleasant and extremely important.

Feedback is a vital part of learning. Research into education and learning has shown that specific, timely feedback is possibly THE most useful thing to help people to learn. Well duh!  As an aside, this is one of the reasons I have reservations about giving too much homework in mathematics. If the students don’t check their work as they go, in the absence of correct feedback they can often entrench wrong procedures and thinking.
As we learn we need to make sure that we are learning correctly, in a physically and emotionally safe environment. This is why flight simulators were invented, so pilots could practice crashing – or rather not crashing, while remaining alive.
This is one of the reasons I fell in love with my Learning Management System. Originally it was hosted by WebCT, then Blackboard and (I hope finally) Moodle. (Clearly a non-specific love-affair!) A good LMS can give non-judgmental, correct, timely, specific feedback FOREVER! It never gets tired. We had a student who struggled with English, who sat one of our on-line tests over 70 times. He got there in the end, with the help of several of my more patient tutors. But there is no way we could have given him the time he needed, in the way the LMS did.
Of course there is only a certain range of assessment possible for automatic grading, but I have been experimenting with different ideas, and have managed to come up with ways to provide worthwhile automated feedback to students in most circumstances. Another great thing about the LMS is that you can collect the results and quickly see what the students are getting wrong, and which distractors are most distracting!
The first reason we need to grade is to give feedback, to help students learn. This is known as formative assessment. In a school setting we can usually make this low stakes, and the students will still participate, but at university level, time pressure means that unless the assessment is “worth something”, the students who need it most are least likely to do it. We found a sizable correlation between participation in the small tests and grades in the course as a whole. We don’t claim causation, but that doesn’t mean it isn’t there!
The other main reason for assessment is to evaluate the learning at the end of the course. The formal term for this is summative assessment. This is what tells the student and future employer how well they did in the student did in the assessment at the time. It may or may not tell anyone how much the student knows, especially some time later.

Miscellaneous thoughts on assessment and grading:

  • Align assessment with learning objectives. Don’t ask what you haven’t specified and taught. (Except for scholarship exams when you can do what you like!)
  • Students will only learn what is assessed – if you want them to learn something, put it in the objectives, tell the students and then assess it.
  • Be clear in your mind what the assessment is for. Normative or summative? Mastery or brilliance? Encouragement through success or scaring them to do some work? Signalling important points to students in later years? Propaganda?
  • Spend the time devising a good test, and you will save time and pain in the marking.  Write-on answer booklets save time.  On-line saves even more!
  • Don’t ask more questions of the same type than you need to. You aren’t getting more information.
  • Be careful with the word “how”. It is almost always ambiguous.
  • Make sure that ignorance of the non-subject-specific context of the question will not affect the ability to answer. An example – There can be questions involving reading tables that assume that the person knows that Shirley is a girl and will therefore use the female sizing chart. For non-native speakers (and even people from other English-speaking countries) this is not a reasonable expectation.
  • Don’t agonise – if a student is borderline you are probably being too generous. It isn’t personal.
  • Do not assign half marks.
  • Be creative. Try orals.

This is not the last you will hear about assessment. I am currently developing a suite of videos, quizzes, writing guides and an app for teaching and learning basic time-series analysis. Assessing learning for this topic is an interesting problem. Watch this space.

Leave a Reply

Your email address will not be published. Required fields are marked *