Make journalists learn statistics

21 January 2013
Protectionism vs empowerment in the teaching of statistics
4 February 2013

All journalists should be required to pass a course in basic statistics before they are let loose on the unsuspecting public.
I am not talking about the kind of statistics course that mathematical statisticians are talking about. This does not involve calculus, R or anything tricky requiring a post-graduate degree. I am talking about a statistics course for citizens. And journalists. đź™‚
I have thought about this for some years. My father was a journalist, and fairly innumerate unless there was a dollar sign involved. But he was of the old school, who worked their way up the ranks. These days most media people have degrees, and I am adamant that the degree should contain basic numeracy and statistics. The course I devised (which has now been taken over by the maths and stats department and will be shut down later this year, but am I bitterâ€¦?) would have been ideal. It included basic number skills, including percentages (which are harder than you think), graphing, data, chance and evidence. It required students to understand the principles behind what they were doing rather than the mechanics.
Here is what journalists should know about statistics:

Chance

One of the key concepts in statistics is that of variability and chance.Â  Too often a chance event is invested with unnecessary meaning. A really good example of this is the road toll. In New Zealand the road toll over the Easter break can fluctuate between 21 (in 1971) and 3 in 1998, 2002 and 2003. Then in 2012 the toll was zero, a cause of great celebration. I was happy to see one report say â€śThere was no one reason for the zero toll this Easter, and good fortune may have played a part.â€ť However this was a refreshing change as normally the police seem to take the credit for good news, and blame bad news on us. Rather like Economists.
With any random process you will get variability. The human mind looks for patterns and meanings even where there are none. Sadly the human mind often finds patterns and imbues meaning erroneously. Astrology is a perfect example of this â€“ and watching Deal or No Deal is inspiring in the meaning people can find in random variation.
All journalists should have a good grasp of the concepts of variability so they stop drawing unfounded conclusions

Data Display

There are myriad examples of graphs in the media that are misleading, badly constructed, incorrectly specified, or just plain wrong. There was a wonderful one in the Herald Sun recently, which has had considerable publicity. We hope it was just an error, and nothing more sinister. But good subediting (what my father used to do, but I think ceased with the advent of the computer) would have picked this up.
There is a very nice website dedicated to this: StatsChat.Â Â  It unfortunately misquotes H.G.Wells, but has a wonderful array of examples of good and bad statistics in the media.Â This post gives links to all sorts of sites with bad graphs, many of which were either produced or promulgated by journalists. But not all â€“ scientific literature also has its culprits.
Just a little aside here â€“ why does NO-ONE ever report the standard deviation? I was writing questions involving the normal distribution for practice by students. I am a strong follower of Cobbâ€™s view that all data should be real, so I went looking for some interesting results I could use, with a mean and standard deviation. Heck I couldnâ€™t even find uninteresting results! The mean and the median rule supreme, and confidence intervals are getting a little look in. Percentages are often reported with a â€śmargin of errorâ€ť (does anyone understand that?). But the standard deviation is invisible. I donâ€™t think the standard deviation is any harder to understand than the mean. (Mainly because the mean is very hard to understand!) So why is the standard deviation not mentioned?

Evidence

One of the main ideas in inferential statistics is that of evidence: The data is here; do we have evidence that this is an actual effect rather than caused by random variation and sampling error? In traditional statistics this is about understanding the p-value. In resampling the idea is very similar to that of a p-value â€“ we ask “could we have got this result by chance?” You do not have to be a mathematician to grasp this idea if it is presented in an accessible way. (See my video â€śUnderstanding the p-valueâ€ť for an example.)
One very exciting addition to the New Zealand curriculum are Achievement Standards at Years 12 and 13 involving reading and understanding statistical reports. I have great hopes that as teachers embrace these standards, the level of understanding in the general population will increase, and there will be less tolerance for statistically unsound conclusions.
Another source of hope for me is â€śThe Panelâ€ť, an afternoon radio programme hosted by Jim Mora on Radio New Zealand National. Each day different guests are invited to comment on current events in a moderately erudite and often amusing way. Sometimes they even have knowledge about the topic, and usually an expert is interviewed. It is as talkback radio really could be. I think. Iâ€™ve never listened long enough to talk-back radio to really judge as it always makes me SO ANGRY! Breathe, breatheâ€¦
I digress. I have been gratified to hear people on The Panel making worthwhile comments about sample size, sampling method, bias, association and causation. (Not usually using those exact terms, but the concepts are there.) It gives me hope that critical response to pseudo-scientific, and even scientific research is possible in the general populace. My husband thinks that should be â€śinformed populaceâ€ť, but I can dream.
It is possible for journalists to understand the important ideas of statistics without a mathematically-based and alienating course. I feel an app coming on… (Or should that be a nap?)

Dr Nic

1. Mayo says:

This is an excellent idea, and I see many instructive videos in the links you’ve provided. Mayo

• Dr Nic says:

Thanks. I’m glad you like our videos. I have four more about time series about to be released when our new app is approved.

2. I guess that the answer of why journalists do not really use stats is because the news would be a lot more boring: lots of stories would be random noise. Even New Zealand’s Chief Coroner overinterpreted data on suicides post Canterbury earthquake.
You are right, these days it is almost impossible to get decent reporting covering average and dispersion; that’s why I tend to look behind some stories and get the study/scientific paper being reported and extract the data from there. The shocking part is often discovering that the news are completely misleading.
Concerning your video on the p-value and choconutties, I’m confused by your null hypothesis. As a consumer, I wouldn’t care that the average bar has 70 g of nuts, but my expectation would be that the bars contain at least 70 g of nuts. I can see that one could get away with this approach in some industries (e.g. wood machine stress grading) but not in others (e.g. climbing equipment where my life depends on meeting standards).
Incidentally, StatsChat makes clear in this post what Wells actually said in context.

3. Dr Nic says:

Hi Luis
Thanks for your thoughtful comment. There is a reply to a question under the p-value video explaining why the hypotheses are around the way they are. Believe me I thought long and hard about that one. Helen just wants to make sure that no one can “prove” that her choconutties are lacking in nuts. I’m pretty sure the choconutties fits in the non-life-threatening category. If I did the video again I would think of a better example. I had no idea when I wrote it that it would have more than a few hundred views by students in my classes.
And thanks for the note about the post in StatsChat. I like his explanation, though I would put more weight on Operations Research of course. I suspect it is no coincidence that this appears a week after my comment about the misquote. Unless it is a coincidence, it would have been nice if he had linked to my post. (Not your problem, I realise)

4. Jay Grant says:

Brilliant idea, I only wonder will journalists care if they can make more outrageous headlines and sell more papers misinterpreting statistics?

5. Peter says:

I, too, have been frustrated as std deviations lacking. Here’s one report I found that does quite std deviations:
http://www.ausstats.abs.gov.au/Ausstats/subscriber.nsf/Lookup/CA25687100069892CA256889001F4A36/\$File/43590_1995.pdf

• Dr Nic says:

Thanks – that is an interesting study.

6. […] Make journalists learn statistics (learnandteachstatistics.wordpress.com) […]

7. Jon Bell says:

Agreed!

8. […] to paraphrase the often paraphrased H. G. Wells. I have already written on the necessity for journalists to have an understanding of statistics. The innovative New Zealand curriculum recognises the importance of an understanding of statistics […]

9. […] I believe statistical literacy should be a compulsory part of all journalists’ training. I have written before about this. I was happy to see letters to the Editor in the 22 October issue of the Listener condemning the […]

10. […] I believe statistical literacy should be a compulsory part of all journalistsâ€™ training. I have written before about this. I was happy to see letters to the Editor in the 22 October issue of the Listener condemning the […]

11. […] as a psychologist, marketer or physician. (And it should be required for many other careers, such as journalism, law and sports […]

12. Erica warren says:

I graduated with a journalism degree in 2005 and had to take a statistics class as part of my degree requirements. It was statistics for journalists. Any formally trained journalist knows basic statistics. I think we are dealing with a dearth of informally trained (ie: untrained) pseduo-journalists and TV personalities posing as journalists. These are not the same.

• Dr Nic says:

Hi Erica
Thanks for commenting. It was great that you had to take a stats class in your degree. I think it is rather overstating it to say that “any formally trained journalist knows basic statistics” as a result of your experience. I don’t really understand what you are saying about a dearth of informally trained pseudo-journalists. I would rather suspect that we have surplus of them.

13. Anonymous says:

As a sports media student who is forced to take Statistics I get where youâ€™re coming from. But I also hate math and Iâ€™ve been avoiding this class since beginning school. Pray for me.