Those who can, teach statistics
30 September 2013
Proving causation
21 October 2013

It is important that citizens can make sense out of the often outrageous claims of advertisers and pro-screening advocates.  It isn’t what they say, but how they say it. What looks like a very large and scary increase in risk, can in fact make very little practical difference. Conversely a large risk can be made to look smaller through the manner in which it is communicated.
I found a wonderful set of notes on the Census at School site, presented as a powerpoint file.
I also found several very interesting and educational sites about risk.
This first one explains about risk and relative risk: Science blog on Cancer Research UK
This one also includes Number needed to treat. Patient Health UK.
And a here is a great summary and set of exercises at the Auckland Maths Association website. You need to scroll down to “Relative Risk Resources”. (I found this after writing the rest of the blog, and it pretty much says what I say, but more succinctly!)

Teaching about Risk

Risk is a great topic for teaching about probability, percentages and perception.

It’s what’s on the bottom that counts!

In exploring risk, there are several distinct processes needed. Depending on the format in which the information is given, students may need to construct their own frequency table, or interpret the one provided. From the frequency table they must calculate the probability, making sure that they choose the correct denominator. Then if they are looking for relative risk, they need to make sure that they again choose the correct denominator. For some reason, the numerator is usually easier. But what can be tricky is the denominator.
We can use as an example the increase in probability of passing a particular statistics course if students use our Statistics Learning Centre materials to help them. We haven’t collected any data yet, so these figures are aspirational (as in a work of fiction!). Because we are talking about risk, we have to frame the outcome in negative terms. We would not talk about the risk of passing a course, but rather of failing one. So we will say that students who use StatsLC materials reduce their risk of failing by 66.7% percent. That is pretty impressive, but how much better it sounds if we frame it in terms of how much their risk will increase if they decide not to use the wonderful materials from StatsLC. Their risk of failure increases by 200%. That sounds pretty drastic.
But what we have failed to mention is the absolute risk, which is the proportion of students who fail their stats courses with and without the help of StatsLC. Here are some pairs of absolute risks that will give the results given:
All of the following sets of numbers show a 200% increase in risk of failure for students who do not use StatsLC materials.

Scenario

Risk of failing, when using StatsLC materials

Risk of failing when they don’t use StatsLC materials

Actual increase in risk of failing.

A

1%

3%

2%

B

10%

30%

20%

C

20%

60%

40%

In Scenario A, the pass-rate for the statistics course has gone from 97% to 99%. In scenario B, the pass-rate has gone from 70% to 90%, and in Scenario C, the pass-rate has gone from 40% to 80%. All of these scenarios could accurately be described by the same change in relative risk. They all double the risk of failing if the student does not use StatsLC.
This is really at the end of the story, based on what is reported. But if we wish to find out what is really going on, the best idea is to build a table of natural frequencies. These are great for calculating conditional probabilities by stealth.
Here is a table of natural frequencies for Scenario C above, using 1000 as our total number of people. Before we fill it out, we also need to know how many people used Statistics Learning Centre materials. 30% of students did NOT use StatsLC materials.

Pass

Fail

Total in category

Use StatsLC

80% of 700 = 560

20% of 700 = 140

700

Do not use StatsLC

40% of 300 = 120

60% of 300 = 180

300

Total pass or fail

680

320

1000

From this table, all manner of statistics can be computed.
What proportion of students who passed, used the StatsLC materials?
The answer is (the number of people who passed AND used StatsLC materials)/( the number of people who passed) = 560/680 =82%. It is important to find the correct denominator.
Then when people calculate relative risk, it is important to be careful about choosing the baseline.
Another question might be, by how much does your risk of failure decrease, in relative terms, if you use the StatsLC materials?
The first step is to find the decrease in absolute terms. The risk of failure, not using StatsLC = 0.6. The risk of failure when using StatsLC has decreased to 0.2. That is an absolute decrease in risk of 0.4. Then we need to express this relative to the baseline. As we talked about the decrease in risk, it will be compared with the larger number, or 0.6, the risk of failing when using the StatsLC materials. So 0.4/0.6 = 0.667 or 66.7%. However, if we were talking about the increase in risk for NOT using StatsLC materials, then we would find 0.4/0.2 = 200%.
A great way to develop interaction and group discussion would be to give individuals in the group different information that is needed for the computation. Later on you could include one wrong “fact”, which they would need to ferret out. Another possibility would be to give students information about different scenarios that they need to present in the best or worst possible light.
These are great teaching opportunities, and worthwhile for everyday life.  It is a good thing they have been included in the NZ curriculum for year 12.
A note to regular readers – I will probably be posting less frequently for a while, but feel free to read back over some of my previous 95 posts if you miss the weekly rant. 😉

Leave a Reply

Your email address will not be published. Required fields are marked *