Khan Academy Statistics videos are not good
Question questions
23 July 2012Forget algebra – is Statistics necessary?
6 August 2012
I don’t like the Khan Academy videos about statistics. But I can see why some people do. Some are okay, though some are very bad. I’m rather sorry they exist though, as they perpetuate the idea of statistics as mathematics.
Khan Academy, critics and supporters
Just in case you have been living under a rock, with respect to mathematics education, I will explain what Khan Academy is.
Sal Khan made little YouTube videos to teach a family member maths. Other people watched them and found them useful. Bill Gates discovered them and threw money at them. Now there are heaps of videos, with some back up exercises, and some people think this is the best thing to happen to maths (and other) education. Other people think that the videos lack pedagogical content knowledge. Sal agrees – he says he just makes them up as he goes along.
Diane Ravitch linked into the Khan Academy debate, beginning with this post, which is what got me looking into this. Two mathematics teachers made videos after the style of Mystery Science Theater 3000 starring two of Khan’s poorer mathematical contributions. The one on multiplying negative numbers was particularly poor and has since been replaced. Critiques of Khan seem to meet with two kinds of comments. One group is people who know about teaching, who are pleased that someone is pointing out that the emperor, though not naked, is poorly clad. The other lot are generally telling the mean teachers to leave Khan alone, that he is the saviour of mathematics teaching, and they would never have understood mathematics without him. The supporters also either suggest vested interest (for people who make educational materials) or that the writers should try to do better (for those people who don’t make educational materials). To be fair, the first group are also calling for other people to make better videos and put them out there.
For a good summary of the pros and cons of KA, here is a recent article in the Washington Post: “How well does Khan Academy teach?”
Khan Academy Statistics videos
So I took a look at Khan Academy statistics videos. I know something about the teaching of statistics. I have many years of experience of successful teaching, I have done research and I have read some of the literature. I have pedagogical content knowledge (I understand what makes it hard for people to learn statistics.) And I have made my own statistics teaching videos, which have been well received. I wrote some time ago about the educational principles based on research into multimedia, which have been used in developing these videos. Unlike Khan I have thought hard and long about how to present these tricky concepts. I have written and rewritten the scripts. I have edited my audio to remove errors and hesitations, I have…anyway – back to Khan Academy.
To be fair, statistics is one of the most difficult subjects to teach, so I didn’t have high hopes.
To start with the list of topics under the statistics heading showed a strong mathematics influence. This may reflect the state of the curriculum in the United States, but in no way reflects current understanding of how statistics is best understood. I couldn’t find anything about variation, levels of measurement and sampling methods, which are all foundation concepts of statistics. I think it would be more correct to call the collection of videos “the mathematics of statistics”. It starts with the “Mean, Median and Mode.” Not exactly a great way to enter the exciting world of statistics. And he mispronounces the adjectival use of “arithmetic”, which is a bit embarrassing. (Note in 2017 – it has now been corrected. – Yay)
Reading Pie Graphs
I summoned up courage to view the video on reading Pie Graphs. It was not good. The example was percentages of ticket sales for Mediterranean cruises over a year. That data should never have been put into a Pie Graph. For two reasons! First there are too many slices of pie. A pie chart should never be used for that many categories. But worse than that, the categories are ordinal – they are months. The best choice of graph is a bar or column chart, with the months in order, as you would then be able to see trend! (I have to stop myself here or I could rave on much longer). My point is that Khan has used a very BAD graph as his example. This is one of the worst things a teacher can do, as it entrenches in the students’ minds that this is acceptable. The only thing good about the graph was that it was not three dimensional, and it is not exploding. It didn’t even have a title. Bad, bad, bad. (Sorry I was meant to be stopping)
Confidence Intervals
I am tempted to say Khan is arrogant to think he can produce something after a few minutes thought. Actually I was tempted to say something rather stronger than that. I have to admit I haven’t watched many of the videos, but I really don’t want to spend too much of my life doing that. I chose one on Confidence Intervals, which nearly had me throwing things at the computer. It never explains what a confidence interval is. The bumbling around was so painful I couldn’t watch the video in its entirety. I’m pretty sure he got it wrong. He was so confused by the end that I can’t say for sure. My own confidence intervals video is one of my earlier ones, so it is a little rough, but I’d wager most people understand better what a confidence interval is after watching it. (UPDATE: Since writing this post I have made a better video about confidence intervals. It explains what confidence intervals ARE!) You can watch it here:
So then I decided I should look at the video entitled p-value and hypothesis tests. This is something I know many people struggle with. It is crucial to understanding inferential statistics. I have spent many hours working out ways in which to teach this that will help people to understand.
The p-value and hypothesis testing
Well I watched most of the p-value video, and was pleasantly surprised. The explanation of how we get the p-value is sound, and once he gets into his flow, the hesitations get less irritating. There is a small error – talking about 100 samples, rather than a sample of 100 observations. Also it is a bad idea to have a sample size of 100 in an example as this can get confused with the 100 in the expression of the confidence interval as a percentage. But it does give a good mathematical explanation of how the p-value is calculated. I’m not sure how well it helps students to understand what a p-value is. For a mathematically capable student, this would probably be enlightening. I have my doubts about most of the business students I have taught over the last two decades.
My main criticism is that the video is dull. It doesn’t provide anything more than the mathematics. But apart from alienating non-mathematical students it isn’t harmful. In fact if I had a student who wanted to know the mathematics behind the statistics, I would be happy to send them there. People have commented that my videos don’t tell you how the p-value is calculated. This is true. That is not the aim. Maybe I’ll do one about that one day, but I figured it was more important to know what to do with one.
Khan Academy videos on statistics aren’t good
My point is, surely we can do better than that! Bill Gates has thrown money at the Khan Academy. Wouldn’t it be wonderful if it were the purveyor of really good practice rather than mediocrity? One blogger suggests that if Khan Academy could use really good videos, it really could be useful.
I have gone on long enough.
I realise now, that asking a busy person to watch my videos is a bit of a cheek. They aren’t that long though. They are funny and clever. They are NOT like Khan Academy. I think they are worth the six to ten minutes each.
Here are links to my three most popular ones. Enjoy.
82 Comments
Posted to Facebook: Another post very critical of KA offerings. If this issue is something you follow, keep track of the comments this draws as it makes its way into the awareness of the Kadre of Kritic-Krushing Khan Kommandos. There’s no way they can let this go by without raising their standard weak objections, shallow retorts, and, inevitably, ad hominem arguments, despite the very specific analysis AND counter-example videos offered by the blog author. This should be “good.”
Thanks. I do hope so.
Then help him! Khan Academy has publicity on a global scale. They are, for better or worse, the center of the online education industry right now. Try and ensure that it is for better. Instead of bashing Khan for his ignorance, why don’t you make actual, helpful suggestions that will let him improve the videos?
I don’t know how to contact him. His video on the evidence for sea floor spreading, mandated to show in Earth Science by the school principal, was painful to watch, with all the bumbling around, changing the direction of his discourse many times, and spewing out factual errors. He needs a script for each presentation that has been independently edited and fact checked and he needs to stay with it. Rehearsal before recording would also probably improve the final presentation. A large proportion of the students objected to the video and the majority refused to watch it. I could hardly blame them. As a professional educator I firmly believe that improvement is always possible and we should never be unconditionally committed to instructional practices that have deep flaws.
Hi Steve
It is disturbing. I find the arrogance and lack of professionalism unacceptable. When I make videos, I prepare my scripts very carefully, and this is in a subject I know a bit about. I wouldn’t dream of making videos on subjects outside my expertise without serious checking.
Oh, but of course… the only thing that matters is that The Students Like It! (At least the ones they care to listen to).
I liken it to the latest diet craze… people who are desperate for something that looks like success with something that’s confounded them are prone to having conversion experiences.
[…] "Khan Academy Statistics videosSo I took a look at Khan Academy statistics videos. I know something about the teaching of statistics. I have many years of experience of successful teaching, I have done research and I have read some of the literature. I have pedagogical content knowledge (I understand what makes it hard for people to learn statistics.) And I have made my own statistics teaching videos, which have been well received. I wrote some time ago about the educational principles based on research into multimedia, which have been used in developing these videos. Unlike Khan I have thought hard and long about how to present these tricky concepts. I have written and rewritten the scripts. I have edited my audio to remove errors and hesitations, I have…anyway – back to Khan Academy.To be fair, statistics is one of the most difficult subjects to teach, so I didn’t have high hopes.To start with the list of topics under the statistics heading showed a strong mathematics influence. This may reflect the state of the curriculum in the United States, but in no way reflects current understanding of how statistics is best understood. I couldn’t find anything about variation, levels of measurement and sampling methods, which are all foundation concepts of statistics. I think it would be more correct to call the collection of videos “the mathematics of statistics”. It starts with the “Mean, Median and Mode.” Not exactly a great way to enter the exciting world of statistics. And he mispronounces the adjectival use of “arithmetic”, which is a bit embarrassing…" […]
Reblogged this on Things I grab, motley collection .
[…] I don’t like the Khan Academy videos about statistics. But I can see why some people do. Some are okay, though some are very bad. I’m rather sorry they exist though, as they perpetuate the id… […]
I’ve been following the criticisms and discussions around this as well. Right of the bat, I think Salman Khan is a hero — I always thought that he was going to set of a revolution in education. That has been true, with 3 Stanford’s courses being offered online, now followed by hundreds on Coursera and this fall edX by MIT and Harvard.
Can’t we praise what he’s done, rather than nit-pick on the few little errors here and there, glossing over the fact that he is helping millions?
Even though his methods/material is not perfect, he’s still better than most Math teachers out there. For that majority, Sal provides an alternative. He makes Match “fun” for those who always sleep through math lectures.
They are working hard to make things better. The lead developer at Khan responded on HN: “We have four ex-teachers as full-time employees. We have two high school math teachers as consultants. One Harvard Doctoral candidate in Education and one post-doc in neuroscience at Stanford are in residence. One UPenn Professor is also likely to begin a sabbatical with us. We have a 3 person team dedicated to working with and getting feedback from our 50 pilot classrooms and the 15,000 teachers actively using KA in classrooms.”
What I deem more important, is that Khan Academy is revolutionizing education first-hand, flipping the classroom. Problem-solving gets done in class with peers, while lectures are being watched at home (at their own pace).
We need more heroes like him and his team. Khan Acedemy is an NGO, nobody is doing it for profit and we need to rally behind these initiatives. The education system as is has way too many holes in it — in dire need of a refresher.
My thoughts too. Instead of being jealous of Khan’s success, ride the wave. Before Khan online learning was something relegated to the college setting. Maybe teachers and academics had vague hopes for something like this, but the for mainstream audience online learning was a neat idea that could never really work.
Khan changed that perception. Online teaching and resources for learners are being taken seriously. Real money and attention is being poured into this area and it is not just Khan Academy who benefits. His success encourages others that this is possible. Sometimes the rising tide really does lift all boats: ride the wave.
We can not realistically expect Khan to teach every subject perfectly. Even where he falls short his material is better than the instruction many students have available to them.
If it were not for the Khan Academy we would not even be having this conversation. Critic his material, that is healthy, but it is petty to attack an institution that is dramatically changing the ways we teach for the better. Why do those who disagree with Khan insist on making this such a divisive issue? Surely there are ways to improve upon Khan without resorting to this us vs. them mentality. Why does your success seem predicated on Khan’s failure?
It is probably worth noting that this post is not a criticism of the Khan Academy, so both of your arguments use the “Straw Man fallacy” (see http://en.wikipedia.org/wiki/Straw_man).
It is in fact, a critique of a specific section of the Khan Academy, specifically the section dealing with the teaching of statistics.
I imagine Mr Kahn would be more than happy to have errors pointed out. It would be nice to feel good about the manner in which critics approached this. The tone should be helpful rather than just a horn blasting someone who is providing a wonderful service.
Sal Khan is “still better than most Math teachers out there.” This line gets used over and over by those defending Khan and claiming he’s the second coming of whatever messiah you believe in. I’d really like to see where this data is coming from. Not a single one of the posts making this assertion provide any research as support or even cite an unbiased poll.
The Khan Academy has great potential to do great good, but a product that has both content errors and pedagogical errors throughout dozens of its videos needs to do better quality control.
In addition, what Khan is doing is not revolutionary; he’s taking ideas pioneered by hardworking classroom teachers years before him and claiming them as his own. The reason KA seems revolutionary is because money shouts very loudly and drowns out other voices so Khan can easily claim that he’s discovered something no other teacher has ever done, thereby bashing the very people whose ideas are making him famous.
In terms of data, it might not be scientific, but telling nonetheless: 174M lessons delivered. This number cannot come from one-time visitors — these are return visitors that love his way of teaching. Millions are learning more, where they had no (free) alternative before. Also telling is the amount of likes and positive comments on his videos (vs negative). These are people who appreciate the alternative, because they consider their Math teacher dull & torturous, hence leaving them unable to retain/comprehend Math..
A survey to back it up would be great, but here’s what Andrew Hacker wrote in “Is Algebra Necessary?”: “A TYPICAL American school day finds some six million high school students and two million college freshmen struggling with algebra.” He goes with a lot of data on the failing matter of algebra among high-school students.
IMHO, the main cause is the de-facto standard teaching style. There are many points I could list why I think KA’s approach is better for the kids struggling with Math, but Sal does a better job at his TED talk: http://www.ted.com/talks/lang/en/salman_khan_let_s_use_video_to_reinvent_education.html
The ideas are not what’s revolutionary — it’s the execution of a simple idea that makes it genius. Online teaching has been around for years pre-KA, but never was there a platform that attracted such a user base.
Google did not invent the idea of indexing the web. Facebook did not invent the idea of Social Media. Apple did not invent the tablet. They all took simple ideas and brought it to excellent execution — and are revered as the real heroes.
It is exactly the zealotry displayed here that led me to examine the statistics videos.
If Khan were still producing charmingly bumbling videos to help his cousin with her schoolwork I couldn’t give a flying fig what they were like.
Precisely because KA is heralded as the saviour of the education system it is imperative that he GET IT RIGHT!
Being free is not an excuse for misleading people.
Khan is not a hero. He is lucky. But he is out of his depth. There are many people in history who have had an immensely loyal and devoted following, sometimes with disastrous results for their country and even the world.
I don’t claim to speak on any other subject, but the statistics videos need peer review.
Khan Academy truly could be many of the things its followers think it is, with examination and critical review by people with deep knowledge of the subject matter and pedagogy.
I cannot climb onto the bandwagon in whole-hearted support of Khan’s Academy nor the one opposed, but I can give my own anecdote. My eldest children have just graduated from grade 12. Having gone through the public school system we can say that our experience has found no mathematics teachers qualified to do the job. Seriously, none. The absolute worst one was the one they had for Calculus this past year. She supposedly has a Masters in mathematics but her grasp of calculus is next to nil and her teaching ability is so bad that its easily argued it is actually counterproductive to attend her classes. The vast majority of her students have to pay private tutors. Yet she is the head of the math department at the high school. My wife happens to be an engineer who, despite not needing to use her calculus skills for a very long time, was able to brush up enough to provide some aid to our kids. And who else did they get to help them, not only with calculus but every other math topic they’ve been faced with in the last 2 or 3 years? You guessed it – Khan’s Academy. We can’t afford private tutors. Now, I haven’t vetted Khan’s videos and have only seen a few minutes here and there over my kids’ shoulders, but those videos have made a huge positive difference for them. His videos may not all be as good as one would hope, but leaving their math education up to the public school teachers was orders of magnitude worse, and that was the stark choice they had. So, Dr. Nic, decry whatever “zealotry” you perceive and even equate Khan’s errors with “disastrous results for their country and even the world”, but know this – when my kids needed an education in math the subset of MarkW’s “hardworking classroom teachers” in our area weren’t capable of teaching math. You weren’t available to them either. Khan was. Don’t get me wrong, I actually do value constructive criticism from those who take the time to evaluate resources like Khan, and I surely do hope he takes all of it to heart. And when you say “My point is, surely we can do better than that!”, I don’t know enough to disagree. But who is “we” and what are they currently doing that makes it a necessity for so many of us peasants to lean on Khan? Rather than mounting a soapbox, Khan took it upon himself to actually do something. Bill Gates saw that someone was doing something and pitched in with money. We have no loyalty to Khan and will switch in a heartbeat if somebody better comes along. Talk is cheap. Go pound on Khan’s door and offer your services, Dr. Nic – now that would be something that might actually benefit my youngest child. In the mean time we’re left with no practical alternative but to leverage Khan’s real world contribution.
Hi Real World Parent
Thanks for your contribution. It is very sad that your mathematics teachers are so poor where you live. There is at least one person doing better maths videos, https://www.youtube.com/profrobbob has much better maths teaching videos than Khan Academy. If your children do statistics I would recommend my videos. It is very hard to find good material on the web. I am writing an update to this post next week, and have been looking for correct, well thought out videos about probability and haven’t managed to find any.
I will work on producing some, which will be available for free on YouTube.
Interesting question – who is we? In my case the blog is written for people who teach statistics, and so I was suggesting that teachers of statistics could make statistics videos. I wouldn’t presume to make videos about chemistry or physics as I don’t know enough, and I have no pedagogical content knowledge.
It is interesting how people have got quite defensive for Khan, when in fact I was quite complimentary about some of the videos.
I’m afraid the probability ones that I have been viewing are worse, so I will be writing my review as an open letter in the hope that KA can improve their offerings so that your children get what they really deserve – correct, well produced learning materials.
I think constructive criticism is important to make sure that such influential and popular materials are worth the attention they are getting.
Hi Real World Parent,
Speaking from the US perspective, for a modest price of ~$300 per course, you can take classes at the Art of Problem Solving (http://www.artofproblemsolving.com/School/classlist.php), which appears to be the best American resource for the extra curriculum high school math. John Hopkins’ Center for Talented Youth (http://cty.jhu.edu/ctyonline/courses/mathematics/index.html) and Stanford’s Educational Programs for Gifted Youth (http://epgy.stanford.edu/courses/math/) have very good high school math-bridging college classes (although they might be a bit more expensive, and are based on above-grade testing — my son took the ACT in grade 7… and scored enough to be admitted to our local public university). So there always are additional resources; none of them will be entirely free, as at the very least one would need to spend their time figuring out which of the immense variety of courses can be and/or should be taken seriously. (We speak Russian at home, so we have yet another layer of possible alternatives from Russian math programs, out of which every one beats any standard American program that I have seen.)
As far as “better than” argument is being given, a standard computer science joke goes like this: Why do they call the preliminary versions of software “beta-version”? Because that’s beta than nothin’. Yes, it is better; but I entirely agree that given the level of exposure these videos get, and the amount of resources that they now have available for their internal quality control, KA should work on improving both the content and pedagogy of their offerings, and get this “right” rather than “beta than nothin”.
Mind you, I wouldn’t trust an MA in math education to verify these videos; however, given the number of retired math professors out there, KA can reach out for such quality control to the real professionals. I was approached by one of the providers of the MOOCs to verify their lectures, but what they promised to pay was may be a quarter of my consulting rate for just the viewing time, let alone commenting time, which is what I can think about when I am asked to do something for money outside of my main job responsibility.
Dr Nic, thanks for posting this to ANZStat.
Dr. Nic wrote:
There are many people in history who have had an immensely loyal and devoted following, sometimes with disastrous results for their country and even the world.
Are you comparing the Khan Academy web site to dictators like Hitler and Stalin? Who’s acting like a zealot here?
Dr Nic: Perhaps someone more like Andrew Wakefield?
From Wikipedia:
” a British former surgeon and medical researcher, known for his fraudulent 1998 research paper in support of the now-discredited claim that there is a link between the administration of the measles, mumps and rubella (MMR) vaccine and the appearance of autism and bowel disease”
Not to imply that Khan is in any way fraudulent, but in many areas, wrong! Unfortunately, I don’t have to scour the scientific press very diligently to find examples of scientific research that is based on an incorrect understanding of statistical principles for use in my statistics classes. These things lead to policy decisions that can affect millions.
Make your own videos then.
I have made several, which you can see on the CreativeHeuristics channel of YouTube. Unfortunately they take a lot of effort to make when you do it properly, and I don’t have the backing of Bill Gates. But I feel I have gone for quality and pedagogical foundations, so I hope you will find them useful. They are pretty popular and get lots of good feedback.
ehh. they are rather boring and bland, need better voice as well, I like Khan videos more
I just wanted to say that I appreciate your dedication to pedagogy of mathematics. One of the reasons I believe people are defensive about KA is because the pedagogical difference may not matter to them. They may already understand statistics (or they may have even learned through KA) and not hit many of the common stumbling blocks that other students hit. Or worse, they may have internalized certain misconceptions about stats that they don’t fully appreciate.
One of the big problems at my university was that professors were brought in because of their research genius, not their teaching ability. Many of them were phenomenal thinkers in their respective fields, but had a hard time with language. It made it a requirement that students in these areas basically teach themselves from the homework problems.
I think this is a common thing. People in mathy areas get used to needing to teach themselves things. This is great! People should be able to teach themselves things! But if you don’t have the same background, a good teacher can make all the difference. And more pessimistically, as I said, you may come to an erroneous conclusion if you don’t have someone to correct your mistakes.
Unfortunately, misconceptions in statistics can mean drugs that shouldn’t go to market, changes in political policies that are based on faulty information, incorrect diagnoses, unhealthy fads, etc.
I think that your focus on pedagogy is phenomenal. I wish I’d had more teachers at college of whom I could say the same.
Thank you. I really appreciate your kind words. I was recently made redundant, in part because my emphasis was too little on research. Research is useful, but I feel that helping people really understand statistics is an important role.
I agree completely. Education institutions have a pitiful understanding of why they exist.
Scientific research at schools with huge endowments seems much better understood by their administrators – bring in lots or research dollars and we value you. Don’t and we don’t. It is still far from perfect as they only value research based on how much research funding it gets (and licensing revenue it generates).
Sadly nearly all universities at some point figured out they shouldn’t focus on educating students they should focus on research and publication. Seems to be an idiotic decision to me.
I don’t know if I’m the only one seeing that but I got the feeling that the only reason you bash him so hard is for promoting your own videos. And I don’t think it’s with this kind of attitude that you’ll get some attention of the Bill Gates Foundation.
Oh excuse me, I’m not worthy to write about this since I don’t got any «… deep knowledge of the subject matter and pedagogy».
I’m pretty sure this is a “no win” situation. Half the Khan supporters say “Make your own then”, but if you do, then they say you are pushing your own barrow. Feel free to watch my videos and critique them!
Khan does videos on MANY different topics, not just statistics. You focus on just one topic, so i would assume it would be a lot easier for you to really go in-depth. Khan’s videos cover chemistry, calculus, bio, organic chemistry etc. He does a good job, because i use those videos to get a general idea and understand the topics in the subject before actually taking the class (or while taking the class).
That is my point, really. Because Khan is not an expert in many areas, sometimes the material is not well presented, and sometimes it is not correct. I wouldn’t presume to make videos on subjects I don’t know much about. Just because you choose to use them, doesn’t mean he has done a good job. I think what you mean is that it is an easy place to find a lot of adequate videos that seem to help students get a general idea. I suggest that it would be preferable to use better videos if they are available, but if not, then Khan is possibly fine in other areas, but is a bit limited in statistics.
I can understand your frustration with the Khan Academy videos, but I believe that his intentions are good and that he is doing the best that he can. He is looking for math content creators who actually know what they are doing more than he does. Have you contacted them about the errors in the videos? Perhaps if they realize that there are problems, they will try harder to get a statistician to redo the content.
[…] I don’t like the Khan Academy videos about statistics. But I can see why some people do. Some are okay, though some are very bad. I’m rather sorry they exist though, as they perpetuate the id… […]
Dear Dr Nic
You, yourself, is a very smart individual. I would think that you should join the Khan Academy to help them succeed in the Statistics field. This way people will respect you more.
There are a lot of people who think they are superior and others people’s efforts are inferior. Very immature. No offense, but your videos on statistics are run of the mill, average videos I was ‘forced’ to watch in math class in the 80’s. Khan’s videos keep me awake and alert and are enjoyable to watch. I keep coming back to them for more ‘voluntarily.’ Your view is biased, you are clearly jealous, you call him lucky when he has helped people pass exams and advance in school. His contribution is much more giving and broader than yours. Please fill out a job application with him and hope you can also be a contributor instead of an antagonist. By the way, we have enough 1980 style videos out there.
[…] for: Khan Academy. I should know better than to suggest that the mighty Khan is less than perfect (my previous post about KA continues to provoke defensive vitriol.) But here […]
Hi Dr. Nic,
Have you considered posting your concerns in the comments of Sal’s videos or writing your suggestions to him? I read an article about 2 physics teachers discovering an error in one of his videos; when they contacted him about the error, Sal swiftly corrected his video and thanked them for it.
I’ve watched him in several interviews, and he strikes me as a friendly and reasonable person who would welcome feedback, especially from a passionate teacher like you. One nice feature about his videos is how quickly people can comment on any confusing parts or mistakes and interact with each other in resolving them; I’ve seen Sal responding to those comments directly and acknowledging those mistakes.
I hope that you two dedicated educators can collaborate to teach all of us better.
Sincerely,
Eric Cai – The Chemical Statistician
http://chemicalstatistician.wordpress.com
https://twitter.com/chemstateric
[…] I’d just like to pop in a reference to Khan Academy because, sadly, it has a great influence. I believe that many of the mathematics Khan Academy videos are fairly well taught, in a “boy-next-door” sort of way. However the statistics videos perpetuate the mathematical view of statistics, as they are a product of an archaic curriculum. Khan has NO pedagogical content knowledge of statistics. This is abundantly clear in the approach and errors. I have covered this in earlier posts. […]
How come he’s famous? How come he’s been useful for so long? Probably because it works. Nobody searches YouTube for the full story. When one wants to get the grasp of, say, confidence intervals, one wants to quickly understand the mechanics. Khan supplies that very well! Later, one goes deeper, beyond Khan to figure out the why and when.
Khan has helped me graduate from a prestigious university, and brought confidence to my skill in math. That skill and confidence in mathematics and statistics provided by Khan later sparked a deeper private interest in math – and because of him I’ve continued on my quest of knowledge and have amassed the skills needed in my daily job (which involves statistics and finance). Without that simple kick start from Khan, I don’t believe I would hqve had such an easy time.
Your videos however, I’ve not seen yet. But instead of pointing out all his flaws right away – know his videos actually help. Tangible results..
I was i initially excited by the prospect of being able to recommend these videos to my maths students for extra help and support (I teach High school) . Unfortunately, while I didn’t find errors in them, many videos jump in difficulty very quickly, skipping a lot of the scaffolding that students struggling to learn need. They aren’t all bad, but as a teacher, the time required to watch each one before I recommend it isn’t there. On the other hand, Khan Academy highlighted what was now possible and with apps like Educreations, Show Me etc.,I can create my own to meet the needs of my students better. Also, other companies like Learn Zillion are producing alternative, better, offerings. So we have Sal to thank for that I guess. But I agree, great idea, but with the amount of money provided, they should be consistently accurate and be based on good pedagogy.
P.S. I did have a look at one of your Statistics videos and it was much better than Sal’s.
[…] Technology has changed the landscape for flipping. With ready access to the internet it is feasible for video and other work to be set remotely for students. Sometimes teachers prepare the material themselves, and sometimes they may specify a YouTube video or similar to watch. This is not as easy as it may sound. As you can see from my critique of videos about confidence intervals, there is a lot of dross from which to extract the gold. And Khan Academy is no exception. […]
Off topic perhaps, but since you mention it in passing, I don’t suppose you’d like to blog a little on the pedagogical knowledge relevant to statistics teaching, would you? A ‘top five statistics student misconceptions (and what to do about them)’ would be kind of a nice thing to see …
Thanks for the suggestion. I’ll put it on my list. I have this underlying fear that I will run out of things to say, and am constantly amazed that I have managed to write over 80 posts so far. That’s a total of more words than my PhD dissertation – and certainly more read!
I’ve been reading this post, and some of the responces.
With all do respect to you (the writer) no one is forcing anyone to use KA, it’s success come from people who open it up, understand the material and decide to continue or not.
Critiq is healthy, your ‘my videos are the best’ and KA are so bad atitude is pathetic.
Im going back to college now with a new major, and seriously sal’s videos have realy refreshed my memory and tought me well high school math.
No one is claiming you will become a mathematics PHD from KA, but your attemp at downgrading this truly revolutionary concept is out of line.
Make your own videos, and be part of education revolution, rather then ditching others who DO
Sorry, I have to chime in here: if you’d read Dr Nic’s article you’d have read that she has made her own videos! What I understand from this article is that the statistics videos are not perfect on Khan Academy. Taking the time to teach a subject so students can understand it thoroughly and build on the material is very important. Dr Nic sums it up toward the end: surely we can do better than peddling mediocrity. This isn’t a torrent of abuse aimed towards Khan Academy, just a request to do things in a better way.
Having watched neither (as I’m sure is the same of many other people commenting here), I know that I’d rather learn from videos that are correct than entertaining; otherwise it’s missing the point ?
I appreciate your work on educational videos. I appreciate everyones work which provides an online alternative to a conventional classroom. I strongly believe that this is the solution to many problems the (mostly western) education system faces like lack of motivation, different skill levels, different interests, bad teachers…
Anyhow here’s why I think Khans videos are better than yours. Maybe that just applies to my way of learning but there are two main reasons why I think Khans style of making things up on the fly – although sometimes it feels a bit amateurish – is the right way to do it.
Firstly, planed, prewritten and edited videos have the same effect on the me as a prewritten and read presentation on the audience. If the content is not extremely interesting or important to me I drift off and do not pay attention anymore. On the other hand if a presentation or a video contains hesitations and emotions i.e. reveals the narrator as a human being, it feels way more natural and interesting and I somehow feel connected.
Secondly prepared videos are to fast and – as mentioned – not natural. They are like a dictionary compared to a novel. They provide correct, coherent and perfect definitions of terms and concepts but they don’t convey the content in the same way the listener thinks. When one is learning a foreign language it is much harder for him to understand the announcer of television news than listening to a conversation on the street. The TV news are way to fast and compact for the listener to follow. Of course you spent much time thinking about the perfect way to teach somebody statistics but in my opinion that’s exactly the problem. You already know exactly what you’re going to tell so your videos don’t have natural breaks and hesitations which are crucial for a student to understand and process what he just heard. Compared to Khans videos, watching your videos I had to pause and replay parts of the videos much more to understand it.
Of course my points only concern the way the content is being presented. I can’t say anything on how correct the content of Kahns videos actually is.
Basically this is an empirical question. If we can first agree on a validated means of gauging student learning in statistics, and then we expose them to (let them know about) different online videos about statistics, which videos produce the most student learning, taking into account that students will perhaps not even view or not pay close attention to videos that are dull? It ought to be possible to set up an experiment along these lines, which can then be analyzed with proper statistical procedures.
[…] I know that statistics is difficult to explain – in fact one of the hardest subjects to teach. You can read my review here. I’ve also reviewed a selection of videos about confidence intervals, one of which was from Khan […]
[…] For those of us that have been living under a rock (myself included) Khan Academy is the latest and greatest thing to come to mathematics education. According to a recent Learn and Teach Statistics blog post “Khan Academy Statistics Videos Are Not Good.” […]
I really enjoyed these videos! I tried Khan during one of my Stats course; however, I did not find the videos sufficiently clear about the concepts that I was trying to understand. In many cases, I found them downright irritating. Thank you, Dr. Nic! I have shared this link with other learners.
[…] statistics can be fascinating, relevant, boring or trivial. My most read blog post is entitled “Khan Academy Statistics videos are not good”. I suspect that quite a few people are searching for statistics about Khan Academy, rather than […]
RE: Khan Academy
To all these folk criticizing Khan’s free education videos, I just want to ask, “Where are the links to your free education videos, your own videos that do this right?”
Perhaps the statistics courses need some improvement. I don’t know about that, but what I do know is that I have seen a lot of huffing and puffing by institutional type individuals, and it seems to me that theirs is a reaction against free and popular education options. And sure, why not? The very existence of the institutions depends upon keeping the public dependent upon the institutions.
Higher education is a money machine. and free education is a threat to that.
Prove me wrong. Show me the links to your free courses, including statistics classes.
I have looked at the free offerings by Harvard and MIt. My impression is that, sure they have thrown us a few little bones of free knowledge. As much as I appreciate those bones, the classes that I found on those sites fell far short of a full range of coursework, even at the undergraduate level.
I am not saying those two universities are wrong to give us a free sample only. But unless and until we really have free public education in this country, people like Khan are a real blessing, so let’s give credit where credit is due.
I’m just wondering if you actually read the post. There are links in there to my free, publicly available statistics videos.
I viewed one of Dr. Nic’s videos briefly. From my perspective, it was well done, with this one exception: For me at least, the frames moved far too fast. Not everyone’s brain processes in the same way, nor at the same speed, due to the different ways in which our brains are “wired.”
The speed of frame changes, in my opinion, need to allow time for the brain to register what is being said. I found it necessary to use the pause button on each frame in order to comprehend the material.
The video would be improved with a gadget that allows the reader to select the speed of the frame changes. This would especially be helpful to people like me, who are deaf or hard of hearing and have to rely upon the captions for understanding. A deaf person needs time, not only to read the words, but then to look at the pictures and finally, to compare the words and images to form conclusions and then to actually store the information in memory.
I hope this comment is helpful I would also like to praise the combination of graphics, linguistic content and computer programming that must have been required to produce this kind of video. A video of this complexity is nothing short of an amazing feat of accomplishment.
Thanks for taking a look. I agree that the videos go fast, and this is a conscious choice. It is easy to pause, think, go back and control the pace if it is too fast. However if the video drags it can get very tedious. You did exactly what is intended. It is also aimed at people in their teens and early twenties, for whom things do go a lot faster.
As you commented, a lot of thought has gone into them. We are now on the brink of producing a whole lot more, now that we have employed an editor.
Well Dr. Nic thank you. I specifically thank you for pointing out the weak Statistics videos at Khan Academy. I quickly reviewed them, found them to be more of a joke and left. I have used Khan Academy to help with Algebra. My goodness, I’m surprised people are so offended by your post. The choice of viewing the videos is up to the individual, but to get weak in the knees out of fear of criticizing Khan Academy is ridiculous. The Statistic videos could use improving and I believe that is your point.
As someone else here has written the Statics videos are not only irritating they’re a joke. “Dare to think for yourself.” ~ Voltaire
Thanks again Dr. Nic!!
your videos are like a talking book, where as khan’s videos are like a person talking to you … which is the reason why people understand more with khan academy … because it seems simple, casual and not restrictive/mundane like a book
however, if your videos complimented a particular book it would make that particular book easier to understand … one more advantage that khan’s videos have is the blackboard software, with that one can actually see the examples being written or drawn that makes learning a lot easier because khan’s videos mix the concept of learning through construction (which many academics prefer: making students work/think through the explanations) with the ease/comfort of an everyday conversation
so basically if you take your video scripts, use them in a conversation (do not fret over bumbling a little bit, it only makes you look human and so called ‘weak’ students can relate to that), use a stylus/graphic tablet and the black board software….. then your videos would be better than khan’s
I just sent my statistics class a warning not to watch Khan’s statistics videos.Then I found this site. Ironic.
I generally like Khan. Personable and trying his best to educate the masses. But, he is not a trained statistician. He has a fair grasp of mathematics and I like his maths videos (at least the ones I watched). But mathematicians don’t have a full grasp of the subtle points in statistics.
1) The confidence interval video is full of misinterpretations. He states that the probability is 95% that the mean falls in the interval. This is classic mistake 101. In fact, I have this interpretation on an exam and if the students agree, they get the question WRONG. How can anyone defend videos that would result in a homework or exam deduction for an introductory stats class? I find the defenses of Khan for his statistics videos on this posting site to be frankly silly. The defenses are emotional and not based on the content itself.
2) I have to disagree with Dr. Nic here in that p-value video is sound. Khan says somewhere that P(xbar = 17.17 given null is true) is the p-value. Oh how wrong is that!! The probability that xbar equals any value is 0. It is not the p-value. Khan should have said p-value = P(xbar <= 17.17 given null is true). It is a one-sided test with a less than alternative. Thus, Khan is doing even a worse job than Dr. Nic states!
3) Yes, his confusion between samples and observations is annoying.
For you passionate defenders of Khan, this is not being nit-picky. These are mistakes, plain and simple and reflect wrong teaching of material. I can't understand how any of the above posts defend wrong delivery. It is insulting to read that some people state that these errors as minor. Wrong delivery of the meaning and calculations of p-values and confidence intervals is as major as it gets. Hits at the heart of statistics.
I don’t understand why there is such a negative tone when talking about Khan Academy; saying that they merely got money thrown at them? If you want to give criticism there is a way to do it. At the moment, all you are getting across is that you are jealous of his success, and are trying to promote your own videos. And you can also CONTACT Sal. If you would care to know, he remakes videos that lack proper explanation. If you just tried to contact him, changes would occur.
As my mom used to say: “Don’t look a gift horse in the mouth.”
I’m not sure the Trojans would agree with you.
I think this person is just a victim of complex! Khan is the man, like it or not!
Good to hear that. I don’t know how anyone would propose doing statistics without math. And sure, there are graphs of all different types and descriptions and methods of collecting data and blaa, blaa, blaa, but when it is all said and done, you gotta do math, or have a computer that does the math. Then you still have to enter the data, perform the calculations and be able to interpret the results. So I think I am with you.
It is not worth it to expose oneself to unnecessary antiquated agonizing jelousy.
My son has been using Khan Academy for a year. Based on our experience, Khan’s video lessons are very helpful. Since the practice on Khan is quite limited, he uses Beestar for online practices. Beestar has many core subject programs including math, language arts, social studies. Its math program is completely free.
My son works on Beestar’s math practice every week. The interesting questions cover almost every math topic taught at school without repetitive work. Besides, the reward system keeps him motivated to compete with other students nationwide. As a parent, I can monitor his progress online by reading report at any time. Sometimes when he finds a new math concept, he would search Khan’s video lessons for explanations. I like the way he uses them. In the past year, he has become very confident in math. I will let him keep using them.
Hi Debby
Thanks for commenting. That is great that your experience with Khan Academy is positive. I am aware that many people have great experiences with Khan Academy. What I am saying is that the statistics videos are not well done, and give the wrong idea about what statistics is about. Sadly so does the curriculum in most countries! I am backed up by many teachers who wish there was a little more pedagogical thought going into Khan Academy material. I don’t know Beestar.
I thought I’d read something valuable…and saw that it’s nothing more than 1) bashing Khan videos and 2) advertising her own videos. Really? If you are going to do former, at least refrain from openly advertising your videos.
And pie graph is really never good for anything. Using that as a “bad” example doesn’t make your videos “good” either.
“A p-value less than 0.05 means that we have evidence of an effect. A p-value of more than 0.05 means that there is no evidence of an effect.” (at about 4:20 in “Understanding the p-value: Statistics help”) No, no, NO! This is flat out wrong on several levels. First of all, you’ve made an arbitrary cutoff (the alpha level) serve as a benchmark for saying that there is “NO evidence” (emphasis mine) of an effect if the p-value is greater than this arbitrary cutoff, and that it IS evidence if it’s less than this cutoff. Hence, an arbitrary cutoff serves as a binary function for what even constitutes evidence. Second, why have an alpha value at all? Why not just report the p-value and let the reader decide whether or not it’s significant? Third, there is much interesting and important history here which gets lost in this sort of explanation. It was Sir Ronald Fisher who invented significance testing and the p-value notion, but it was Jerzy Neyman and Egon Pearson who thought that they improved upon it by introducing alpha and beta and type I and II errors and all that, which we can call hypothesis testing (as opposed to Fisher’s significance testing). Fisher disagreed that it was an improvement, and Fisher’s criticisms were spot on, certainly in my opinion. To Fisher, a p-value did have inferential meaning, whereas to Neyman, such a concept was almost meaningless. He was concerned instead with what he called inferential behavior. The underlying philosophical differences are subtle but very, very important! Unfortunately what is by far most commonly taught today, including in your video, is a bastardized hybrid of which neither Fisher nor Neyman would likely approve. For Neyman a cutoff for one action versus another was critical, and since he valued behavior instead of inference, ALL that matters is whether or not the p-value exceeds or does not exceed a pre-specified alpha level. That means that reporting “p < 0.05" (for conventional alpha) is ALL that really matters. Again, Neyman was concerned with inductive behavior, not with inference. In contrast, Fisher did view p-values as having inferential value, and while he made tables for various cutoffs, working as he did in an time before fast computers, I doubt that he would see the need today for such cutoffs. Instead, I think he would agree that it would be better today to report the actual p-value and let the reader infer how significant or not it really is. For Fisher, but not for Neyman, a lower value was stronger evidence than a higher one, and in fact squabbling over a particular cutoff for using the s-word (i.e. "significant") shows immediately just how subjective such a cutoff really is. If we take p-values as having evidential meaning, as Fisher did, then there is actually no point in a pre-specified cutoff. That is, the cutoff has meaning with regard to some action to be taken but not with regard to the strength of the evidence. (There is more to this topic than I can go into here, but this is one consequence of their different philosophical views.) Fourth, a p-value cannot really be described without a discussion of the sample space and the statistical model chosen, INCLUDING THE STOPPING RULE. A p-value without this information is, technically, undefined, since one could use the same data and get different p-values by changing the statistical model, or the stopping rule. Fifth, since a p-value is literally the sum of probabilities of events that could have happened but did not, given that the hypothesis to be nullified happens to be true, it violates the likelihood principle. This has important implications for the philosophy of evidence. I would suggest Googling "Richard Royall nature of statistical evidence" for some very interesting insights into the nature of statistical evidence. Obviously there is much more to this topic than I can write here. I would suggest teaching the conflict behind Fisher and Neyman and how their different views impact statistics. It makes the subject much more interesting knowing that real people had real differences of opinion.
Thank you for your full explanation. I am aware that there are some inaccuracies in the video. In the next edition I will attempt to address some of your concerns.
And thank-you for your genuine interest in improving statistics education. I’m glad my comments were helpful. I do think that the likelihood paradigm (between Bayesian and frequentist) deserves more attention than it gets, and Richard Royall’s got some short articles that are a good place to start on that. I also firmly believe that the philosophical aspects lurk just beneath the surface, are very important, and they inform the computations, as well as help to make the subject more interesting (together with the historical gossip by leaders of the field who often disagreed profoundly).
Another minor point: according to SI usage (but unfortunately not according to common, illogical and inconsistent usage!), there should be a space between the number and the unit. It’s distracting to see in your videos “23g” for example, when it should be “23 g”. The SI rules are very clear on this and also very sensible. For an educator, I think that this is a minor point that’s still worth knowing.
Thank you for this also. I will do better in future, and encourage my editor to also follow these rules.
Your math may be good, but your English is not. “Maths” is not a word. Math is both singular and plural, with the differentiator being context.
I supposed you say “underwares” too?
Hi George. In most English-speaking countries the shortened form of mathematics is “maths”. I know in the US they leave off the “s”, but in the UK, NZ and Australia we leave it on. And I’m not sure what you mean by underwares. Do you mean underwear?
[…] to do and then practising. I suspect a large proportion of maths teachers also liked doing that. Khan Academy videos are wildly popular with many learners and far too many teachers because they perpetuate this […]
[…] to do and then practising. I suspect a large proportion of maths teachers also liked doing that. Khan Academy videos are wildly popular with many learners and far too many teachers because they perpetuate this […]
[…] also wrote a critique of Khan Academy videos, explaining why I felt they should be improved. Not surprisingly this ruffled a few feathers and […]
Hi Dr. Nic! I enjoyed your article and read about half of the comments. I agree that there is a big need for educational videos with strong teaching. I too have spent many hours thinking about how to teach – in my case, Math, and specifically algebra and trig as well as Kinder and 1st grade math. I have made some of my own videos, but I agree with you that to do it well takes a LOT of time. I am currently exploring career possibilities for myself, and while I’d LOVE to create math videos, or do one-on-one tutoring, or support the homeschooling community with quality math instruction, I can’t figure out how any of those options would earn a decent amount of money. How do you monetize your videos??? Thanks for any advice or support you have for me, a fellow math-lover who wants to deliver quality math instruction on videos.
Hi Jen. Thanks for your support. I hate to say it, but if I knew how to make a reasonable living out of this sort of thing I would be thrilled! You have to get HEAPS of clicks on YouTube to get any money at all. Unfortunately most people expect to get things for nothing, especially in education. We have online statistics courses for high schools students in New Zealand, that gives us a baseline income, but without that we would be stuck. One person who does ok, I think, is Rob Tarrou, but he has heaps of videos. Good luck!
[…] I do wonder what message it sends when people like Sal Khan of Khan Academy and Mister Woo are applauded for their well-intentioned, and successful attempts to take a […]
Haha, people trying to promote there own stuff by bashing others is great to see…