This book is -- hands down -- the best book on probability and randomness written for the general public.
I am typically disappointed with pop-science books written for the general public... they usually don't present enough data for me to make up my own mind about their conclusions... and when they do present data, they almost never provide enough details to determine whether or not their results are "statistically significant." In other words, how do they know that their "supporting data" isn't just a great big coincidence???
This book bucks that trend big time, and the results are very impressive.
The author wrote this like a history book about the field of statistics, and how it evolved (slowly) over they years. But, every time the author introduces a new concept in statistics, he also shares real-world situations where people made terrible mistakes because they didn't understand these basic principles. It highlights very well that in general, people are terrible at recognizing randomness, and thus will always be controlled by it!
Randomness is very unsettling to people, as such we have a tendency to give order and purpose to the world... People frequently see patters where they just don't exist. One great example of this was discovered fairly recently, called "regression towards the mean." The author's example was as follows:
A psychologist was visiting a group of Air Force instructors after World War 2 to help them design a new training program. He was telling the instructors that positive feedback was much more effective at getting people to learn than negative feedback. At which point, an instructor jumped up to yell at the psychologist for talking hogwash.
"When my students have a good day and I praise them, the next day they slack off and don't do as well. But, if they do badly and I yell at them well, the next day they do MUCH better. So don't tell me this 'positive feedback' garbage works, because it doesn't!"
Surprisingly, both the psychologist and the instructor were right! Regression Towards the Mean means that on average, people perform at the average of their abilities. It doesn't sound monumental, but people forget it sometimes. If you flip a coin enough times, eventually you'll get 100 heads in a row. Likewise, if you do a task enough times, eventually you're going to have a 100-item winning streak that is entirely random luck... and people will incorrectly assume that it's because of greater competency. Sure, you need to be competent enough to perform the task, and you need to do the task very frequently... but after that, any winning streaks are probably just dumb luck.
It's the same reason why some mutual fund managers do better than others, but only for a few years... it's why Roger Maris beat Babe Ruth's home run record, and then very few records after that... it's why some CEOs do amazingly well at one company, but then crash and burn when put in charge of another. You need only be as talented as the average to have a great winning streak...
My other favorite section was when the author covered false positives on medical tests. Most people -- most doctors even -- aren't good enough with statistics to understand when medical tests lead you astray. Towards the end of the book -- after a highly readable introduction to statistical theory -- he presents the following question:
- assume breast cancer is present in 0.8% of the population
- assume a mammogram says a healthy person has cancer 7% of the time (false positive)
- if a mammogram says you have cancer, what are the odds you actually have cancer?
Most people would assume that the answer is 93%, since there is a 7% false positive rate. When asked to a bunch of doctors, the average answer was actually around 70%. But the correct answer is much different: if you test positive for cancer, there is only a 10% chance you actually have cancer!!!
How is this possible? Do the math... out of 1000 people getting a mammogram, 8 will have breast cancer (0.8% incident), and be told so. However, because of the false positive rate of 7%, another 70 people will be told that they have cancer, when in fact they don't! That's 78 people who test positive for cancer, but only 8 actually have it... therefore, if you test positive for breast cancer, there's only about a 10% chance you actually need to worry. In fact, even if you get 6 positive mammograms in a row, you still have better than a 50/50 chance of being totally healthy! Medical tests for rare diseases are fraught with this kind of problem, and it's a shame doctors aren't better at telling their patients the true odds.
Overall, I would recommend this book to everybody. It is really easy to read, and it is chocked full of examples where very smart people made very bad decisions... simply because they didn't understand how randomness rules our lives.