There is something that really bugs people about chance. Fictional detectives sometimes say “thare’s no such thing as a coincidence”; I hope real detectives don’t say this, because if you think that every pattern means something you’re well on the way to Beautiful Mind-land, looking for coded spy messages in the recipe section of the newspaper.
W.H. Auden once said that those who speak of random events have forgotten how to pray, which I take as one of many examples of how Auden’s great poetic gifts were sabotaged by his need to have opinions about everything.
And of course Einstein, who was notoriously unhappy with the role of probability in quantum mechanics, claimed that God does not play dice with the universe. (This odd invocation of divine authority reminds me of my engineer father-in-law’s habit of deflecting unanswerable questions with “Because that’s the way God wants it,” which one of his grandkids misinterpreted as a sign that he had gotten religion in his golden years.)
But perhaps the current generation has reconciled itself to these things–where people used to drive the language police crazy by describing any coincidence as “ironic,” nowadays the young folk will say “How totally random!” I’m not sure going to the grocery store and running into an old friend is any more random than going to the grocery store and not running into an old friend…still, it’s probably a step forward.
This is mostly by way of introduction to my favorite example of the weirdness of probability. There was a game show back in the day called Let’s Make a Deal, where couples would dress up in hideous costumes (try imagine how bizarre you had to look for people in the ’70s to know it was a costume) and the host, Monty Hall, would offer you hideous stuff that was hidden behind one of three doors (console stereos with flasing lights, a year’s supply of Tang…). The Monty Hall problem (I think the name is a parody of the famous Monte Carlo distribution) goes like this. Your awesome prize is behind a randomly chosen door, the other two have booby prizes. You are asked to choose a door and you say “I’ll take what’s behind (say) door #1.” Monty then directs his lovely assistant to open one of the other doors (say #2), revealing a booby prize (he never exposes the real prize when he does this), and asks if you want to stick with #1 or switch to #3.
What should you do?
Most people say what I said when I first encountered this puzzle, which is that it doesn’t matter, the prize is equally likely to be behind door #1 or #3. And like me, most people are wrong. You definitely want to switch, because 2/3 of the time you will win the prize. (Of course, if you don’t want a year’s supply of Tang the whole thing is kind of pointless, but we’re not supposed to think about those things when we’re doing a math problem.)
So how can this be? Shouldn’t door #1 and the other hidden one (2 or 3, in this case 3) have the same probability of having the prize behind them? Isn’t that what ‘random’ means? Look at it this way: if we do this 300 times, the prize will be behind door #1 about 100 times, and behind one of the other doors 200 times. When we pick door #1, we’re going to be right 1/3 of the time. When Monty chooses one of the other doors to open, he doesn’t move the prize–it will still be behind #1 1/3 of the time. Now we know it isn’t behind the other opened door (he always picks a door that doesn’t have the prize), so in the 200 cases where it’s not behind #1, it has to be behind the remaining door, and that’s your best bet.
The trick here is that Monty knows where the prize is–if he didn’t, he would sometimes reveal the prize when the second door was opened–and his use of that knowledge changes everything. In general, one might say that randomness in the macroscopic world is usually tied to ignorance: if you knew the precise velocity and spin of a coin, you would be able to predict whether it would land heads or tails, but as long as you don’t know, it’s random. Similarly, people used to get ‘random’ numbers from tables of the digits of pi or e. Of course the digits are worked out from a formula, but there are no repeating patterns, and if you don’t use the formula the digits are completely unpredictable.
Weirdly, this seems not to be true on very small scales. As you probably know, quantum mechanics thinks of particles as having not a fixed location but a wave function that specifies the probability that it will be in various places if we measure its location. Plus, it is impossible to know precisely a particle’s position and momentum at the same time. It is tempting to suppose that the randomness lies in our ignorance, that is, an electron has a position all the time but we only know it approximately, and the wave function describes how well we can predict its position (which is called, in this scenario, a ‘hidden variable’).
But according to a result called Bell’s Theorem, that can’t be true. Or if we want to hold onto the idea that particles have specific attributes even when we’re not observing them, we have to buy it at the expense of something even freakier called non-locality. The simplest solution seems to be to regard probabilities as the basic reality. Just one of several ways in which quantum mechanics continues to make my head hurt…for another, look up “quantum eraser.”
Here’s the Wikipedia article on Bell’s Theorem, if you’re curious: