The Rule(s) of Three and the Probability of Nothing

A Re-edit and Posting of a 2008 Blog

From 1827 Pike's Arithmetic

In my youth, back when dinosaurs roamed the earth, there was “the rule of three”… singular, one, and even then the name was often described as “archaic”. More modern books tended to develop “properties of proportions” or similar terms for the problems of proportionalities. Now there seem to be an abundance of them; including one for witches, and one about businesses. There is not space enough to talk about all of them so I will mention three, of course.
The first rule of three is as old as math, and shows up at least as early as the Hindu mathematician Brahmagupta, and in Fibonacci’s famous Liber Abaci(1202). It was once so common that it was introduced into common language. Abraham Lincoln is quoted in his biography as stating that he learned to "read, write, and cipher to the rule of 3."   So common that student's often wrote verse like the following, in their copy (practice) books.

Multiplication is vexation;

Division is as bad;

The Rule of Three doth puzzle me,

And Practice drives me mad

The most common and longest living form was the direct rule (although there was an inverse rule as well), in which case three numbers would be given and a fourth sought so that the ratio between the third and fourth would match the ratio between the first and second; a:b = c:d. Today students use the ideas in elementary school to complete fraction equivalences, “2/3 is the same as 10/?” Some of the ancient examples grew incredibly complicated.

I suppose the reason I chose to address three of the many “rules of three” is because of the rule of three from language and literature. Three just seems to be the right number for lots of things, there were Three Musketeers, Three Stooges, and Three Coins in the Fountain. It was Goldilocks and the Three Bears, and “bah bah black sheep” had “three bags full.” Comics in the newspaper usually have three panels and many jokes involve a three part ritual where the punch line is the third element, such as the t-shirt with “Great Cities of the World” on the top, and below, one after another, “Paris, Rome, Fargo”. The first two make the last funnier. In language the examples range from “Blood, sweat, and tears, to veni, vidi, vici. If you don’t think there really is a mental tendency to have three terms, consider that in Churchill’s speech, he actually used four; “I say to the House as I said to ministers who have joined this government, I have nothing to offer but blood, toil, tears, and sweat. “  But who cares what he said, you will hear the phrase almost always as "Blood, sweat, and tears, and it was common usage before and (however long they last) after the band.

The final rule of three I would mention is from statistics, and is of more recent origin. It is also, I think, a really clever solution to what is a really difficult problem. Suppose something never happens; how can you assign a probability to it? It is not that it might not happen some day, just not so far. It is just such a problem the statistical rule of there was created to handle. Suppose you stopped at the same gum ball machine every day, but unlike the normal gumball machine, this one did not have a glass you could see into the gumballs inside. You buy a gum ball every day and get red ones, and green ones, but never a blue one. After a while you begin to wonder if they even put a blue one in the machine. So one day, after 20 days of getting all the other colors, over lunch you ask your local statistician (doesn’t everyone have lunch with a statistician?) how to figure out if there really is a blue one in there. He pauses, fork poised in mid-air, and informs you that you can be 95% sure (a common statistical benchmark) that the proportion of blue gum balls is no greater than 14.3%. He had mentally taken three, and divided by one more than the number of failed efforts, to get 3/21 or 1/7 as the upper limit of the possible fraction.
The idea is base on a simple extension of the binomial probability. If you knew that P % of the gum balls were blue, then you could calculate the probability that None showed up in 20 days. The probability would be (1-p)²⁰. Working back through this calculation many times you might notice that the number followed a pattern, a rule of thumb to calculate without tables and calculators, and that turns out to be 3/(n+1), the statistical rule of three, giving a probability for the Blue gum balls as 0< P < .143.  If you wanted greater certainty, you can use the rule of seven, which says that 7/(n+1) will give the 99% interval boundary. So in the case of your gumballs, you can be 99% sure the percentage of blue gumballs is less than 1/3. (Of course this problem assumes a population that constantly replaces the gum ball removed so that the probabilities remain constant.

But what if after a long string of failures, you have a success.  How does this change your confidence interval?   Thanks to a recent post from John D Cook I now can tell you that as well.  

So suppose you had worked your way as before with twenty failures to get the blue gumball, and then after the aforementioned lunch with a statistician,  you get a blue gumball on the 21st try.  Now what can you say about the expected percentage of blue gumballs.  

After the first success according to the Beta distribution would give a 95% confidence interval of appx. [.1/n, 4.7/n]  .  For our imagined 21 tries, this would be about [.0047, .224]  So our confidence interval has opened up considerably.  

It appears, if I understand correctly, that the blue gumball could have occurred anytime among the first 21 tries and thus would still be the CI.  So if we went another nine tries without success, we would adjust our CI to {.1/30, 4.7/30] ... [ .00333, .157], back much closer to our expectations before we ever had a success.

Comparing this interval to the binomial confidence interval you learned in high school math, p +/- 2 sqrt(p*(1-p)/n).  The customary warning on the normal expectation is beware of p being too high or too low.  Using one success in 30 tries we get a 95% CI of [-.03, .099]... perhaps the negative lower bound is a sign that we have strayed to close to zero with our p-hat.  A nice topic to spring on your AP stats teacher when you get to confidence intervals, but please be kind.