Reasoning about probability is hard

The Monte Hall Problem

Here is a puzzle that often stumps people that can illustrate some ideas.  It is the “Monte Hall” problem:  There are 3 closed doors, behind one of the doors is a prize, behind the other two are gags.  You are asked to pick a door.  You reason (correctly) that the prior probability of the prize behind each door is the same: 1/3.  So you pick door 2 because you like its color.

Monte then opens door number 3 and reveals a gag.  You are relieved, but not for long. He poses the question, “Do you want to change your choice from door 2 to door 1?”

Do you?

You ask your two friends, Simplicio and Salviati for their advice. Simplicio says, “Monte eliminated one of the choices; We had three, now we have two.  Now it is between door 1 and door 2.  Its obvious that the prize could be behind either – its 50-50 which.  You like door 2’s color.  It was your pick.  It doesn’t matter so you might as well stay.”

Salviati frowns but being a good natured fellow he tries not to criticize.  He just clears his throat, smooths his frock and slowly starts talking.  “I think Monte has provided some very useful information.  You see, Monte knows which door hides the prize even though you do not.  Before Monte opened door 3, there was a 1/3 chance that you had hit the prize door with your guess, and a 2/3 chance that you missed the prize.  Don’t you agree, dear Simplico?”

Simplico nods.

“Now that Monte has opened a door, it really does not change the fact that you had and have a 1/3 chance of being right. Monte revealed no new information about whether the prize is behind door 2.”

“If you do not switch, 1/3 is your probability to get the prize. However, if you have missed (and this with the probability of 2/3) then the prize is behind one of the remaining two doors. Furthermore, of these two, Monte has opened one, leaving the prize door closed. Therefore, if you have guessed incorrectly and now switch, you are certain to get the prize. Summing up, if you do not switch your chance of winning is 1/3 (the same as when you first picked door 2) whereas if you do switch your chance of winning is 2/3.”

“Another way of putting this is:  Suppose Monte instead of opening door 3 had offered to combine doors 1 and 3 and said you could keep the contents of the combined doors if you switched as long as you gave him back the gag. Then you would switch in an instant because the probability of the combination of doors 1 and 3 having the prize would be 2/3 whereas sticking with door 2 would have a probability of winning remaining 1/3.”

Simplico has learned something about conditional independence and how probabilities change when new information is obtained.  This is the essence of probabilistic logic.

Another Probability Logic Example

This example is adapted from and example by Tom Minka. 

A new friend tells me that he has two children and he does not elaborate. Assuming that chances of a child being a boy or a girl are even, and the sex of different children are independent, the probability, a priori, of my friend having one boy and one girl is ½.  The other possibilies, two boys and two girls each have probability of ¼.

Suppose I ask him whether he has any boys and he says “yes.”  The probability that the other child is a girl is, by the reasoning above, twice that of the probability of the other child being a boy.  So the probability of the other child being a girl is 2/3. 

Suppose instead that I happen to see one of his children come up to him and that child is a boy.  What is the probability that the other child is a girl?  The probability is ½ because observing the outcome of one coin flip (or the sex of one child) is independent of observing the outcome of another coin flip (the sex of the other child).

The two cases are different.  They appear to many people to be a paradox because at first blush it would seem that we are conditioning on the fact that at least one child is a boy.  But a closer look reveals that the events are really different: In the first, my friend answered my question and revealed information about his two children.  In the second, the event was the appearance of one of the children.  This difference in how the arrival of new information changes our probability calculations is a bit different from how we are accustomed to applying information in formal logic. Probabilistic logic provides a new expressiveness whereby the way that we acquire information changes the implications that we may derive.