lies, damned lies, and statistics

"There are three kinds of lies: lies, damned lies, and statistics."

This quotation, which Mark Twain borrowed from Benjamin Disraeli, is probably the most popular saying about statistics (I have no statistical evidence to back up this claim). However, the statistics themselves are not lies—it's the interpretation of them that sometimes are. Uncovering the lie can be tricky; it could be a data collection error, a poorly worded survey, placement of causation when only a correlation is observed, or just bad logic.

Yesterday, Mike Bianchi of the Orlando Sentinel wrote an article about the impossibility of a team with home court advantage winning a best-of-7 series after losing the first two games at home. This article was written before Game 2 of the Orlando/Philadelphia series and he was discussing the odds of Orlando being able to come back down 0-2 if they lost Game 2 (which they didn’t). Here’s his reasoning:

"Dating to 1947, there have been 378 seven-game series played in the NBA and only three teams have lost their first two playoff games at home and ended up winning the series. Translation: the Magic would have less than a 1 percent chance of winning the series should they lose tonight."

In this scenario, the statistic isn’t wrong, Mike Bianchi is wrong. It does measure something, just not what he says it does. Essentially, it is a rough estimate of the probability of a team with home court advantage losing Games 1 and 2 at home and winning the series. This is not the probability of a team with home court advantage winning the series that already lost Games 1 and 2 at home. In this case, instead of having the total number of series played as the denominator you have to use the total number of series where the team with home court advantage lose Games 1 and 2. Let’s call that number x, which is undoubtedly much smaller than 378. Therefore, 3/x is much larger than 3/378. The subtle distinction in phrasing creates a significant difference in numerical value.

To illustrate the point, I’ll use data that goes back 20 years (partly because I haven’t collected data that goes back to 1947 and a lot of the formats were significantly different than they are now; instead of 2 games at home then two away they would often trade every game). Out of 188 series played, there were two occurrences in which the team with home court advantage lost Games 1 and 2 at home and went on to win the series and eight occurrences where the team with home court advantage lost Games 1 and 2 at home and went on to lose the series. Applying Bianchi’s logic, had Orlando lost last night, they would’ve had a 1.06% (2/188) chance of winning the series. If we continue to use his logic to calculate Philadelphia’s odds of winning the series had they won last night, it comes to 4.26% (8/188). It now becomes clear that there’s an error in his logic since there’s 94.68% (100 - [4.26 + 1.06]) chance that no one wins the series in this scenario.

Bianchi’s error is that he uses a given event (that the team with home court advantage will lose Games 1 and 2) that is highly improbable and includes that low probability into his calculation. In the example above, rather than using 188 as the denominator, we would substitute it for the total number of series where the home team lost Games 1 and 2. So the actual historical value is 20% or 2/10, rather than the 1% chance Bianchi would give a team in this situation.