Occam’s Razor

In fairness, Occam’s Razor is not simply “the simplest explanation is the most likely” but that’s what it has come to mean in most communication.  The full idea of Occam’s Razor is that we should lean towards simpler explanations until adding complexity yields additional explanatory power.

Sometimes, we hear a simple explanation, and (whether we are aware of Occam’s Razor or not) we tend to believe it.  It’s so obvious, so simple, so straightforward that we do little to question it more.  When dealing with software, we see this behavior regularly when conducting reviews of projects or explaining metrics.

For example, imagine that I have a set of data which shows that running more test cases yields more defects.  I see this pattern across a hundred projects, and so I take my data, in isolation, to several project teams.  The first team sees the data and says “well, that doesn’t apply to us, because on this project, we did data driven testing.”  The second team sees the data and says “that doesn’t apply to us, because on our project, a lot of the code was delivered late.”  The third team says, upon seeing the data, “that doesn’t apply to us, because on our project, we did a lot of exploratory testing and didn’t have formal test cases.”

Taken in isolation, any one of the explanations seems reasonable.  And yet, though each team can find reason why the relationship that appears in the data is a fluke for their team, eventually one has to bring it all together and look for a unifying explanation.

The simple explanation is that more testing yields more defects.  It’s elegant (and reasonably obvious).  By comparison, imagine trying to explain (or explain away) the relationship in the data using each team’s viewpoint.  “The reason this data is no good is because of exploratory testing, late delivery of code and data driven testing… ” and on and on as each team arrives at an explanation that individually seems reasonable enough, but when aggregated together suddenly looks like a whole lot of story telling.

Patterns and relationships, whether causal or simply just correlated come from somewhere.  The likelihood of a complex pattern arising out of pure random chance is low, and yet we attempt to explain away each data point with an anecdote about why the data doesn’t apply here.

Why believe all these complex explanations which offer no stronger explanatory power than the simple explanation?  The trick here is to see the frame of reference where the simplest explanation is no longer winning out.  If you look at any one project, their explanation seems just as simple, maybe more so, than the one that the overall pattern of data suggests.  But, if you take all the simple, but different, explanations and add them all up to form an explanation for every data point in the set, suddenly the story isn’t so simple anymore.

I encourage you to look broadly for reasons why patterns appear in your data, and to challenge the notion that even if the team can tell a good story about their data point, that it may not be supported by Occam’s Razor.  Each explanation may be simple, but only in isolation.