Why value delivered is a misleading metric

A recent conversation I was having turned to the idea of standing development teams and measuring performance in terms of value delivered.

There’s been some debate back and forth between colleagues and myself about whether value is something that IT should measure. After all, it commingles elements IT can’t control (the goodness of the business idea) with their delivery. But, that aside, let’s assume we ought to measure IT including value in some way.

The idea of measuring value delivered seems to make sense for fixed sized teams. Under typical circumstances you always want to account for opportunity in your measures. For example, if you were comparing two teams, one of 10 people and the other of 100, you’d intuit that 100 people are going to turn out more value than 10 simply by virtue of being an order of magnitude larger. However, if your team size is fixed, then adjusting for team size doesn’t matter… You’d be taking value delivered and dividing it by the same number every time.

However, there still is a problem with just measuring value delivered. Let’s say you are chugging along delivering business value and the business says “we’d like even more value! Can you hire another person?” Your team size would still be reasonable, so you say yes. Indeed, with the new person on board, value delivery rises, so everyone’s happy right? Well, not necessarily. Adding the person added value, so your line slope of value delivered would rise, but did it add enough value to overcome the additional cost? Well, that depends. If a team of 10 was delivering $1m in value then a team of 11 ought to deliver at least $1.1m in value in order for the proportion of value delivered to stay constant. On the flip side, if your team shrinks from 10 to 9, you’d expect value delivered to drop some as well. In fact, if it dropped from $1m to $950k, the proportion of value delivered to team size would actually increase! And by the way, if the value delivered didn’t drop with fewer people on the team, what does that say about the contributions of the former member(s)? When your business folks say to you “IT costs too much” what they are perceiving is the value they’re getting for the cost, not just the value and despite the statement, not just the cost either.

Of course, if you only measure the value half and not the costs half (whether that’s in people on the team or hours billed or whatever) then you’ll never know this. Capers Jones in his research has pointed to evidence that larger projects experience lower productivity, or in essence, smaller amounts of value delivered per unit of effort exerted. The idea of simply attempting to maximize value delivered underpins the mythical man month that Fred Brooks wrote about so many years ago – the incorrect belief that if ten people are good, then twenty must be better. To know the optimal mix for your organization, then you must attempt to measure productivity, not just the value half.

The problem with value delivered is that it’s a useful measure only as long as the team stays fixed in size in perpetuity. That’s unrealistic. If the team size changes for any reason (hiring, quitting, leave of absence), even temporarily, you must account for the rate of value delivered and not just the sum total value. The BLS reports that median tenure of an employee is just around five years, so you ought to expect some instability in your team over time. Otherwise, when someone comes knocking saying you aren’t delivering the value they expect, you’ll have no basis for the conversation.

Leave a Reply

Your email address will not be published. Required fields are marked *