Is variability in software development good?

I myself wrote an article not too long ago on the subject of variability in process. Under some circumstances I think variability might be desirable, although I wasn’t particularly referring to software development. Last week I attended a webinar hosted by ITMPI and given by one of the employees at QSM. His talk was about the measurement of productivity in IT, specifically focused on how to account for variations in productivity when estimating. The talk was pretty good, but one of his early slides bothered me.

On that one slide he argued that software development wasn’t like manufacturing and therefore productivity couldn’t be measured like manufacturing does. Unfortunately, he offered no alternative during the talk. Instead he focused on how to measure unexplained variations in project outcomes and to aggregate this into some sort of vague productivity calculation. On the whole, useful for estimating if you just want to know the odds of a certain effort outcome, but not so useful if you want to learn about what factors impact productivity.

It’s true that software development doesn’t have a lot in common with manufacturing and the analogies drawn are often strained. That’s not so concerning to me, as the spirit of what management is often asking is right – what choices can I make to do this faster, better or cheaper. In that context, productivity isn’t just something you find out about after the fact, it’s something you want to understand.

With my own research, we’ve found measurable evidence that certain activities do make a difference in productivity. Colocation is worn about 5% better productivity. Committing resources to projects is worth about 0.4% for every 1% more committed on average your team is to a project. Which gets back to the question I posed in the title: is variability good?

In short, no. But the longer answer is, just like any process, you have to know what to control. With a highly thought intensive process, there are things you can and should seek to control to produce more predictable outcomes. It is true that software development is a more like the design of a product than the manufacture of one, but that doesn’t mean that anything goes.

Velocity and standard work

LEAN has a concept called standard work times which is an expectation of how long some task should take.  For example, it should take you 1 minute to mill part X from a block of steel.  Having expectations of how long something should take is important in manufacturing, because it helps you assess whether an individual or team is meeting those goals.  The measure for standard work shouldn’t come from some desire for a certain output, it should be based on how long those things really take.

In software, lots of measures of standard work have been generated over time, but we don’t seem to use them much.  I recall, from my earliest days of programming, that a programmer was expected to create about 5000 lines of code per year.  I’ve also heard estimates for debugged lines of code per day (10-25 or so, if I recall correctly).  Capers Jones’ expressed measures of productivity in FP delivered per day.  We seem to have some understanding of what a typical productivity expectation is.

And then we ignore it.  For some crazy reason we decide that the “laws of physics” don’t apply to us and we need a measurement that is unique just to us.  In Agile, they call it velocity.  It’s a measure of the team’s productivity based on story points per day.  What’s a story point?  Well, that’s a relative sizing mechanism the team determined.

The problems with velocity are numerous:

  1. It’s only good for the exact team you’re currently on.  Sadly, team members come and go.  As anyone can tell you, people are not protect-able resources.  Even if you don’t want them to quit, they will.  And when they do quit, your measure of productivity walks out the door with them.  The new person will need ramp up time.  The new person may be more or less productive.  The new person might not gel with the team.  And until then, your expectations of productivity are gone.
  2. It’s only good within the team.  You can’t compare cross team productivity if everyone’s got their own way to measure it.  I’m positive that I’ve argued before that comparing yourself to other companies isn’t necessarily a worthwhile activity, but being able to compare yourself to yourself is.  How do you know you’re getting better (or worse) if the ruler keeps changing size?
  3. It can accept lower productivity.  Sure, even if you accept #2 above, that you don’t want to compare yourself to your competition, you have no gauge as to whether better is even achievable.  Software is a relatively slow process, so you don’t get lots of data points out of it.  If you have to wait for your team to build up a measure of productivity, you could be accepting a lot lower productivity than is likely achievable.  Being able to compare yourself to a typical expectation of productivity provides a measurement it will take far longer for you to build up on your own.

I understand that there are people out there who believe software is more of an art than a science; that software will be practiced by small teams doing great things.  I happen not to agree with that.  It may be true some of the time, but certainly not all or even most of the time.  In most cases, software is more likely to be like building houses – unique but largely assembled from similar pieces.  And with that in mind, software is more predictable than we like to admit.  But predictability means you can have expectations of standard work times, and that you needn’t allow each team to define productivity in their own vision.

Inconsistent colors

I really like chocolate, so many days after lunch, I’ll stop by our local Lindt store and pick up a few pieces of chocolate for a sweet bite in the afternoon.  But there’s one thing that drives me crazy about Lindt… for years, they really didn’t offer much besides chocolate bars and their truffles.

When you bought a mixed bag of their truffles, you always knew what you were getting by the wrapper.  Brown are hazelnut, orange are peanut butter, red are milk chocolate and blue are dark chocolate.  For years, Lindt quietly and unconsciously trained us to associate certain colors with certain flavors.  Then, they introduced new lines of chocolate.

Suddenly you could buy small, individually wrapped chocolate squares, crisp caramel clusters, coffee flavored chocolates, tiny little chocolates that played on famous desserts like creme brulee and more.  They also introduced marzipan chocolates – one with a milk chocolate shell and the other with a dark chocolate shell.

Can you guess what color the milk chocolate marzipan candy was?  You’d think it ought to be red, right?  Well, no, it’s blue.  Why?  Who knows.  And the dark chocolate one?  Obviously, it’s red.  Years and years of training people that red = milk and blue = dark, and then you introduce a new chocolate where the only difference is the type of chocolate shell and you switch the wrapper color?

It seems like a little nothing, but every time I go into Lindt to get a chocolate – and I really like marzipan – I have to read the label to figure out which one I should be buying.  It constantly confuses me, and I don’t really like milk chocolate.

Now, at Lindt, are they really wasting much of my time?  No.  But what if you did this to your own employees?  What if you used a certain color scheme and then suddenly made it mean the opposite?  One thing I constantly read in articles from LEAN.org is that it’s these kinds of questions that get asked on Gemba walks.  Why are we inconsistent?  Why is this different or special?

Inconsistent colors may not seem like much, but it stands to confuse and slow down your business just ever so slightly.  It’s not the one time you do it, it’s all the little times that you do it in various places that slow everyone down.  It adds up.

Too much choice

I just finished watching an old (2006) TED Talk on the Paradox of Choice.  This fascinating 20 minute presentation is on how the modern western society ideal of choice = freedom and freedom = welfare may be dead wrong.  In fact, he argues, too much choice leads to greater levels of dissatisfaction through:

  • Increased regret and anticipation of regret
  • Opportunity costs (feeling bad about missing out on what the alternatives offer)
  • Escalation of expectations (forcing us to believe that anything less than perfect will dissatisfy)
  • Self-blame (once you have a choice, then if you make the wrong choice it’s your fault)

Dr. Schwartz (no relation to me), provides some interesting real life examples to what choice costs us as a society.  For example, for every 10 additional funds you can invest in your 401(k), participation drops about 2%.  Yet we know that unless you invest and save for retirement, you’re going to be in deep trouble later.

There’s an opportunity here in process design to attack some of these issues head on.  Standard processes means less choice for the individuals in the process.  But counter to our ingrained cultural belief in the individual and individual choice, less choice may in fact make our employees happier and those who have to interact with us happier as well.  Doing things in a standard way may not just be good for consistent results, but it may be good for morale as well.

The value of standards

Every day I learn something about process from watching a four year old – namely, my four year old.  She’s practically a pro with my computer when it comes to playing games, and she usually plays the games on NickJr’s site.  There’s basically three interactive things you can do on NickJr that my daughter is interested in – Games, Videos and Creating Art.  And so, throughout the entire site, no matter which kids show you are talking about they denote things in exactly the same way.

All games, everywhere on the site, are denoted by an orange color and the game controller.  It’s NEVER done differently (at least I haven’t seen anywhere).  All videos?  The green and white icon.  All art projects are denoted by the purple and white crayon.

My daughter can play on NickJr for a long time without needing a thing from me.  If she gets bored, she can choose a different game or watch videos or create something.  Navigating the site is never a problem.

Compare that to Scholastic.  My daughter can’t find a game on there to save her life.  Not a one.  Despite the familiar characters she clicks on, despite the words “games” being used in many places, she can’t find the games.  Heck, I have trouble finding the games.

It’s a simple matter of standards.  NickJr has them, Scholastic does not.  There’s always room to be creative, and NickJr saves that for the games themselves where they can provide verbal instructions as you go (just in time training, anyone?).  Scholastic is creative about the navigation, but it’s like a bad treasure hunt just to try and find something fun to do.

It’s a very tangible example of how standard ways of doing thing simplifies the NVA part (like finding the game) and maximizes the VA part of the experience for your customer (actually playing the game).