Why LEAN? Why not Agile?

Agile software development has been a hot topic for a while now.  There are many companies offering training for Scrum Masters and Agile software development plus a host of books on how you could implement an Agile software development methodology in your company.

In truth, many things about Agile software development are quite in-line with LEAN thinking, but there are some differences which I believe makes LEAN a preferable approach over adopting Agile methods.  Yet we often perceive these two things to be one and the same.  Yet I believe this to not be the case.

According to Boehm and Turner in Balancing Agility and Discipline:

A truly agile method must include all of the following attributes: iterative (several cycles), incremental (not deliver the entire product at once), self-organizing (teams determine the best way to handle work), and emergence (process, principles, work structures are recognized during the project rather than predetermined).  Anything less is a “lightened defined process.”

It is a couple of these considerations that appear out of balance with what LEAN is trying to accomplish, namely self-organizing teams and emergence – depending on how these are interpreted.  Iterative and Incremental may be as well.  In cases where the unit to be delivered is well understood, why subdivide the work arbitrarily?

Mr. Cho, Vice Chairman of Toyota said it best:

Brilliant process management is our strategy.  We get brilliant results from average people managing brilliant processes.  We observe that our competitors often get average (or worse) results from brilliant people managing broken processes.

Indeed, self-organizing does not necessarily mean making the best choices, just making their own choices.  Combine Mr. Cho’s observations with that of Barry Boehm and Richard Turner:

There are only so many Kent Becks in the world to lead the team.  All of the agile methods put a premium on having premium people… (p. 46)

And, as Boehm and Turner remind us:

A significant consideration here is the unavoidable statistic that 49.999 percent of the world’s software developers are below average (slightly more precisely, below median). (p. 47)

These grim realities about people remind us of an important point.  There is not an unlimited pool of great people to staff our Agile development projects.  Furthermore, even our best people have bad days, and individual variation can produce unacceptable results some times.  True LEAN thinking seeks out ways to eliminate these risks – to get consistently high performance in spaces where human fallibility can lead to customer dissatisfaction, financial loss, and even potentially loss of life. 

It seeks to do so without introducing new wastes in place of old ones eliminated – refactoring and constant testing (continuous integration) aren’t activities your customer would reasonably pay for.  We are doing these things because we haven’t considered how we might not.  Letting each team discover what’s “best” for them may compete with solid data to the contrary.  There’s no issue with the purest sense of emergence, because it could embody continuous improvement, but it also could be perceived to mean “as you see fit, regardless of it is the best known choice at the moment.”  In my opinion, it’s software development, not a lifestyle preference that we are pursuing.  Preference should take a back seat to data driven choices about which process to pursue.

The question raised is “can we achieve brilliant results in software without brilliant people?” and I truly believe the answer is a resounding “Yes!”  To combat long schedules, high costs and poor quality, we must work to eliminate activities which fail to add value.  Software development is fertile ground for waste with development policies often implemented as a death-by-a-thousand-cuts.  Each production incident generating new checks and balances to make sure that “thing” never happens again, while failing to create quality at the source of injection for more general problems.

We can step back and take another look at how we get work done.  We can use simple but effective techniques to keep bugs from ever being written in the first place and from escaping where they are injected.  And most importantly, we can shift our employees from thinking about the work to thinking about how we work so that everyone is contributing to the continuous improvement of our software development practices.

If we continue to compare Agile methods only to the worst-of-the-worst waterfall methodologies, then a quote from my mentor seems apt: “a meal of rice and beans is gourmet, compared to a dead carcass on the side of the road.”

Batches?

I’m attempting to translate LEAN thinking to be clearly applicable to software development.  One of the items I’ve been thinking on is batch-and-queue.

The essential LEAN concept is that creating large batches of anything is inherently wasteful. It creates inventory (which we argue does NOT have the value that traditional accounting assigns to it), is overproduction (stuff before anyone needs it), adds transportation (you have to move the inventory to storage), and adds defects (parts in inventory become obsolete and have to be reworked, which could also be considered overprocessing). The nice thing about manufacturing is, parts are easy to count. Pretty much, if you pick it up and it moves in a single piece, it’s a part. That is, anything that holds together as a unit can be a part in inventory, whether that part is a gear or a half-built radio. They’re easy to count.

Not so with software. What exactly is a unit of software? Some people say it’s a line of code. Some people argue it’s a function point. Some would say a story or requirement. And that’s mostly where we stop with software. So I asked my mentor about this. If a customer asks for you to create product X and it contains 50 features, we often decompose the features and try and deliver them piecemeal to the customer (Agile does this as does iterative development and multi-stream waterfall even). I can see why we might do this, because we can deliver some functionality to the customer and they get some benefit but not all the benefit they would realize by having all the parts they ask for delivered. From this vantage point, breaking down a set of features into smaller units and delivering them is like taking apart batch-and-queue and achieving single-piece-flow.

As I thought it over, ultimately I concluded that like a car, the customer didn’t ask you to deliver wheels, then a chassis, then an engine, then seats, then a steering wheel, etc. They asked you to deliver a car which had a set of features. In the same way, when the customer specifies new software to do something, it should be maintained in the form which provides value to the customer. Delivering it piecemeal isn’t single-piece-flow, it’s delivering only part of what the customer wants. Just because we haven’t figured out how to efficiently deliver everything the customer wants doesn’t make delivering it in dribs and drabs ok. This viewpoint of delivering iteratively what once was a package of requests is trying to reform the customer into being convinced that they are getting a good deal because you can’t do what they really want.

Making someone else do it isn’t lean

Being lean is about the elimination of non-value added work, but this consideration should not be made in a vacuum. At one time I was working with a team of testers who were telling me how they were getting leaner and not doing testing when it wasn’t necessary.

I asked them how that was possible. They said “we’ve asked the developer to decide whether or not testing is appropriate” and went on to explain the various elements of the decision tree the developer would have to go through. I said, “why don’t you just make that decision? You’re just as capable of determining it.”

“But then we’d have to do that work, and that wouldn’t be lean.”

Forcing work out of your team and on to someone else’s team is not leaning out the process. It is just pushing it around. Consider the auto industry; many of them claim to be leaner for having forced their suppliers to do most of the assembly work for them. Yes, it is true that the maker of your car didn’t do the work anymore, so they didn’t have to pay the labor or spend the time doing the assembly, but that didn’t mean the work went away. It just went somewhere else.

It’s a reminder that we have to look at a process holistically to truly lean it out. It isn’t a question of “do I have to do this thing?” but of “do any of us have to do this thing?” A waste is a waste no matter who is responsible for creating it. If you ask the first question, you do what the QA team did and just pushed the waste around. If you ask the second question instead, maybe you just decide you don’t need to do the thing at all.

I’ll leave you with a good counter-example from a software textbook I was reading about designing for the users instead of just solving the problem you could with the computer (sorry, I can’t remember the name of the text at the moment). Essentially what was happening was that the post office was delivering renewals, new memberships and cancellations all to this company. The company wanted to solve the labor problem with software, but ended up making it worse. One of the things that was problematic was that each type of request (renewal, new member, cancel) needed a different process flow, but that meant all incoming mail had to be opened and then sorted by the request type before it could be processed. That’s just pure waste (motion, transportation, or over-processing depending on how exactly it was done). The solution to this un-lean thing was to push the problem to the postal service. No, the postal service didn’t start opening and sorting the mail by the contents. Instead, they assigned each type of correspondence a different PO box, so when you stuffed your request into the pre-labeled envelope the post office did what it does best and automatically sorted it into the right bin.

Sure, the occasional piece was in the wrong place, but by leveraging the post office’s existing capability to examine and sort the mail (which they were going to do anyway) you eliminated the NVA step of having the re-sort it at the destination. A simple change, no more work for anyone involved. That’s leaner!

Making someone else do your work for you, that’s just a way to make them mad.

Just silly

I love cheese.  Mostly soft cheeses like brie, but pretty much anything with character.  So when we venture away from home to my in-laws, we take the opportunity to frequent one of the several cheese shops in town.  Better than buying a delicious cheese, we get to try a vast array of cheeses and then pick out three or four small samples to take home and enjoy with a glass of wine and some crackers.

As I was standing at one of these cheese counters the other day, I noted a gentleman to my right was ordering something from the prepared foods counter.  It wasn’t a cheesemonger that I usually go to, but variety is the spice of life, right?  Anyway, as this person is talking to the woman behind the counter he says “I’ll have some of this, and some of that” and so on.  Then, at last, he says “and a pound of sliced turkey.”

“Oh, I can’t help you with that, ” says the woman. “You’ll have to go over to the deli counter for that.”

“Ok, ” he replies, and takes two steps to his immediate left and turns ninety degrees counterclockwise.

Behind the same continuous counter, the SAME woman pops round the corner of the counter and says “ok, how can I help you?”

WHAT!?!  In the words of the villain in Zoolander “I feel like I’m taking crazy pills!”

Look… if you are capable of serving the person two steps to the left, then you are capable of serving them right where you are standing.  Regardless of what your established process might be, having to restart the transaction because someone isn’t in quite the right place is just silly.  Build processes that meet your customers needs.  This can’t be the first person who has ordered both prepared foods and stuff from a deli counter at the same time.

A process without decisions

Maybe this is a rule of good process design that I’d never heard of, because it seemed suddenly obvious to me when I thought about it today, but I’m going to write about it anyway.

I’m going to propose that a key feature of a good process is one that has no decision points in the process flow.  That’s right, no decision points.  A decision equals an opportunity to make the wrong decision and therefore the opportunity to waste time or jeopardize quality.

If a process is to be consistently good, then the last thing you want to do is give an individual (especially software developers) the opportunity to make the wrong decision.

I’ll give you an example – the creation of an analysis document.  Let’s say you have a process which allows the developer to choose either the long (and therefore rigorous) format or the short format for the document.  The decision you want the developer to make is “select the format appropriate for the scale of the change.”  Which one would you choose as a developer?  If you were well intentioned, you might really consider the question.  Still, it’s the kind of question you could screw up.  The decision is based on judgement and maybe your developer judges it incorrectly.  The developer chooses the long form when the short form would have been acceptable and wastes all kinds of time filling out an unecessarily complex document.  Or, the developer chooses the short form and thus misses the necessary rigor and inserts a major design defect into production.  Either way, the decision allows for the wrong choice to be made some of the time.

What about a not so well intentioned developer?  A lazy developer, perhaps.  Not that those kind of people exist at your company.  Oh no, all your employees are well intentioned all the time…

A lazy developer would always opt for the short form.  Maybe s/he doesn’t value analysis or maybe s/he does but thinks writing things down is silly.  Who knows, who cares.  The short form isn’t always appropriate.

Here’s what I’m proposing.  No short form, no long form, and certainly no choice about whether or not to fill out the form at all.  NO DECISION!  Just have one form that scales to meet the need automatically.  I know what you’re thinking.  How can that work, we have 246 sections to our standard design document and a developer needs to make a decision about whether each section is appropriate or not.  WRONG!  Why do you need 246 sections to your document?  Do you think each of those sections is critical to success?  Have you bothered to figure out which sections, if done well or not well actually affect performance?  Probably not.  You probably like lengthy forms with nice headers and sections and instructions in each section about how each section should be filled out.  You’ve ignored the idea of the critical few – and there are a critical few – that really affect process performance.  Everything else is noise.

It is possible, I have done it, to have a single document format, nay a single process, which you follow unwaveringly through new development, enhancements and even simple bug fixes.  The fewer decisions in the process, the better.

With that in mind, I propose this measurement of the goodness of your process.  To figure this out, you have to get down to the micro level.  If associates are making decisions about which sections of a document to fill in, that’s a decision you should count.

Decision density = 1 – (# of decisions in a process / # of steps in a process).

For example: 

  • analysis
  • analysis review
  • decision: analysis OK?
  • design
  • design review
  • decision: design OK?
  • code
  • code review
  • decision: code OK?
  • test
  • decision: test OK?

That would have a decision density of 4 decisions / 7 steps = 42.85%.  I’d say that decision density of less than 50% is a good start.  The lower the better.  There’s probably a better way to look at it, since you might want to weight micro decisions (those decisions made without consultation with a peer or group) as being worse than bigger decisions in the process (like the decisions made after an inspection step). 

The short story is: avoid decisions in the process.  More decisions means more of a hand-wavy methodology than process.