What are our machines?

When you set foot inside a factory, picking out the machines from the people is pretty easy.  The machines are large, made of metal, typically quite heavy and fail basic tests of being living things – they don’t breathe or eat.  It’s easy to pick out the people as well – they’re the ones getting maimed by the machines if used improperly.

Joking aside, when you peer into a software development shop, separating the machines from the people isn’t so easy.  Sure, you might be quick to point to the computers, but that really isn’t where the work of software development is done.  Software development is largely a translation process – converting someone’s idea into functioning software.  The actual typing on keyboards, compiling and running is a very small component of the process.  Indeed, the machine that we should consider when applying LEAN to software development is the human brain.

Though mysterious in many more ways than it is not, the human brain is a form of machine and has (some) predictable results for certain inputs.  It suggests that the careful design and continuous improvement of software development practices are feasible.  But it also suggests that the tools we have today put too much energy into too small a portion of the real work that’s occurring.  Too many tools focus on making the act of coding faster, while it’s the interpretation of requirements and the design of algorithms where the real risk is.

Few advancements have made a dent in this.  Despite 3rd and fourth generation languages, the rise of object oriented code, SOA and more, there’s been no silver bullet to the challenges that face us.  I’m convinced we’ve been solving the wrong part of the problem all along – writing tools to simplify a relatively insignificant part of the process.

Process vs Purpose

I ran into this neat presentation about what motivates people over at LSS Academy.  The content may come as familiar, but the presentation style is worth watching.  It’s all done in dry erase markers.

In it, the speaker talks about the fallacy that money motivates people, which it doesn’t.  But he does talk about the things which do motivate people – a sense of purpose, autonomy, and mastery.  As I was watching this I began to wonder how process fits into all of this.  Could process destroy these things?  I believe the answer is no. 

Process is not in competition with having a purpose, being able to act with autonomy or mastering a skill. 

Purpose:  Purpose isn’t defined by how you do things.  It is defined by what your goal is and ripping off your customers is never a goal of lean.  Dr. Deming, among others I’m sure, believed that good process had not only an obligation to meet the company’s needs, but also to be socially responsible.  He recognized that although we have set some boundaries for our value stream, it necessarily operates within a larger frame of reference that should not be ignored – the well-being of all people.

Autonomy: I guess it depends on what your definition of autonomy is.  If it means “free to do whatever I want whenever I want” then yes, process competes with that.  However, if it means having the freedom to make decisions about how the code is structured or what’s the best algorithm, process only provides guide rails.  But more likely, it simply means you don’t want to be micro-managed and process is all about that – simplifying the work so less oversight is required.  Even more, the nature of your autonomy with process is simply changed – from freedom to build whatever code you want to freedom to use your insight to change the process for the better and then replicate that improvement to everyone.

Mastery: A good process provides a framework from within which you are shielded from the most common mistakes, but it doesn’t prevent you from mastering a skill.  Process is not about dumbing down the work, it’s about removing the obstacles which allow you to think about how the work is done rather than the work directly.  Indeed, it could lead to an even higher level of mastery.

When I watch compelling presentations about human behavior, such as the one I linked to, for a brief moment it makes me think “hey, all this process stuff is just plain wrong.”  Perhaps I’ve just got confirmation bias, but I don’t see process and purpose as enemies, but partners in continuous improvement.

Implement to own

Here’s an idea:  if your developers are throwing poor quality code over the wall, why not have them on the other side to catch it?  In theory, if the problem was motivation  this would work.  If your developers don’t feel responsible for good quality, one way to make them is to have them have to suffer the consequences of their own choices – by making them fix the bugs they create.

Unfortunately, I think this idea falls flat for a lot of reasons:

  1. Your customers don’t benefit from this.  Though the same person who created the bug now has to fix it, you still allowed the bug to be created.  Just because a knowledgeable person fixes it quickly doesn’t mean you didn’t impact your customer.
  2. It replaces process with single points of failure.  Now, only the individual who created the buggy code is the one who has to live with it.  That person becomes a hero – solving bugs that they created just in the nick of time!  It’s like the firefighter who starts the blaze just so he can put it out.
  3. People don’t show up to work to do a bad job.  Sure, the person is now responsible for everything they missed, but we’re assuming they did a poor job out of sheer laziness instead of out of inability to do a better job.  I think this is a pretty dim view of our developers.  It’s also a cop out for not defining a better process.
  4. It encourages full-employment rather than good quality.  Believe it or not, developers I have talked to consider bugs to be job security.  We tend to think developers want to write new code, but many are more than happy to just be coding, so it doesn’t necessarily motivate the right behavior anyway.

At a glance, I see why someone might go down this route to try to solve a quality issue, but I don’t think the theory holds water.

Developer “flow.” Better for all?

Developers are commonly believed (and I agree) to enter a state of focused work that is commonly called “flow” (not to be confused with Flow in LEAN).  Flow, for developers, means that they work at a higher productivity level, with fewer defects, because they can work uninterrupted.

Intuitively, this makes sense.  It’s much easier to work on almost anything if nobody is bugging you.  This need for flow has encouraged some developers to post signs outside their workspace – “do not disturb!”  I too, when analyzing a data set, have fallen into a flow where I seem to be able to get a lot done.  At some point in time, I get stuck, usually when I have an unanswered question about some content in the data or missing data that I need input for.

When I hit these points, I reach out to whatever contacts I have to try and get a speedy answer.  My flow has been interrupted, and to restore it I need to get the missing piece of information so I can get back to work.  In fact, a great deal of my job depends on these little moments of discovering what I don’t know and then going to find the answer.

When I used to program, I’d often find this to be the case as well.  Systems I worked on were quite large, so it wasn’t uncommon to run into a section of code I had never visited before.  Still, I usually knew who did know about that section of the code so I’d reach out to that person to fill in some knowledge gaps for me.

My anecdotal experience about both entering flow and it getting interrupted made me ponder something.  Is allowing a developer to post a “do not disturb” sign to maintain flow really the best thing to do?  Sure, we all want to eliminate unnecessary distractions, but what about distractions which could help someone else get on with their day?  What if the distraction isn’t a phone call from your spouse asking you to pick up milk on the way home (something that could wait) and instead is a question that if unanswered will either block or at least greatly delay a coworker?

Being a data person, my reaction was to simulate it, so I built a simple model.  Here’s how it works.  A team can have any number of developers (I used 5, 10, 15, 20 and 25 for my graphic).  For each minute of the day a developer may randomly have a moment of self-interrupted flow – a realization that they need some piece of missing information from another developer.  The model then allows for two possibilities.  One model allows the developer to go ahead and interrupt another random developer to get the answer.  If the developer can interrupt another, both developers pay a random amount of lost productivity (5 to 30 minutes I made it) PLUS another penalty of 15 minutes (a number I’ve heard quoted before) to regain flow.  The alternate model does NOT allow for the developer to interrupt another developer.  Instead, that developer becomes blocked until a certain time of day when all developers stop (namely lunch or the end of the day). 

At the end of the day, we simply count up the total number of minutes that all developers were in a “flow” state.  You can see the results in this g raphic.  The red square dots represent the model where interrupting another developer is allowed.  The black circles represent the model where interruptions must be held until pre-determined times of day.

Notice, regardless of the size of the team, when the rate of interruption per minute is relatively low (about a 1% to 5% chance per minute that you’ll be interrupted), it is better in this model to interrupt another developer with your question.  Essentially, the amount of time wasted by one developer waiting to get the information they need hurts more than the cost of the interruption to both parties.  Even though two resources aren’t getting anything done for some amount of time, it’s better that they both can get back to work and make progress.

Beyond that break even point of 5% interruptions, not breaking flow is somewhat better, but not much.  In those higher rates of needing help, regardless of which way you go nobody makes much progress.  Allowing some developers to have flow in those cases entirely wastes another developer’s time, but the interruptions would become so frequent that nobody can do work either.

Clearly, this model is far from perfect, but it is instructive.  One should not allow developers to have “flow” at all costs.  While it is good for the individual developer, it is worse for the team as a whole.  And while all interruptions don’t come from developers (managers have questions too), I think it’s fair to say that if not getting the question answered would block your progress for longer than the time to regain flow then it a good choice to interrupt.  The black and white of “don’t interrupt” is easy to remember, but the outcome may not be as desirable as we imagine it to be.

Not one of the critical few

Ingrained in the Six Sigma school of thought is the critical few – the 80/20 rule. It is an important rule. In practice, there are a handful of things which often allow you to make big leaps from an incapable process to a capable one. There are more subtle characteristics of the process which can be refined to continually improve the performance, but this isn’t step change, it is refinement. And then there’s a class of things that just don’t matter.

Recently, while attempting to facilitate a process design effort, I spent a lot of time thinking about the things that don’t matter. That may have been because that’s all anyone spent their time talking about. And as facilitators, we were enablers of this dragging on. Having been instructed to drive to a single standard process and toolset, we discussed every little one-off thing that people wanted to allow for in the process to see if we could squeeze them out. A day’s worth of 25 people’s time to design a process spent talking about the equivalent of the carpet color.

We wanted perfect compliance to the standard, and that meant a standard which was not necessarily all-inclusive (because some of these one-off requests were truly ridiculous by any standard). This is where I believe we got off track with process work. Process design is about controlling the critical few things which will make the difference in process performance.

But that is not what we were discussing. We were discussing nuances, oddball cases, odd uses of the process, and data elements that some teams wanted and others didn’t. We talked about the 1% and largely ignored the 99%. We talked about things that weren’t going to make the difference, whether they were one way or another.

To begin with, we didn’t know what was going to make the difference. We hadn’t studied the existing processes to understand what made them work – what really mattered and what didn’t. This created unnecessary room for debate because we were unable to bring adequate materials to the table to help the team work through their differences. We had little to no information on what mattered and what didn’t.

Instead of define-measure-analyze-improve-control we just went right into improve. And there we got bogged down discussing every little quirk, because we didn’t know what else we ought to be talking about. Or more importantly, what we shouldn’t be talking about.

Instead of a conversation that was “do we really need that? How many of our teams use that process step?” we could have instead said “sure, it doesn’t matter to me if you allow for that.” And we’d be saying that not because we didn’t care but because we actually knew what did matter. Everything else, the little things that we debated with the teams could have instead been bargaining chips that we could dole out in heaps and have given up basically nothing that really mattered. We could have had a strong position, not because we won all the arguments but because we knew which battles were worth fighting and which were worth conceding.

Had we known what things were not one of the critical few things, we could have appeared very agreeable and allowed the teams as much “leeway” in the process as they claimed they needed. All along we’d be giving up nothing. Nothing that really mattered anyway.

It’s a reminder why a thorough measurement and analysis of a process is important. It isn’t just discovering what the current state is (measurement), but it also understanding why it works (analysis). And from there, narrowing down the bits of process that really do matter, and just letting the rest go. Some things just don’t matter.

A single point of failure

This is a thought provoking entry from Curious Cat Management on the single point of failure.  How many days do we allow an individual to hold the company, or some part of it, hostage to his/her exclusive knowledge?  How often do you allow the performance of your team to be dictated by the individuals?

And what do you do when those same indivduals have an off day, or an off week, or a personal problem, or even the sniffles…

How wonderful is it when your key employee comes through for you in fighting the hundred new fires that came up today?  Did you ever stop to think that the hundred fires might have been set ablaze by the same individual?

True, there’s nothing exciting about adherence to a process.  There’s nothing exciting or remarkable about any individual when a well designed process eliminates the art from regular high performance.  What is left to differentiate someone if they are not the lynch pin in your system – your single point of failure.

Instead of rewarding the individual for a stellar performance, how about you reward the individual for bringing up the entire team’s performance.  It reminds me of the activities of World War II, when the leaders looked to the individuals in the trenches for ingenious solutions to the things that stood in their way.  While individually recognized for their contributions to the war effort, the benefit was not in the individual working harder or faster, but in making it possible for everyone around him or her to be more productive.

Don’t reward people for standing out above their peers, reward them for not standing out above their peers because they’ve brought their peers up to their level.

Avoiding Non-compliance

If you’ve built yourself a good process and proved that it works, you want people to follow the process.  You know that it makes a difference and it is really frustrating when people continue to cause problems by not doing “what’s best for them.”  Like a couple of prior posts, this one is about why people don’t do what you want them to do.

 I’ve already been down the People Are Selfish road so I’m not going to rehash it.  Let’s take that for granted.  Today I wanted to give you another option for getting people to do what you know to be the right thing.

Here’s the idea, and it’s super-simple.  Make it easier to do the right thing than the wrong thing.  “What!?!?” you say, “I’m not paying for that kind of advice!”  Of course, you don’t pay me anything, so…

When process compliance is a problem, you have to look at what it takes to get the process right.  Going back to that people are selfish (and lazy) thing, if there’s a shortcut to be taken then people will take it.  If taking that shortcut hurts the performance of the process, then you must make it hurt more to go down that path than to not. 

Take, for example, my old team at my last job.  No software developer likes writing documentation (or thinking through a problem) and they want to get right down to coding.  I already knew that this was going to be an issue.  The process I had designed worked because it relied on peer review to assure that everything was getting thought through.  That also meant that being sloppy was not an option for my employees.

Why were people sloppy?  Well, they figured if they didn’t get the design right in the first place that they’d just go back and fix the code later and that fixing the code later would be easier than figuring out the right thing now.  Of course, that’s not true.  Finding a bug later in the process results in effort that is orders of magnitude worse than finding and fixing it early on.

So, what could I do to make people do the right thing?  I couldn’t make people write good documentation.  However, because I had set up controlled access to the source control I could make them write more documentation.  In fact, this is exactly what I did. 

If a person wrote a defect and it made it to QA, they would have to write a complete set of documentation in order to fix the bug.  That meant even for the most simple one-line code defect I required an analysis document, an analysis review by a peer, a design document, a design document review by a peer, the code, a code review document by a peer, and a test results document.  It is really annoying to fill out all those documents for a single line code change.  It is so annoying, in fact, that it drives down the defect rate.  The cost to the developer for screwing up is now significantly greater than the cost to get it right in the first place.

Developers, or anyone for that matter, do not see the costs they incur on someone else for their screw-up.  If my laziness results in the need for a full-time support person, it doesn’t hurt me directly at all.  But, if my laziness creates work for me, then I’m less inclined to be lazy.  I hate doing stuff over.  I hate filling out documentation.  I hate being punished for something that I had the capability to avoid.

So, while what I really wanted was for people to do a good job at the documentation there was no way I could enforce it.  But, since doing a bad job at it could result in a significant punishment of even more documentation, I had an indirect means of getting what I wanted in the first place. 

And that is making it easier to do the right thing rather than the wrong thing.