08.25.09

Software Development is like Manufacturing

Posted in Agile Development, Application Lifecycle Management (ALM) at 3:06 am by Tony Baer

For those of us on the outside looking in, it might be tempting to pigeonhole the agile software development community as a highly vocal but homogeneous minority that gained its voice through the Agile Manifesto. In actuality, the manifesto was simply a call to action to, in essence, value working software over working plans. The authors didn’t pretend to monopolize knowledge on the topic and purposely made their manifesto aspirational rather than prescriptive.

And so, not surprisingly, in a world of software development methodologies, there is no such thing as an agile methodology. There are many. There is Scrum, so named for the quick huddles in Rugby that call for fast decisions based on a basic assessment of conditions on the ground. Proponents call scrum responsive while critics assail it for lacking the big picture. In actuality, properly managed scrums do have master plans for direction and context. Other variants such as eXtreme Programming (XP) drive development through testing to ensure 100% test coverage. In actuality, agile methodology is a spectrum that can borrow practices from scrum, XP or other methodologies as long as it meets the goal of working software with minimal red tape.

More recently, debate has emerged over yet another refinement of agile – Lean Development, which borrows many of the total quality improvement and continuous waste reduction principles of lean manufacturing. Lean is dedicated to elimination of waste, but not at all costs (like Six Sigma). Instead, it is about continuous improvement in quality, which will lead to waste reduction. It aims to root out rework, which is a great waste of time and money, and a process that subtracts rather than adds value to the product. Lean techniques were at the crux of new enhancements to Danube’s agile planning offering which we discussed a few weeks back.

The best known proponent of lean development, Mary Poppendieck, makes the metaphor that software development is very much like a manufacturing process. Having spent serious time studying and writing about lean manufacturing, and having spent time with the people who implemented lean at the pioneering NUMMI auto plant nearly 20 years ago, we can very much relate to Poppendieck’s metaphors.

In essence, developing software is like making a durable good like a car, appliance, military transport, machine tool, or consumer electronics product. In essence you are building complex products that are expected to have a long service life, and which may require updates or repairs. We also contend that the metaphor means that software should also carry a warranty, where the manufacturer vouches for the product’s performance over a specific operating life.

But not surprisingly, there is hardly any consensus over lean in the agile community. We came across a well-considered dissent by Pillar Technology consultant Daryl Kulak that criticizes the very idea of comparing software development and manufacturing in the same sentence. Kulak maintains that unlike manufacturing, software is not comprised of the kinds of uniform discrete parts that are found in manufacturing. There’s a bit of irony there, as proponents of SOA, and before that component-based development, often called for just that. But Kulak is correct that most software is not uniform discrete parts, at least according to common methods of software construction and design. And besides, there is always some local environmental variable that will cause variations in how software is configured to run on different targets.

That’s a key part of Kulak’s argument. “We don’t discover its actual character until we’ve started building it.” He also argues that the core systems-oriented principles that are at the heart of lean could lean to “ ‘mechanical Agile,” where we view our teams as machines and our people as a kind of robot. This type of team can only follow orders and cannot innovate or adapt.”

Kulak and others have good reason for their cynicism, as it wasn’t that long ago that CFOs actually believed the promises of outsourcers that software factories would make software development predictable in time and cost. That had great appeal during the post 2000 recession. Obviously – in retrospect – software development is not so cut and dried – business needs are always changing (that’s why we have agile) and software deployment environments always have their unique idiosyncrasies.

(There has been a more recent refinement of the software factories idea that focuses, not so much on offshore factories. Instead the emphasis is more on patterns and industrializing aspects of software development, e.g., the portions of an app that could be exposed as a reusable service. But that doesn’t eliminate people from the equation, it amps up the architecture.)

But back to core issue, lean cynics miss the point. When we visited the NUMMI plant, we were struck by how lean actually spurred production line workers to get creative and develop solutions to problems that they carefully tracked on the plant floor. We saw all those hand-drawn Pareto charts on the walls, and in fact the reason why we were there was to study implementation of the first computerized production management system that would help people track quality trends more thoroughly. What we saw was hardly a mechanical process, but one that encouraged people to ask questions, then base their decisions on a combination of hard data and personal experience. It was actually quite a step up from traditional command and control production line management.

There’s no reason why this approach shouldn’t work in software. This doesn’t mean that lean approaches will work in all software projects, as conducting value stream analyses could get disruptive for projects, such as the ever evolving Facebook apps of the Obama 2008 campaign. But the same goes for agile – not all projects are cut out for agile methodologies.

So to software developers, stop thinking that you’re so unique. Obviously, lean manufacturing methods won’t apply verbatim to software development, but look at the ideas behind it. Stop getting hung up on minor details, like software isn’t a piece part.

Update: The NUMMI auto plant in Fremont, California has unfortunately become a casualty of the restructuring of GM and contraction in the US auto market. So it would be tempting to state that NUMMI was a case where the surgery was successful but the patient died. The lesson for software developers is that process only ensures that the product will be developed; it doesn’t ensure success of the product.

7 Comments »

  1. Justin Hunter said,

    August 25, 2009 at 4:25 pm

    Tony,

    Good thought-provoking post (and nice link to Daryl Kulak’s dissenting views).

    One area of Lean Manufacturing, Six Sigma, the Toyota Production System, and other manufacturing-related approaches that was both (a) highly important to all of the above in manufacturing settings, and (b) directly relevant to software development is Design of Experiments. W. Edwards Deming who was hugely influential in quality initiatives in manufacturing in Japan and the U.S. was an applied statistician. He Design of Experiments methods as a fundamental part of anchoring quality improvement initiatives and yet very few people who write about software development seem to be aware of the field.

    From wikipedia: “Design of experiments, or experimental design, (DoE) is the design of all information-gathering exercises where variation is present, whether under the full control of the experimenter or not. Often the experimenter is interested in the effect of some process or intervention (the “treatment”) on some objects (the “experimental units”)…”

    There are two clear instances in which Design of Experiments are currently having a significant impact in software development:

    Application Optimization applications, like Google’s Website Optimizer allows users to easily vary multiple aspects of web pages (images, descriptions, colors, fonts, colors, logos, etc.) and capture the results of user actions. A link to a recent YouTube Design of Experiment that increased signups by 15.7%. (The experiment involved 1,024 variations): http://bit.ly/10S1t0

    In addition, software test design tools, like the Hexawise test design tool my company created, enable dramatically more efficient software testing by automatically varying different elements of use cases that are tested in order to achieve an optimal coverage. An IEEE Computer article I contributed showing, on average, over 10 pilots, tester productivity (in defects found per tester hour) more than doubled as compared to the control groups which used their normal methods of test case selection: http://tinyurl.com/nhzgaf

    Unfortunately, Design of Experiments methods – one of the most powerful methods in Lean Manufacturing, Six Sigma, and the Toyota Production System – are not yet widely known in the software development industry. This is despite the fact that (a) they will consistently deliver measurable benefits and (b) sophisticated new tools make them more accessible than ever.

    - Justin Hunter
    Founder of Hexawise

  2. What Else Can Software Development and Testing Learn from Manufacturing? Don’t Forget Design of Experiments. « Hexawise Blog said,

    August 25, 2009 at 7:56 pm

    [...] 25, 2009 by Justin Hunter Tony Baer, from Ovum recently wrote a blog post titled: Software Development is like Manufacturing which included the following quotes:  “More recently, debate has emerged over yet another [...]

  3. bum's software gear said,

    August 25, 2009 at 10:39 pm

    I agree to that.. Software developers are not really unique. Ideas are more important when it comes to it.

  4. Jean Tabaka said,

    August 27, 2009 at 1:31 pm

    Tony,

    Thanks for raising this discussion and providing valuable (and correct) concern over TPS and Lean Manufacturing as a model for software development. I think the zealotry associated with these TPS models is one of the biggest roadblocks to its usefulness. There are some extremely valuable principles to extract. Actual practices are where organizations can inadvertently misinterpret principles and expect to have a manufacturing line of software product delivery.

    In the Michael Kennedy book, he moves away from lean manufacturing and instead concentrates on lean product development: how do we effectively develop new products, not how do we manufacture them? In this regard, there is useful translation of the principles into practices. For instance “Amplify Learning” can invite a practice of a set-based (aka concurrent) design.

    In set-based design, delay decisions on design, explore numerous prototypes, think about sets of design alternatives, and engage suppliers. All of this leads to products of superior design and products that are actually delivered faster. And faster delivery creates quicker feedback into design alternatives moving forward.

    A useful paper on this clarification is “The Second Toyota Paradox: How Delaying Decisions Can Make Better Cars Faster” by Liker, Ward, Cristiano, and Sobek. Yes, it is still about cars. But pay attention to the focus on product development, not production.

    I think the embrace of TPS in software forums also misses another key component of “Amplify Learning”. And this is the effective use of A3s coupled with value stream mapping in order to eliminate the waste of losing learning. John Shook’s book “Managing to Learn” brings us great insights on this fundamental practice for identifying problems, seeking root causes and coming up with countermeasures. Not A SOLUTION but potential countermeasures.

    Hope this gives some other perspectives.
    Thanks,
    Jean

  5. Daryl Kulak said,

    September 2, 2009 at 5:29 pm

    Tony,

    Great post and thanks for the link!

    I’d like to make a refinement on how you’ve interpreted my view of Lean and Agile. I did say that we don’t have nice discrete units in software, but what I meant was user stories and storypoints, more so than the software itself. We do our planning based on stories, not based on components, at least we should.

    It is truly different to plan with unit items (stories) that are a) of varying sizes and complexities and b) likely to blow up into ten or twenty times their size once we dig into them. Our difference is in the conceptual nature of software, which is not limited by the laws of physics. Nassim Taleb says we have two worlds in dealing with randomness: Mediocristan and Extremistan. Manufacturing problems belong in Mediocristan (non-scaling) and software problems belong in Extremistan (scaling).

    That’s what concerns me, that we are trying to look at our world as more predictable than it is. Taleb says that is what happened in the financial world, that people weren’t planning for the downsides in their “risk analysis,” and the financial world is big-time Extremistan.

    I’m not sure if I’m adding clarity or noise as I write this, so I’ll just stop here. Thanks again for the good discussion on this point, and for the link to the Pillar blog.

  6. Ryan Martens said,

    September 3, 2009 at 9:10 am

    Tony,

    I like to think of Software as a blend of managing the “design factory” and the “widget factory.” When many people hear manufacturing, they think of only the “widget factory.” Once lean is applied to the “widget factory” floor to drive a standard of work that yields higher quality and less waste, the focus of the lean organization almost always moves back to the product development or “design factory.”

    In software much of widget making is automated in the build, test and deploy process to a single push of the button without much of any assembly line. Agile certainly stresses early testing with lots of automation, but the entire agile process of working close to the customer is an attempt to manage the design process. I think it is very important to denote both sides of the problem and the solution. It keeps many people from only seeing one side of the value of agility.

    To me it is hard to see where Capital A – Agile does not apply. What software development effort would not benefit from a better customer feedback process, earlier testing, smaller batches of work, less hand-offs and shorter wait times in queues?

    Thanks for the article Tony – always a pleasure.

  7. Lean Manufacturing Techniques said,

    January 11, 2011 at 2:03 am

    I forget whose idea it was, but I once heard a much better explanation of software development in terms of manufacturing than what is usually presented (if you simply must use a manufacturing metaphor):

    In manufacturing, you spend a whole bunch of time up-front doing design work and end up with a blueprint for what you want to build. You (loosely speaking) feed that blueprint into a bunch of machines who churn out your end product The traditional software-as-manufacturing metaphor maps “creating blueprints” to “writing a spec”, “bunch of machines” to “developers”, and “end product” to the code that those developers produce.

Leave a Comment