The Agile versus Waterfall argument divides software developers. There are those that believe iterative development to be superior way of developing software, and those that believe up-front planning to be a superior way of developing software. A couple of weeks ago I had a short but passionate argument with one of my co-workers. They are super super smart, and we each stand on opposite sides of the up-front planning divide. My colleague’s argument is that you need to plan in order to execute fast and efficiently at scale. That fast execution at scale requires considerable up-front planning.

This puzzled me greatly. Real Options shows that up-front planning is inferior as an approach because you commit early and destroys lots of options. Lean tells us that planning is a form of Muda or waste. This is particularly interesting because if people smarter than me think this, then real options could be a flawed concept and I break the model… and learn as a result. After much discussion and thought, I think I may understand the problem. The insight comes from thinking about the problem using the Cynefin model for social complexity developed by Dave Snowden.

The Cynefin model consists of four quadrants (snigger – In joke) of “Simple”, “Complicated”, “Complex”, and “Choatic” and a fifth quadrant “Unordered” for when you do not know which of the other quadrants you are in. A “Simple” system is over constrained and has cause/effect behaviour. “Complicated” is the realm of the expert. “Chaos” means there are not enough constraints. “Complex” means there are just enough constraints to create an emergent behaviour which can be nudged and tweaked into a system with desirable behviour. “Complicated” means an expert can reduce the problem to component parts with predictable behaviour. “Complex” means there are feedback loops, agents and limits that create an emergent behaviour that cannot necessarily be predicted. Essentially, the difference between the two can be two can be distilled to “A complex system interacts with its context and evolves as a result, and a complicated system does not interact with its context or evolve.”

If you know in advance what is needed, you are in a complicated system. You can specify and plan in detail to create an optimal delivery at scale (A long lead time and short duration). Hence the space shuttle. Complicated systems struggle to react to changes in context which can cause analysis paralysis. In product development, complicated systems are managed and dominated by experts. The axioms of this paradigm are untested assumptions. This is the realm of early commitment and the slaying of valuable options. As a cautionary note, these systems have the potential to suffer a catastrophic failure if they choose to ignore the context changes.

If you are in a situation where the context is dynamic and an expert cannot predict in advance, you are in a complex system. You cannot specify and plan in detail because the context changes before you can deliver. Development in a complex system consists of several safe to fail experiments followed by fast developments to enhance beneficial developments and dampen non-beneficial developments. In product development, decisions are made by studying data and the impacts of safe to fail experiments and small iterative investments. Products and their context co-evolve. The axioms of this paradigm are hypotheses which are to be challenged and tested. This is the realm where real options are an optimal strategy (Check out Dave’s paper from 1857).

So the answer is quite simple. If your product is in a complicated context, use experts and up-front planning. If you are in a complex context, create options and learn to respond faster than your competition.

A final note. If you think you are complicated context, and you are operating in a market with competition… I’m sorry, you are wrong. You are actually in a complex context.

Despite my best efforts, I now find the Cynefin framework to be a useful tool to understand* and operate** the context within which I work. If you want to find out more about Cynefin, sign up for one of Dave’s course at www.cognitive-edge.com. Oh, and refer to the quadrants as quadrants. He love’s that.

*Phenomenology
**Ontology

For nearly two years I have been helping my client do agile rather than do agile myself. I have been a coach helping teams adopt Scrum, Theory of Constraints, Specification by Example and other stuff. Although its my first attempt at “helping others do it” rather than “doing it”, I have been approaching management as a coach for almost a decade. Thankfully I am working with other experienced coaches who have been helping me learn the ropes.

For the past few months, the agile coaching team have been preparing for the day when the client will achieve sustainable agile. Sustainable agile means they will be self sufficient rather than needing to rely on external agile coaches. Thankfully, our team leader, Tony Grout, has some experience in this space and we are building “self-service” “first-aid” material for the organisation. But it’s not enough. We need the organisation to take on the yoke of coaching.

For the past few weeks I have been working with dev and test managers to prepare the way for a “Management Skills Matrix”. In technical parlance, preparing the management fitness landscape to make it less hostile to the “skills matrix” meme. (Note, So far all the managers love the idea). The management skills matrix would be a public skills matrix posted on the wall in one of the coffee areas…

There is one big difference between the management matrix and team matrix. At skill level 3, the manager feels confident enough to coach a team in the skill. Now our target is to get at least three managers in every location who can coach each skill. As we achieve that goal, I can then retire from coaching for each skill until the organisation is self-sufficient.

This was the point that I realised that I was an organisational dysfunction.

Some of the more experienced coaches had suggested I should work for the team doing what was right, rather than work for management. It felt right because management did not have a deep understanding of agile but I had a stronger feeling that I should be aligned with management who represented the goals of the organisation. The management skills matrix helped me realise that I should not work with the team at all. Instead I should work coaching the leadership of the organisation so that THE LEADERS COULD COACH THE TEAMS. That way, there would be no misalignment. Management would know why they were doing each agile practice. There would be no disconnect between the teams and management. By training the teams, I am perpetuating a disconnect between the teams and their management… I am perpetuating an organisational dysfunction.

But training every manager in every skill is impossible. Some managers just do not want to know about the intricate details of Agile. Thats fine, training every manager in every skill is the wrong goal. The goals is to ensure that each skill can be coached by at least 3 managers. (On a process note, I’m using 3 as an illustration. The final answer might be 1,2,3,4,5  or who knows depending on context).

I should treat coaching skills in the same way I manage staff liquidity. When a coaching request comes in I should ask the managers who wants to take it on and get them to do pair coaching with me. Currently the way I work as a coach perpetuates me as the key man dependency.

It answers that age old question? Who should go Agile first? The team or the leadership?

GIVEN that management want Agile

WHEN they hire a coach

THEN the coach should start with management.

So now I have to change the way I work so that I’m no longer a dysfunction. For those of you who know me, you know how hard that will be. 😉

Recently I’ve heard a couple of people say that if you increase WIP, you increase the number of options in the system. This is a fundamental misunderstanding of the relationship between options and WIP.

Increasing WIP reduces the number of options available from the system.

Consider the following two projects, both of which have five developers who can handle two tasks at a time.

1. In the first project, each developer takes on two tasks each. There are ten tasks in progress. They commit to all ten tasks at time zero.

2. In the second project, all five developers swarm on one task at a time. They commit to each task, one at a time, at times 0, 1, 2, 3, 4,…

In the first project, they commit up front. In the second project, they defer commitments. In the second project they do not commit early, they wait until they have finished the previous task before committing to the next one.

Of the two projects, the one with lower WIP has more options and hence more value. Stated another way, two projects that deliver the same tasks, the one with lower WIP has more value due to the options it contains that the other does not.

I was chatting to my good friend Jamie O’Shaunessey about start ups and enterprises. I explained the 20 – 20 – 20 rule* that the american army use as a strategy for any engagement. The rules are…

1. Within twenty hours, they will have special forces on site to provide recon and intel.

2. Within twenty days, they will have the navy and marines on site to control strategic positions.

3. Within twenty weeks, they will have had the whole army on site if necessary.

The key point is what are you optimising for? Initially, you want to optimise for speed. Hang the cost, hang the risk, just get eyes on site. Then, you want to optimise for risk. Rebalance the response network so it can be ready for the next crisis. Free up special forces and replace them with marines. Then cost (and risk) become an issue and you supplement the marines with regular army so that you free up some of the marines for the next action.

This maps very closely to the optimal strategy for a successful start-up. In its early days, the startup should be optimised for speed. The startup needs to get to market as fast as possible, prove the existence of the market and secure the customers. Who cares about tomorrow when you are trying to stay alive today. As a result, key man dependencies are not a concern and it likely that a skills matrix like this will exist with several significant key man dependencies.

At this stage, the startup may be acquired or receive significant investment, both of which facilitate a rapid growth phase. The teenage years of a startup so to speak. This is the riskiest time for the startup. Its customers will be looking for signs of deterioration of the product if it is acquired by a new owner. At this time, the startup should be focused on risk. They should be ensuring the stability of the product (pay down technical debt) before the product grows / expands, and they should be paying down key man dependencies (liquidity debt). It is at this point it needs to bring in the marines. It needs to hire those developers with enough experience to shave the Yak. Those developers who can quickly pay down the technical debt, establish a solid test automation/continuous integration framework, establish a mature self responsible development culture and acquire the knowledge to address the liquidity debt. The established startup developers have the most options so they should be assigned no work other than to be instantly available to answer the questions of the marines. (This frees the original team members up to work on developing new markets.) After this period, the skills matrix should look something like the following.

Note that there are no key man dependencies.

However Marines ain’t cheap. Anyone looking for developers with five plus years of experience in Agile development skills (Ok, Ok, I mean eXtreme Programming, So sue me) will know that they are not cheap, and they are often contractors/consultants (especially in London where I’m based. Other parts of the world may be different but I’m guessing there is a still a premium to be paid). Once your startup has reached this state, it can look to reduce the costs per head. They can start to hire cheaper developers who do not have extensive Agile experience. They can hire graduates and bring them up to speed. The team can then use the skills matrix to decide if and when to release the expensive marines.

Sadly, the common pattern I’ve observed is that startups switch from optimisation for speed to optimisation for cost, missing out optimisation for risk. How can a company spot whether it is has missed the optimise for risk step? The have the following behaviours:

1. They have dragon slayers/hero developers. A simple skills matrix will reveal these key man dependencies.
2. They do not have a consistent culture focused on quality and risk, with self disciplined developers and product owners who appreciate the value of quality and risk.
3. It will take longer and longer between product releases. They will have infrequent major releases rather than the continuous delivery of small releases (i.e. They will be waterfall rather than Agile as the management will not trust the quality of the application.). Every one will hold their breath when they do a release rather than treat it as a business as usual activity.
4. There will be (at least) two cultures. One, a start up culture focused on speed that is in constant conflict with new joiners who feel frustrated at how slowly they come up to speed.
5. Others that I cannot think of now as I have a cold.

So what does a company do when it discovers that it has missed out the stage of optimising for risk?

Simple. Call in the Marines. Go hire enough of those experience Agile developers to make a difference to the culture and pay down the liquidity debt and technical debt. It ain’t cheap, but then again, neither is failure.

* Like all good thought leaders I never let facts get in the way of a good story. If you want to see the real thing, check out the right stuff. I cannot find the link and it would great if someone let me know what it is.