A few years ago I was lucky enough to work with Steve Freeman. Steve is the Gordon Ramsey of Software Development. If you have a badly formed code base, expect to hear about it. Steve worked with the graduate, Mark, on the team to “refactor”* some code. They created tests that clarified the intent of the code. Most impressive to me was that they reduced the size of the code base by 80%. Imagine you are editing a 200 page book with random duplication. Steve and Mark’s work reduced that to 40 pages and removed the unnecessary duplication. Everything was where it should be, and you only had to find something once rather than find a random number of instances. (Note: Some teams print off the code they delete and put it in a “book of dead code”) This was vital work as we were about to make significant changes to the code. The refactoring they did was pure risk management. It meant the chances of missing a necessary change were reduced as previously the code had been cut, paste and tweaked to death. Delivery risk ( endless looping in the test phase ) was reduced, and the risk of missing something in testing that damaged the production business was also reduced.

In Commitment we use the analogy of a kitchen to describe technical debt. A clean kitchen is one where things are in the place a professional chef (e.g. Gordon Ramsey) would expect them to be and it is stocked and ready to go. A kitchen where things are in the wrong place or the utensils and surfaces have not been cleaned and put away has technical debt as the chef needs to clean up and prepare the kitchen before they can start. The clean up and preparation is normally the first thing Gordon does when he visits a kitchen on one of his shows.

Each team will have a definition for good code. A definition of where things should be. Anything that does not conform to that standard is technical debt. Working on code that contains technical debt introduces significantly more risk than code without technical debt. The risks introduced include:

1. Delivery Risk. Technical Debt tends to blow up in the test phase where it cannot be managed, only responded to. The project Steve and I worked on was plagued by a team that did not have automated tests. The same bug resurfaced several times due to a lack of regression testing.

2. Business Case Risk. If the code contains technical debt, there is more likelihood that it will behave in an unpredictable manner and the changes will not deliver the value expected.

3. Risk to existing Model. If the code contains technical debt, there is more likelihood that it will fail in production in a way that damages the existing production system.

As such, from a risk management perspective it is often prudent to pay down technical debt before making changes. This is where Steve’s suggestion that the team regard the code base as a portfolio of sold call options is really useful. For each component of the system, the team can identify how much technical debt exists. The measure for the technical debt should be how much effort it will take to pay it down. This effort should consider how complex the debt is, and the staff liquidity matrix, namely how many team members can work on the component.

The product owner can then create a “tornado map” where they map out the high level road map for the next six months to a year. The further out, the more uncertainty there is. Using the tornado map, the product owner and team can work out which technical debt should be paid down now and which can be left. In some cases the technical debt may take a long time to fix, or contain too much complexity to estimate effectively. In those cases, it may be necessary to pay down the debt early. In other words, risky option positions should be closed down early.

Technical debt is much to do with the skills and perception of the team. One team may consider their code base to be clean whereas in reality it contains a lot of risk. As such, it is necessary to hire someone experienced at identifying dead code.

Like Gordon Ramsey, good developers don’t create so much technical debt in the first place, so it is cheaper to hire good developers in the first place so you do not need to pay for dead code. So my advice to you is to hire Steve Freeman, the Gordon Ramsey of code bases, for his sixth sense about code, someone who can say “I see dead code”.

* It was not a refactoring as there were no automated tests. Part of the process was putting the tests in place.

As suggested by Steve Smith, here is an example of what a Tornado Map would look like.

Although I apply this thinking, I have not created a map. This is the what the one in my head looks like that I use for this purpose.

The backlog is down the side with the Epics grouped into those expected to be done in the next month, quarter and year.

Across the top are the components of the system. I would expect them to be similar to those on the skills matrix for the team. The technical debt is displayed below each component. It is expressed in team weeks (i.e. SWAGs).

From the example, you can see the most immediate problems are the loader and especially the build system.

After the build and loader are sorted, the next problem is the blue component which will almost certainly be hit this quarter. The red component has less technical debt and might not even need to be changed.

Even though the yellow component contains a lot of technical debt, it is not going to be touched for over a quarter and is less important than the build system.

I’m a huge fan of Dave Snowden’s work, especially as it comes to Scaling Agile. In particular I love the “Recipe Follower and Chefs” metaphor. It captures the difference between what I call theorists and practitioners, although I should really refer to practitioners as praxis-ists. To me, the theorists are the ones who make up the recipes rather than the ones who follow them.

The problem with recipes is that they ignore context. A practitioner operates within a context and so cannot ignore context. The recipe for making a cup of tea is simple… unless you are on a beach, or in the Himalayas where the boiling point for water is much lower.

I love Scrum for its simplicity. Its an approach that can be explained in half an hour yet takes a life time to fully master. Scrum has three roles “Product owner”, “Team” and “Scrum Master”. The product owner makes all the decisions about the product, they prioritise (order) the backlog. Simple! In the context of a single team, Scrum is great.

The problem with Scrum is when you have multiple teams. Or rather, Scrum is not designed for a multi-team development and assumes a single team environment. There is no problem with Scrum, the problem is that reality does not fit into the context that Scrum was designed for.

So lets look at this problem in more detail. Imagine an organisation with two teams (TeamA & TeamB), each of which has a backlog owned by a product owner. TeamA has an MVP called MVPA which requires work from both TeamA and TeamB. TeamB has an MVP called MVPB which also requires work from both TeamA and TeamB. The product owners prioritise their own MVP in their respective backlog… TeamA will has MVPA at the top and MVPB at the bottom. TeamB has MVPB at the top and MVPA at the bottom. This means that MVPA and MVPB will not ship until both are complete.This might seem a crazy situation but it is exactly what happens if a mechanism is not put in place to manage it. Product Owners will prioritise the requirement of the last senior executive to shout at them to get their work prioritised. Furthermore, this is an abdication of responsibility by the executives of the organisation who should be providing direction and focus. In effect the executives have devolved responsibility for the companies direction to the product owners of the Scrum Teams. Although the product owners may understand their product, they do not necessarily have an organisation wide perspective.

What we need is some mechanism to unpick these dependencies so that either MVPA or MVPB is at the top of both backlogs. What are the solutions?

1. The immediate response it to organise the teams into proper Scrum Teams or “Feature Teams”. Each team should be able to deliver a full MVP without any dependency on other teams. This is the ideal situation and should be the first approach. There are contexts where this is not possible. There are organisations and products that are so large that specialisation is needed and a team of 6 to 9 developers cannot cover the entire space. This is the context of Scaling Agile. If you are following a recipe, this is where you will struggle as the Scrum recipe does not work. Its like trying to follow a recipe to make an omelette without eggs.

2. In this situation, we decided to use the “theory of constraints” to find the constraints and create an organisational backlog. We were no longer following a recipe, instead we were having to experiment like Chef’s do. The product owner recipe actually became a problem for us. The theorists ( Recipe Maker ) told us that the product owner should always own the ordering of the backlog. The context meant that to deliver fast, the product owner needed to prioritise their backlog according to a higher level “organisational backlog”. Scaling the Product Owner role creates a direct conflict with the principles of Scrum. If we follow the Scrum Principles we have no control over delivery at the organisational level, however if we create an “organisational backlog”, we violate the Scrum Principles.

For an organisation trying to scale the Product Owner role, this presents a very real challenge. We had executives who saw the need for an “organisational backlog” but also the need for product owner autonomy. In some ways this was a good thing as the executives were thinking hard about not undermining the product owner. On the other hand, it hampered our ability to deliver.

Note that this does not even consider how the “executive backlog” is prioritised / ordered. Whether the prioritisation is top-down, bottom-up, collaborative or directive, democratic or autocratic. Simply the need for an “organisational backlog” causes the Product Owner role recipe to “fail”.

If you are Scaling Agile, you need to hire a (team of) Chefs. You need to hire people with years of experience applying Agile in the role of product management, architecture and IT (risk) management. This team will be coaches to help you handle the situations where the recipes fail. You will also need the people churning out the recipe books and training the teams in team level Agile practices. Your teams will need to follow the recipes but you will also need Chefs for when the context causes the recipes to fail.

A final word of warning, beware of Nigella Lawson. Nigella simply copies other people’s recipes (She has a vast library of Cookery books) but does not invent recipes herself. She is famous but she is not a Chef as should not be listened to when the recipe fails. Instead, hire the Chef from your local gastro pub where you like the food.

I’m conscious that I am not an expert in Scrum. If I have misunderstood something, please let me know and I will update the post.

At XP Day a few weeks ago I gave a lightening talk introducing “Roleism” and I wrote this up after being prompted by a couple of people present.

“Meet the Parents” is a funny film with one underlying “ism”. Its a film about “Roleism”, a prejudice that one person is better than another because of the role they do. In the film Greg Focker is considered inferior because he is a nurse rather than a doctor. Furthermore, it is assumed he is a nurse because he could not get the grades to be a doctor. The underlying message is “Why would someone choose to take a lower status role in society just to help others?”

As a Business Analyst and Project Manager I have often experienced roleism in the Agile Community. Friends will introduce me “He a business analyst, but unlike all the rest, he’s a good one”. What happens if we replace the term “Business Analyst” with “Indian”, “Woman” or “Gay”. How does that make you feel? The assumption is that all Business Analysts are useless but I’m alright. Once you get away from the bright lights of the Agile Conferences you discover that many Agile Teams work with business analysts or architects or some other “untouchable”. The reality is that prejudice on the basis of role is no way near as serious as the other forms of prejudice and I do not mean to imply it is in any way. Then again, when we look at the correlation of roles with gender and ethnicity, we might start to feel less comfortable about “Roleism”. “Roleism” might sound like a joke, however it is the basis of the caste system in India which is perpetuated by poverty and access to education.

I’ve experienced “Roleism” in the Agile Movement for many years. I’ve seen a video of an Agile Thought Leader describing business analysis practices as evil. The implication being that anyone using those tools is also evil. The thing that caused me the most concern about that incident was that no one challenged them. No one said “we should not demonise or de-humanise sub-groups within the community”. It was tolerated.

I’ve worked in a company where “Roleism” is institutionalised with leaders saying testers are not as good as developers. Perhaps if the testers had had he same educational advantages as developers….. but that starts to make us feel uncomfortable.

Why is “Roleism” important now?

Many Agile practices came from studying the best developers in the community. XP was based on the practices of Kent Beck, Ward Cunningham, Martin Fowler, Ron Jeffries, Chet Hendrickson et al. These were built upon by Tim Mackinnon, Steve Freeman, Nat Pryce, Janet Gregory, Lisa Crispin, Lisa Hendrickson, Liz Keogh and many others. Practitioners, not theorist.

There is a huge demand for scaling Agile to the Enterprise. This means that some of the roles we do not need at the team level start to come back into the picture when we are working across teams at the organisational level. All of a sudden we need Business Analysts and Architects and Project (Risk) Managers and and and…. The problem we have is that “Roleism” means the good ones stay away from the Agile community. We do not have the Kents and Wards and Rons whose practices we can model. And into that vacuum steps the trainers and the theorists. In absense of practices developed by the best practitioners, the theorists and the trainers make them up. Their clients expect them, so as experts they are expected to provide them. And we know what happens when the theorists start calling the shots. In a few years we will find ourselves being controlled by people who do not know what they are doing because they have been “helped” by people who do not know what they are doing.

Imagine a movie called “Meet the Project”. It stars a veteran project manager who does not trust a business analyst. The business analyst claims they could have been a developer but really wanted to help projects instead. The business analyst is the butt of the jokes, an outsider, until they are eventually cast out of the project. Only then does the project manager discover that the business analyst once contributed to open source. Who is the butt of the joke now?

I am increasing concerned about the projects that people are calling Agile. I encountered an “XP” project that took eighteen months to deliver the first release, and nine months for the second. Even worse, the project was the flagship Agile project for the organisation. I encountered a project that has yet to deliver a single release after two years that claims to be Agile because they are using Scrum. I think anyone involved in Agile would agree with me that these projects are NOT agile.

Whilst defining an Agile test would result in endless discussion and debate, I think we can all agree on a test to demonstrate that a project is NOT Agile.

Here goes. As a minimum, an Agile project…

• Delivers demonstrable value each iteration.
• Sustainably delivers a high quality product.
• Releases in short iterations.

All three are needed because any two can be gamed. e.g. You can deliver value quickly but not high quality or sustainably.

A project is NOT Agile if it does not do all three. Doing all three does not mean that the project is Agile.

Some more detail on what these mean…

Delivers demonstrable value each iteration

Each iteration delivers value to the end customer or the business. A stakeholder (not necessarily the product owner) can see the value delivered. Delivering software is not enough, especially if it does not deliver value to anyone.

Sustainably delivers a high quality product

High quality means high quality code (with no bugs) that performs according to user needs and has no user experience bugs. Sustainable means the teams are not sprinting and collapsing, but rather have a sustainable pace that they can maintain indefinitely.

Releases in short iterations

The project delivers value in short iterations. Each iteration is a month or less. I choose one month because that was considered best practice over a decade ago and should not be considered a difficult goal by any stretch of the imagination. In reality, many organisations deliver value several times a day.

The key measure for the iterations is the lead time rather than the time between releases.

So there you go. A test to show things are NOT agile, rather a test to show things are Agile. Note that there is no mention of Scrum, XP, Safe or Kanban, or any of the Agile practices. It is the qualities of the project that determine whether it is Agile, not the practices ( Hat tip to Alistair Cockburn ).

This thinking is not mine, it is the result of many hours of discussion, especially with Marina Oliviera, Tony Grout, Alan Hawrylyshen, Paul Hammond and many others.

This morning I saw a discussion on twitter about “Estimates being wrong”. This struck me as really odd. I’m a huge fan of Todd Little and Troy Magennis on the subject of estimation. They have taught me that the relationship between actual and estimate follows a log-normal ( or rather Weibel ) distribution.

When we make an estimate, it occurs at a time t based on an information set ( I0…It) with a filtration funtion F(t). There is a huge amount of uncertainty involved in the estimate. It is obviously going to be wrong. A better question is, is the estimate useful?

We estimate in order to make decisions. Do we want to do this thing? Do we want to do something else?

In Capacity Planning we use estimates to help us work out the approximate demand on a team during the quarter. We can use this information to identify which teams are constrained and this helps with creating an organisational backlog. Our process asks the Product Owner for the estimate, and NOT the engineering team. This means the engineering team cannot be held accountable for the estimate if it turns out to be WRONG. It also means the team do not try to get it RIGHT. The process allows the Product Owner to ask the engineering team if they choose, it just does not require it. The executive in charge of the process was given Todd Little’s paper to read so they understand the nature of the estimates. The goal is not that they are right, it is that they are consistent in approach. The estimates are good enough to be useful to help identify constraints and form the backlog. No one cares if they are right or wrong.

Something about this made me stop and think about Cynefin. Much of my work recently has been assisted by understanding whether the person I’m dealing with operates in a “Complicated’ world where outcomes are knowable or whether they believe the work is unknowable/emergent in a “Complex” world. I’ve noted that certain words are useful cultural markers to help you spot which. Right and Wrong indicate certainty which implies a belief in expertise. Hypothesis, Useful, Better or Worse indicate an appreciation of the uncertainty inherent in the process. People are not computers, so it is common to find someone struggling to find the appropriate words to use.

So if you hear someone referring to estimates as right or wrong, then you know they are thinking about expertise. If they focus on the usefulness of the estimate and the context, they are thinking about complexity.

Am I right or Am I wrong? And is this useful?