A little while back a friend complained about “5 Whys” in Feature Injection. “People don’t like it when you ask “Why?” over and over” they said. “Well, no one actually asks Why five times. There are many ways to ask why without actually asking “Why?” I replied. Here is a random ten questions off the top of my head. Please add your own in the comments.

If you want to find out more about Feature Injection ( or Story Mapping, User Stories or Impact Mapping), come and join Christian Hassa, David Evans, Gojko Adzic and me at the Product Owner Survival Camp on 21st Jan in London Town.

Random Ten Alternatives:

1. What do you want me to do with this stuff you’ve put in the system?
2. Now that we’ve stored it, what do you want to do with it?
3. When would we use this?
4. How would we use this information?
5. What reports / screens would you display this on?
6. What would the report / screen look like?
7. How would you use the report?
8. Who would use the report / screen?
9. When would it be useful to them?
10. Errr?
11. What?
12. Eh?
13. Huh?
14. And?
15. Then?
16. Really Blackadder?
17. Draw it please?
18. This is valuable how?
19. I don’t think you are telling me everything?
20. Go on!
21. Why?

Cost of Delay is often mentioned as a possible solution to the difficult problem of deciding the order in which you want to invest in two or more software development investments. ( For an introduction to cost of delay, check out Don Reinertson’s talk and Joshua Arnold’s blog and experience report presentations. Both are well worth the time invested ). I would like to highlight two capabilities your organisation may need before it can effectively use Cost of Delay. These are:

The ability to estimate value.
The ability to convert value to a common “currency”.

In effect, Cost of Delay assumes a level of corporate maturity.

The ability to estimate value

Todd Little wrote an excellent paper that shows the difference between actual effort and estimated effort follows a log-normal distribution ( Troy Magennis would argue that its a Weibull distribution ). IT professionals are pretty good at estimating things it would seem. The same is not true when it comes to estimating value or return. I once worked on a project where the business case estimated the annual profit for an investment would be €100M. In the first year, the investment gave a revenue of only €500K. The estimate of return was out by several orders of magnitude.

Lean Startup and a experimental metric based approach to predicting the improvement in a metric is a much more accurate approach but still is not that accurate.

Rather than a single value, estimates should take the form of a range with a confidence interval. For example, the return will be $1,000 to $1,000,000 with a 90% confidence interval. (Check out Doug Hubbard’s presentation )

So which is the better investment, the one that delivers -$50,000 to $2,000,000 with a 90% confidence interval, or one that delivers $1,000 to $1,000,000 with a 80% confidence interval. My maths is not good enough to compare these two investments. Now let’s consider that this is the cost of delay for the two investments. Which do I choose.

It is quite likely that the two or more potential investments are from different people or groups. How do we ensure that they have adopted a consistent estimation process. One possible solution is to engage the finance department to act of the coaches for putting together business cases. The finance department should not tell people how to build business cases, instead they should coach people and share useful practices. ( I almost wrote share “best practices” but could not face the strangling sound from Dave Snowden). The finance department can ensure a level playing field and help raise the game for the people writing business cases.

The ability to convert value to a common “currency”

Not all value is equal!

With the rise of business metrics, it is possible and desirable for organisations to focus on a particular part of the “funnel” that does not directly lead to a “dollar” value. An investment may be to increase the number of installed customers, or improve the usage or stickiness of customers.

Even with the same metric, there may be more value to customers on a particular platform, or in a geographical or demographic grouping. What are more valuable? Customers in the developed world or in developing markets? Teenagers or Pensioners?

In order to compare an investment to increase usage with teenagers in the USA versus revenue from Baby Boomers in Europe versus New customers in Brazil, the organisation needs an exchange rate to a single currency (The Corporate Peso or the “Corpeso”). This exchange rate needs to be set by the executives of the organisation taking into account market opportunities and the organisations vision. The exchange rate becomes a strategic tool to steer the software development investments. Some organisations may to choose a simpler approach and focus on a small handful of metrics instead.

Simplified Cost of Delay

It is possible to introduce as simplified version of cost of delay focusing on the two or three basic shapes. The delayed return is calculated by multiplying the rate of loss by the delay for a delayed product intro. A step cost for things like fines, and a total loss for things like missing the Christmas season.

There is a danger with introducing the simplified version in that people devalue cost of delay, especially as they are already doing the simple shapes. You risk that cost of delay is seen as adding unnecessary complexity to something simple. This may innoculate the organisation against using the full cost of delay in the future.

Cost of Delay is a great concept which will work well in certain contexts. If you try to implement it in more complex contexts, you need to consider the organisational maturity needed to support it.

This is one of those posts where I’m looking for feedback. Ideally practitioners who’ve made this work but also thought leaders who’ve put serious thought into the matter.

For the past year or so, I’ve been thinking about metrics. When measuring the effectiveness of your development investment process, I’ve come to the conclusion that you need three outcome metrics… A “quality metric, a “value” metric and a “time to value” metric. You need the three metrics because any two can be gamed. For example, you can maintain high quality and a low time to value by releasing software that delivers no value.

Quality metrics are fairly straight forward… number of bugs released is a fairly standard metric. Value metrics are fairly easy to define but sometimes harder to measure… e.g. profit, revenue, number of customers. The tricky metric is the “time to value”.

For years I’ve been chanting the “Cycle Time” mantra from the Lean world. When it comes to looking at metrics in the real world of software development, “Cycle Time” is a great inspiring concept but it ain’t much use helping a company performing an Agile Transformation. The problem is knowing when to call the start on the investment. When you trawl through the historical data of real investments, it’s difficult to compare them. The “time to value” metric is really about risk management. A development system with a short “Time to Value” will be less risky than one with a longer one. Initial thoughts are to consider the start of any work on an investment to the time it is released. This penalises investments where a small amount is invested up-front to surface and address risk (the value of business analysis). As a result, the approach is often to ignore the analysis in the “time to value” metric and focus on the point that the “team” commits to delivering value. i.e. The point it is pulled into a sprint. Ignoring the analysis means that over-investment can occur to reduce risk. In effect, “time to value” metrics are messy in the real world. To illustrate this, consider the following three investments:

The three investments have the same start date and release date, however (1) is clearly better than (2) which is clearly better than (3). In the real world, it is even harder to compare investments as the investments are spread across several teams with different rates of investment. This is a solved problem in finance where investors need to compare different bonds and match assets to liabilities of differing size and frequency. Duration was one of the simplist and earliest ways of comparing the risk of bonds. The approach converts a number of cash flows into a single cash flow at certain point in time. This is quite simple. Imagine a seesaw in a children’s playground. All of the cash flows in the investment are placed at the time they are made on one side of the seesaw, and all of them at a single point on the other side. They are placed such that the seesaw balances.

Duration ( tD ) = Sum of ( Cash Flow * Time to release ) / Sum of Cash Flows.

( Note: In bond maths, the Cash Flow would be the present value of the cash flow.).

Using the examples above, gives the following results…

1. (28*1 + 1*2 + 1*3) / 30 = 33 / 30 = 1.1
2. (10*1 * 10*2 + 10*3) / 30 = 60 / 30 = 2
3. (1*1 + 1*2 + 28*3) / 30 = 87 / 30 = 2.9

The lower the value for the duration of the investment, the lower the risk. The thing I like about this metric is that it drives the behaviour we want. Smart investments where analysis is used to reduce investment risk before a short time from commitment to investment.

The time for the investment should be the point that the team makes a commitment to the investment. In Scrum, this would be the start of the sprint.

So I’d like to hear others thoughts on this.

Discount Factors and the time value of money.

Whenever you discuss investments, thought leaders cannot help trying to add complexity to the situation. My advice is ignore the time value of money as its effect is not that significant. Adding time value of money makes the calculation much less intuitive to people using the numbers.

If time value of money is used, the inverse ( exp[ + rt ] ) of the discount factor ( exp – rt] ) should be used. This is because you are calculating the value of the investment at the time of release in the future whereas in finance you are calculating the current value of a future cash flow.