Estimates are crack cocaine for managers and executives. Estimates are an emotional crutch that give managers an illusion of control over an inherently risky process. As well as being an expensive waste, estimates can drive unhealthy behaviour in teams. The problem with estimates is that there are different forms of estimate that are appropriate for different decisions and different levels of understanding, and unfortunately managers do not always use the appropriate estimates.
To begin with, an estimate is a “tea bag”, it is part of the solution needed to create an output for the manager. The output is required to satisfy a need for the manager. There are two outputs from an estimation process:
- How much will it cost?
- When will it finish?
In actual fact, you need to answer the second question before you answer the first.
The cost of an IT investment is the duration of the investment multiplied the day rate of the investment. Consider a team of eight developers who spend half of their time on support and half on new investments. In addition to the developers are a product owner, UX, managers, etc. amounting to a further sixteen head count on the project. If the investment is estimated to run for a further thirty days, then the cost will be 30 * 20 person days = 600 person days. i.e. All the people involved in the investment rather than just the four developers. That is why an understanding of Theory of Constraints is so important to managers, especially those who have been trained in cost optimisation which tends to create constraints and increase costs.
So the cost of the based on the duration of the investment, or rather, when will it finish. How do we determine that? There are two measures we can use:
- Count of stories
- Story Points
According to the work of Todd Little and Troy Magennis, there is little extra accuracy using story points over the count of stories. However there is a real advantage of the story count as it is available much earlier in the process than the story points. A reasonable estimate of the count of stories is normally available very early on in the process. Story point estimates are normally available about a sprint before the story is developed. That is, after the product owner has worked with the UX and Data dudes in the Product Owner Squad to create the prototypes, test them with users, write the gherkin format acceptance criteria and then slice the epic into stories that can be estimated by the team in the product refinement session. The alternative is to create a whole bunch of pure waste to create faux story point estimates much earlier than they should be available. Faux story point estimates that are not following the normal process and are thus not really story point estimates, and provide a false level of comfort.
Estimating when an investment will finish is best done by looking at a Cumulative Flow Diagram and projecting based on the rate stories are being created and completed. Consider the following Cumulative Flow Diagram. The black dotted line is “now” and the red dotted line is when the investment is due to finish. The red dotted line is calculated by projecting the rate at which stories will be created and finished. (Troy Magennis’s “Cool and Fabulous Spreadsheet” does something similar with numbers instead lines, and in a much more grown up way using statistical confidence intervals).
A week later, the manager looks at the CFD and sees that the estimated finish date has moved because the rate at which the stories have completed has increased. This may be because the amount of support work has reduced due to higher code quality, or because the team are improving and going faster. The manager can investigate and potentially identify changes to the process to improve the estimate of the number of stories.
A week later (in an alternate reality), the manager looks at the CFD and sees that the estimated finish date has moved because the number of stories has increased. This may be due to unforeseen functionality, or because a story was more complicated than expected and needed to be broken down. The manager can investigate and potentially identify changes to the process to improve the rate of delivery of stories.
A week later (in yet alternate reality), the manager looks at the CFD and sees that the estimated finish date has not moved because the number of stories has increased but that has been offset by an increase in the rate that stories are completed.
As well as understanding that there is a change in the estimated finish date, the manager can understand the source of the uncertainty that has created the change.
Now lets consider the same graphs drawn using story points. To begin with, we need to get some “experts” in room to estimate all of the stories, because we would not want to get the whole team to give an estimate as that costs too much. That means we have guess what the stories will be (before user testing and all that). Our “experts” will only be too pleased to provide a story point estimate, even though it is not a story point estimate. It is an expert “estimate” rather than one by the team doing the work, and it is not based on a “Definition of Ready” story, its a faux story point estimate. This means at some point during product refinement, the estimate will change to a real story point estimate and we will have some stories estimated using story points and some using faux story points. This mixture of story point methods is misleading and as a result, the manager will pressure the product owner to complete all the stories to a state of “definition of ready” so that they can get a reasonable estimate. There is no sensible* intervention that a manager can do to improve the quality of the estimates.
( * We could adopt a waterfall approach to creating stories. In order to improve the estimates, we could increase the risk and cost to the organisation substantially which isn’t really sensible ).
Even worse, there are now three reasons for a change in the estimated finish date, as well as the number of stories created and completed, we now need a third dimension for the change in story point estimates. In other words, to understand what has happened, the manager needs to study two graphs** and the difference between them. What was obvious on the CFD, has now become complicated to understand and needs an “expert” consultant to explain. This lack of transparency leads the manager to fail the Agile Risk Framework Audit that requires them to know and understand what is going on in their investment.
( ** I tried to think how you would represent in a useful way this but gave up as it was too complicated, and it was silly anyway)
One day, the manager has a revelation! Instead of using faux story point stuff to estimate the finish date (and hence cost), lets encourage the product owners to write stories of a broadly similar size. From now on, the largest story allowed will be an “eight” (or a “five” for teams with one week sprints). No more stupidly massive “thirteen” or “twenty” stories. That will mean I can get my estimates for free without any stupid and wasteful “faux story point” estimation sessions.
Todd Little Key Papers