“In theory there is no difference between practice and theory. In practice there is.” This week I saw one of those examples.
ROI $$ (agile) > ROI $ (waterfall)
A great compelling argument for those promoting agile over waterfall. The only problem is that it is naive and WRONG. The argument can only be had in retrospect. The organisation runs two projects, one agile and one waterfall, and compares the ROI on the two projects. I am not aware that anyone has run such an experiment. It is a religious belief without proof.
In reality, the comparison of the two approaches is done before the project starts. The comparison is made on the expectation of the ROI. Those performing the evaluation compare what they expect the ROI to be. The fact that they are considering waterfall indicates the project has a knowable outcome and is in the complicated domain of the Cynefin framework. The functionality is known. If at this point, it is known that the functionality could be reduced, then it would for both. However, when we look at the costs, waterfall only has one test and production release and agile has lots. Given the information at the time, Agile will cost more compared to the waterfall approach. The fact that they are considering waterfall also would tend to indicate a risk averse culture at play which means they wont consider risks or changes.
Expectation ( ROI (agile) ) < Expectation ( ROI (waterfall) )
Which means that if you encounter someone competent from Finance or a Finance background and start talking about ROI, you are going to sound like an idiot. Someone will use your lack of understanding to undermine your argument to their advantage.
Now lets consider Risk Adjusted ROI. Risk Adjusted ROI is exactly as it sounds. It adjusts the ROI for risk. There are many types of risk associated with an IT investment. Steve Freeman and I wrote an article in The Agile Times (published by the Agile Alliance in 2005) where we list the three categories of risk.
- Delivery risk – Failure to deliver the investment within the agreed constraints
- Business Case risk – Failure to deliver the identified value
- Damage to the Existing Business Model – The investment damages the exisitng business model
Agile can be used to manage all three categories of risk better, but only if it is applied properly. A Scrum or Kanban project that only releases once a year is not managing risk (and is not really a Scrum or Kanban project).
Now lets compare the expectation of an agile and a waterfall project. All of a sudden, all of the real options that exist in an agile project and make it less risky, can be considered in the valuation. i.e.
Expectation ( Risk Adjusted ROI (agile) ) > Expectation ( Risk Adjusted ROI (waterfall) )
And this is where theory and practice come in. In theory, the ROI argument is good enough. You read a few books and write down that ROI is the way to compare agile and waterfall. The practitioner tries this out and it does not work. They then have to dig deeper into the theory to come up with a something that does work. “As simple as possible but not simpler”. “A simple as possible” is the battle cry of the theorist. “but not simpler” is the retort of the practitioner. Finding “As simple as possible but not simpler” for your context can only be determined by praxis, the co-evolution of theory and practice.
For me, a humbling example of this on a personal level was when I was trying to come up with a definition of business value in 2003. I read “The Goal” by Eliyahu Goldratt and thought “A project creates value when it increases revenue or reduces costs”. I asked the business manager on the trading floor and he said. “Nope! Eighty percent of our investments are to protect revenue, either retain customers or build risk management systems.” – To quote Jabe… “Praxis makes perfect.”