One of the key risk management controls for an Agile Risk Management Framework is to limit the amount and time between investment and return.

A development organisation needs to deliver investments where the Lead Time and Weighted Lead Time are within agreed thresholds. Limiting the lead time also limits the financial value of inventory in the development organisation that is not delivering value to customers. An organisation may limit both the lead time (or weighted lead time) and/or the financial value of inventory (See CFD below).

This leads to a new way of looking at the RAG status:

• RED – The investment is over the lead time (weighted lead time) limit.
• RED (Trending) – The investment is expected to breach the lead time (weighted lead time) limit.
• AMBER (Trending) – The investment might* breach the lead time (weighted lead time) limit.
• GREEN (Trending) – The investment is not expected to breach the lead time (weighted lead time) limit.

Organisations in transition often struggle to make lead time targets. It is possible that the organisation might allow longer lead times provided software is delivered into a pseudo production environment, and that there is a commitment from executives across the entire value stream to achieve the target lead time within an agreed time period.

* As indicated by a tool such as Troy Magennis’s Monte Carlo Simulation.

The IT Investment Risk Management Framework for Agile development would be very different to that for Traditional approach. By its very nature, Traditional operates in the Obvious and Complicated domains of the Cynefin framework. Traditional assumes the correct answer which means it is not well suited to manage risks in the Complex and Chaos domains. The entire approach to risk management between Agile and Traditional is different. Traditional funds a list of features (Scope) to be delivered whereas Agile iterates towards a business value goal.

A traditional approach mandates a list of deliverables that need to be created to satisfy the SDLC. Agile mandates a list of risks that need to be managed in the context of the investment. This difference in approach means that two projects doing different things could both be satisfying the Agile Risk Management framework. Even stranger, there may be two projects following the exact same process but one of them satisfies the risk management framework BUT THE OTHER DOES NOT! For this reason, the policing service within the governance framework is critical and very different to traditional SDLC policing.

Traditional SDLCs are typically practice and deliverable based. There is a simple yes/no checkbox approach to policing the IT investment. e.g. Do you have a functional specification? And a project plan? And a test plan? Do you have 100% automated test coverage?

Agile Risk Management Frameworks are much more principle based. The policing service needs to be able to identify where there is risks because the approach is not quite right. e.g. On one team, the scrum master does all of the updates to Jira. On another team, everyone on the project updates Jira. Both projects are transparent but one might have a key man dependency. The policing service might ask to see a Skills Matrix or ask other team members if they know how to use Jira. Simply put, its not black and white.

The key role of the policing service is to challenge claims of “transparency”. To ensure that the perceived transparency is genuine. Having shed some transparency on risks, the policing service can suggest resources for the team to learn to improve. As they have a responsibility to the organisation, the policing service are responsible for the making the risk visible and recording it so that it does not become forgotten. Most teams do not intend to obscure information, often the problem is a lack of understanding and experience.

The diagnostic skills required of the policing service that allow them to identify risks that should be managed are similar to those of an Agile Coach. Even though the skills overlap, the Agile Coaches should never be used to implement the Policing Service. They can coach the policing service and help them acquire analysis skills. They should not act as the policing service, otherwise it will destroy trust between the team and the coach.

It is the responsibility of the policing service to ensure transparency on all risks. As such they should not report to a delivery organisation where there might be a conflict of interest between their responsibility to their management and the organisation. e.g. Consider private security guards versus independent police force.

It is the responsibility of the policing service to identify where the IT Investment Risk Management Framework is flawed. Either too constrained which limits the ability of the organisation to deliver, or not not constrained enough so that the organisation is exposed to risks that are not managed.

The IT Investment Risk Management Framework is the set of constraints that development organisations need to operate within in order for the associated risks to be managed appropriately.

The executives in an organisation have a responsibility to rhe investors when managing the risk surrounding the delivery of Business Value.

Executives responsibility for Business Value Metrics

The key metric that an executive is responsible for is:

• The percentage of the IT Investment Portfolio that can demonstrate whether an investment generated a return.

There are two sub-metrics beneath this metric for the executive.

• The percentage of IT Investments that are linked to a metric. i.e. Has the product owner stated the metric that will improve as a result of an investment (e.g. Story, Epic, Initiative).
• The percentage of metrics that can be displayed. i.e. Is the metric available in a format that the improvement in the metric can be demonstrated. Note that the metric might be manually captured.

In effect the executive should be able to observe a graph for each metric that is similar to the one below:

as well as a summary of investments that looks like the following:

In the event that a metric is not available to be displayed, the executive should be committed to delivery of that metric within an appropriate time frame.

Risks Being Managed

These key metrics demonstrate that the executive is managing key business value risks. The risks managed are as follows:

The percentage of the IT Investment Portfolio that can demonstrate whether an investment generated a return. – This metric ensures that the executive has transparency into whether a return is being delivered by their portfolio of investments. The exact return delivered by a particular investment may be difficult to assess however the impact of the portfolio investment is transparent. It is not necessary to specify that the executive checks the return regularly as the metrics graph provide transparency that is not enhanced by taking minutes of metric review meetings. However, it is to the executives benefit to be on top of their metrics by reviewing them regularly so that they can intervene in a timely manner. A failure to intervene would be evidence that the executive is not performing their role properly.

The percentage of IT Investments that are linked to a metric. – This metric ensures that everyone in the executives organisation is focused on delivering value rather than functionality. It provides transparency on those that cannot prove they intend to deliver value (no metric assigned). This transparency also allows the executive to take a portfolio view and rebalance the portfolio if appropriate.

The percentage of metrics that can be displayed. – This metric indicates where a return can and cannot be demonstrated. If the return cannot be demonstrated, there is a massive risk that an investment might deliver no return ir even worse destroy value.

The Business Value Metrics Framework

All metrics should be part of an organisation wide standardised metrics framework. This has two benefits:

• The standard framework makes it easier for product owners to identify the value they are delivering because they do not have to work it out for themselves.
• The standard framework makes it easier to compare the value of disparate investments.

The business value metrics framework should be “open sourced” within the organisation with a product owner facilitating the addition and modification of metrics, and the decisions regarding the additions and modifications being made by the Business Value Metrics Steering Committee. The Business Value Metrics Steering Committee should be appointed by the board as the definition of business value in an organisation is key to its success.

A business value metrics framework is vital when an organisation relies on third party consultants to write business cases as there is a substantial conflict of interest. For example, there is a tendency to suggest attributes of a product as business value (e.g. Lead time) rather than business value (Conversion Rate). Product attributes are easy to deliver compared to those that depend on the needs of the customer and the market.

The Business Value Metrics Framework can be constructed using “Break the Model”. Identify an example (Tea Bag), Reflect if it is in-scope, Create an Example (Tea Bag -> Cup of Tea -> User Need -> Business Value) and then Model (Add it to the Hierarchy). The product owner performs this process based on the examples that are presented to them as not fitting in the model.

The Business Value Metrics Framework will probably map to your organisations Business Model. e.g. Freemium = Get users, Increase Activity, Get Revenue. Traditional = Get Revenue, Increase Activity, Prevent Churn.

Risk Adjusted Return on Capital is a good starting point.

• Risk
  • Investment Lead Time (Weighted Lead Time)
    • Key Man Dependencies
  • Quality
    • ITIL measures
    • Failure Demand
  • Employee Happiness
    • Turnover
• Revenue
  • ARPU (Average Revenue Per User)
  • Activity (Average Visits per Month)
  • Number of Customers
    • New Customers
      • Conversion Rate
      • Customers into the Funnel
    • Churn
• Costs
  • ACPU (Average Cost Per User)
    • People Costs
    • Overheads

Disruptive Innovation versus Efficiency Innovation

The executive should ensure that the portfolio has the appropriate levels of Disruptive, Sustaining and Efficiency Innovation. Otherwise the organisation might discover its market being disrupted. Check out Clayton Christensen’s lecture for more details.

* The concept of the Business Value Cascade was created by Mark Gillett, Senior Vice President at Microsoft, responsible for Skype & Lync products. The first analysis and implementation was performed by Keith Beatie, now a partner at McKinsey.

An IT Investment Risk Framework is a set of constraints that an organisation imposes on its development organisation to ensure that IT Investments are delivered in a way that manages the risk involved. It should create failure containers so that any failure does not propagate across the organisation like the failure at Knight Capital which caused the organisation to fail.

The boundaries imposed by the framework should be negotiable rather than fixed, otherwise the framework may fail catastrophically. (Hat tip to Dave Snowden’s Cynefin Framework.)

The Risk Framework should NOT specify how the IT Investment is made, simply the way that IT Investment risk is managed. The Risk Framework specifies a number of Commitments placed on the development organisation when they accept funding for an investment.

By comparison SAFE and LESS are frameworks that seek to optimise the delivery of value to the organisation. They provide a number of enabling constraints in the form of principles. The SAFE and LESS frameworks can both be deployed within a IT Investment Risk Management Framework providing they satisfy the its constraints. In effect, they are an Option that the development company may adopt.

In summary an IT Investment Risk Management Framework is a commitment placed on a development organisation whereas the SAFE and LESS frameworks are options available to the aid development.

I love working with smart people. They make it easy and fun to solve nasty problems. Tony Grout is one such person. He is a really deep thinker on the subject of theory of constraints. One of the problems faced by many large organisations is that they have a prescriptive software development life cycle or SDLC for short. The organisation insists that all IT Projects follow the SDLC. The SDLC is normally very binary in nature. If your project is of type XYZ, then you need to do ABC. This poses a problem for Agile development which doesn’t follow the same philosophy. Rather than one size fits all, Agile development adapt to the risk profile of the context they are operating in. Often without realising it, most Agile development follows Alistair Cockburn’s Crystal methodology.

At Lean Kanban London Day, Tony Grout and I presented our view on the Agile Organisation. (Hat tip to SAFE).

A key part of that organisation for any Large IT Organisations is the Governance Function. This consists of four key parts (A metaphor of speed restrictions for drivers is provided in brackets to aid understanding):

1. An IT investment risk management framework. (The law stating 30 mph in residential areas. 20 mph is areas with people at heightened risk such as schools and hospitals)
2. A policing* function to ensure that all investments operate within the framework. (Speed cameras and traffic police)
3. A governance function to ensure the risk management framework evolves appropriately according to context. This function also acts to help interpret the framework in a particular context. (Courts and law making bodies)
4. The responsibilities of ordinary citizens. (Driving tests).

Separate to the governance function is the capabilities that allow development teams to deliver IT investments. The problem with many SDLCs is that they combine the risk management framework with the capabilities needed to deliver them. The IT investment risk management framework should be the smallest set of constraints that the organisation imposes on the development and change organisation to ensure that the risks associated with the IT investment are managed appropriately. The constraints should be helpful and instructive rather than arbitrary. The constraints should give comfort to investors that their investment will be protected against the three categories** of risk, namely:

1. Delivery Risk – Will the functionality be delivered.
2. Business Case Risk – The IT Investment delivering the stated return.
3. Existing Business Model Risk – Will the IT Investment damage the existing business model.

Tony and I categorised the risk management framework into the following attributes of an Agile development. If an IT Investment does not satisfy these criteria, it should be considered NOT Agile and thus managed under the existing SDLC. These categories are:

We even use them as part of our training.

The policing function is particularly important for an Agile IT Investment Risk Management Framework as context is so important to the approach, and it is easy to game the rules. The policing function should work closely with the organisation’s Agile Coaches to help development teams learn to operate within the rules. This policing function should never be performed by Agile Coaches as it would destroy the relationship between the coaches and the development teams.

The IT Investment Risk Framework specifies the impact of each risk category on each part of the organisation.

Our next posts will expand on the framework, starting with the detail for “Delivering Value” for the “Exec”.

* By policing function, I am referring to one like an idealised 1950s Britain where the police are considered public servants (Dixon of Dock Green) rather than some countries where they are considered a domestic army to suppress the people.

** These three risks were first articulated by Steve Freeman and myself in an article published in the Agile Alliance’s Agile Times in 2005(Date ?).

If you ask most people what the constraint is in an organisation, they will tell you that it is the budget. You can tell this from the amount of energy that companies put into managing and controlling budgets and funding. Much of the upper management spend all of their time and effort controlling budgets.

However budget is not really the constraint. Try this thought experiment. Imagine you have infinite budget. Imagine you have all the funding that you require. Imagine that your company has been bought by an infinitely rich benefactor who says “money is no object”, and means it. What is the constraint now?

Queues form in front of constraints when demand exceeds capacity. This lack of capacity at constraints results in work items being delayed. Constraints reduce the queues in front of downstream processes, and those downstream processes potentially run out of work to do. These downstream processes hungry for work will generate work to keep them busy. The net result is that increasing budget results in more work in progress. Increasing budget does not necessarily increase output from the system… much to the frustration of those funding everything.

The first constraint is the capacity of each teams to satisfy demand.

It takes time to shift the capacity of an organisation. Small corrections in capacity involving the movement of staff or work and typically take a couple of months. Increasing the capacity of the organisation which involves hiring new staff can take a few months. Capacity is only increased when the person joining a team is competent. A new person occupying a role is initially a drain on capacity.

The second constraint is the capacity of individuals within the team.

The capacity of a team is a function of the constraints within the team. That is, the capacity of the team is limited by the capacity constraints of the individuals in the team. If the demand for a certain skill exceeds the capacity of the individuals in the team with that skill, then a queue will effectively form in front of the skill capacity and those working behind the constraint will be starved of work and take on lower value work. Even in cross functional feature teams, it takes time for a new member of the team to come up to speed.

Managing the Capacity of Teams in an Organisation

Tony Grout and I developed Demand Mapping at Skype with a large group of collaborators including Lisa Long, Ram Rao, Marina Oliveiro and John Horton. At the same time Dan was developing Delivery Mapping with his clients. This became apparent when Dan North bought me in to one of his clients to introduce it there.

Demand Mapping starts with the understanding that the constraint in organisation is the teams (or scare resource such as servers / server space) rather than the budget. The goal is to create a backlog that optimises the value that can be generated from the (currently) fixed capacity of teams. A secondary goal is to identify constrained teams with no capacity and teams with spare capacity.

Consider a team that consists of four cross function scrum teams that all maintain and develop “Component X”. For the next three months, that team would have twenty four team weeks of capacity ( Four teams times twelve weeks times 50%* ) to work on “Component X”.

We have five initiatives that require the capacity of the “Component X” team. We order the initiatives in terms of the value we expect** them to deliver:

• Initiative 1 requires 100% of the capacity
• Initiative 2 requires 50% of the capacity
• Initiative 3 requires 25% of the capacity
• Initiative 4 requires 100% of the capacity
• Initiative 5 requires 25% of the capacity

We can now choose between the scenarios of Initiative 1 only, Initiative 4 only or Initiatives 2, 3, and 5.

We repeat this for all of the teams, creating a portfolio that optimises the delivery of value.

We are left with the portfolio of initiatives for the organisation to deliver, and the capacity utilisation of each team. The teams deliver the first whilst management works to re-balance the second using the tornado map (See Todd Little’s risk presentations) to determine future demand on the teams.

Managing the capacity of individuals within the team.

Rohit Darji and I developed Staff Liquidity and the Skills Matrix a couple of years before I discovered Agile. Dan North took these tools and evolved them into the more elegant and useful Skills Mapping***. He extended the values that individuals self score themselves on to the following.

1. My current skill level.
2. The skill level other members of the team would say I have (Moral Hazard).
3. The skill level I want to have.

Ideally the team then self organises to remove key man dependencies and cross skill to remove constraints within the team. Unfortunately some organisational cultures mean that management need to intervene in order to ensure that skills transfer occurs.

To summarise, manage constraints caused by teams and within teams. Acknowledge that the budget is rarely the constraint. Thats why you need Demand Mapping and Skills Mapping in your tool kit.

My thanks to Joshua Arnold whose post “Resources are the constraint” inspired me to write this.. initially as a comment responding to his post.

* The maximum capacity a team should allow is 50% otherwise queues will naturally build up.

** Expect is probabilistic term. A summary of the range of value based on the probability of them reaching that value. Normally we use a “HIPPO” version of the expectation of value rather than use a formula such as Black Scholes to calculate the value of the option.

*** Check out Dan’s talk at YOW! for a fantastic introduction to Skills Mapping and the rest of Business Mapping… Demand Mapping and Initiative Mapping.

This is a response to the discussion with Martin Burns on Joshua Arnold’s blog about WSJF and SAFE. Martin tweeted “real SAFe talk is Program & up. By defn only 1% of people impacted”

And there you have it. By definition, According to SAFE, only 1% of people are involved in Program and Portfolio level.

Tony Grout and I worked with a large number of people at Skype to create something that Dan North and I now call “Delivery Mapping” (Part of “Business Mapping”). It is a collaborative framework that allows large or small groups of product owners to collaborate on the creation of an organisational level backlog. Rather than tell people what to do it provides some minimal constraints in order to force collaboration and ensure that the right conversations are happening. The outcomes of “Delivery Mapping” are a strategic organisational level backlog based on the team level constraints, and a map showing which teams are constraints, or have no work on the strategic backlog (i.e. Team level utilisation).

To create the organisational backlog, every product owner was involved, along with all of product owner management team. The ideal sweet spot was that tech leads and test leads worked with the product owner in this process. This means that about 20% of the organisation collaborated to create the backlog. Even more are needed to address the team rebalancing effort. It meant that a significant portion of the organisation were involved in strategic decisions, all playing a small part and aware of the bigger picture. Compare that to SAFE where only 1% of the organisation are expected to make all the decisions.

I actually advocate SAFE. They have an interesting story at the program level with the Agile Release Train (ART). It exists on a continuum at the extreme end to LeSS with its self organised feature team creation. That said, the fact that SAFE thinks only 1% of people are needed for the Portfolio and Program indicates that its creators (SAI) do not appreciate that the constraint exists at the team level. They still think that the constraint is the budget (with associated plug compatible programming units). It also indicates that they think some elite group should be solely responsible for creating the portfolio. Our experience shows otherwise.