For any Executive thinking of introducing Metrics, they need to understand that there are three level of Maturity:

• Step 1: Map Investment to Metrics.
• Step 2: Use Metrics to demonstrate Success.
• Step 3: Use Metrics to assist Decision Making.

The final stage of maturity is to use metrics to assist in making investment decisions.

(c) Map image copyright Wikipedia.

John Snow plotted cases of Cholera Outbreak on a map in Victorian Soho. From the map he was able to deduce that the cholera was centered on the Broad Street water pump. The map gave context to the data that allowed him to achieve an insight that changed the medical world’s understanding of Cholera.

The third level of metric maturity requires you to create a “map” by combining two or more metrics that gives insight into how moving one metric will move another. My mentor in the field of metrics is Jamie O’Shaughnessy. He measured the relationship between the conversion rate of customers and the page load time for the web site. The plot of these two variables is shown in the graph below.

Further more he plotted the overall average page load time across the World (Green Line) and in the average page load time in Remote-astan (Red Line). Jamie was able to reduce the page load time by installing an additional server near to Remote-astan. He knew that reducing the page load time to three seconds in Remote-astan would lead to an doubling of the conversion rate, and it did. He was able to take a complex non-deterministic metric (conversion rate) and map it to a complicated metric (page load time) that could be adjusted in a deterministic way. In addition, he knew that reducing the page load time below three seconds was unlikely to improve the conversion rate.

In summary, the third level of maturity means that you can create a map showing the relationship between complex (non-deterministic) metrics and complicated (deterministic) metrics. This map provides contexts that allows you to make better investment decisions with a higher chance of success.