About a year after implementing Capacity Planning ( aka Delivery Mapping ) at Skype, the product leadership team asked for reporting to support the process. Tony Grout, Lisa Long and myself implemented the “Strawberry-jam-o-meter” and the “Wrong-order-o-meter” with the product leadership team ( report names inspired by Dr Seuss books which were a favourite of one of the product executives ). The product organisation consisted of about two hundred product owners working with about three hundred scrum teams. Capacity Planning meant that for a year the product organisation had come together once a quarter to prioritise an organisation wide product backlog based on the available capacity of teams.

Skype consisted of several front end products (Windows, Windows Tablet, XBox, iOS, Android and Windows Phone) that implemented back-end products (Audio/Video Codecs, Transport, Login). The architecture was constructed using Reverse Conway principles with the aim of allowing each scrum team in a release vehicle (a set of scrum teams) to be able to deliver independently.

The organisation wide backlog (known as the meta backlog) was a set of business goals whose success was measured using metrics. For example, reduce the day one churn (number of users that only use the product once and never return), conform to regulatory rules to prevent significant fine or enable asynchronous chat by moving it chat from peer to peer to server based.

Anyone who has worked in a large organisation knows that it is pretty much impossible to set up your organisation such that every business goal can be delivered by a single scrum team. Several teams are often required to achieve a business goal. Large organisations need to create a single ordered backlog to provide focus and create alignment between the teams.

Once the leadership team had established the process to create an ordered backlog, they were able to get better insight into what was actually happening in the delivery. To that end, they reduced work in progress by over 50%. They also created two reports:

The Strawberry-Jam-O-Meter was used to identify teams that had the most backlog items in progress. Inspired by Jerry Weinberg’s “Law of Strawberry Jam”.

The Wrong-Order-O-Meter was used to identify teams that were working on items in the wrong order according to the organisation wide backlog that the product owner organisation had specified.

The reports were to help the leadership understand which teams needed their help and support. The reports provided bad smells that helped them identify issues to resolve. To quote Jabe Bloom, the reports Were an “invitation for the leadership to go to the gemba”. There was an understanding in the product leadership team that the “behaviours” were often a result of constraints in the system. For example, the team is blocked waiting for another team and moved onto the next item. In some cases the product owner was not aware of the impact of too much work in progress which was a “learning opportunity”. In one case, the product owner on the 16th priority was very aggressive at getting their own delivery done such that they put the 1st item at risk.

The two reports provided leadership with a system wide view so that they could identify constraints. The leadership were more interesting in solving system wide issues rather than issues with an individual team.

Layout and Calculations

The Wrong-Order-O-Meter was very simple. It showed the team name and the effective position in the backlog that they were working. It was possible to drill down on each team to show the items they were working on.

Team	Effective Backlog Position
Griffindor	3.5
Slytherin	3
Hufflepuff	1

Drilling down into Slytherin’s backlog:

Organisation Backlog Item	Priority	Effort	Priority Weighted Effort
Improve App Store ratings in Asia	1	8%	0.08
Reduce churn in Asia	2	35%	0.7
Improve on-boarding conversion rate	3	14%	0.42
Increase new customers into Funnel	4	35%	1.4
Items not on the Organisation Backlog	5	8%	0.4
Totals		100%	3.0
Effective Backlog Position			3.0

Fifty percent of the effort in the organisation will be for “Items not on the Organisation Backlog” because the teams will not be able to work on the backlog item as there will not be enough capacity in the teams acting as the constraint in the system. For this reason, the leadership needed the strawberry-jam-o-meter as well to identify teams that are spread too thin both for items on the backlog and for items not on the backlog.