A significant risk in any business is that you do not know where you are. Value is the thing that we aim to deliver, but value data or metrics are how we measure our success at achieving our goal. The only way we know for certain that we have achieved our goal is when the value data shows us success. For example, we aim to deliver $2million from a feature. The only way we know we have achieved that is when we have the $2million in our account.
- How do we know that it is the new feature that has delivered the value?
- Do we have to wait until the money rolls in until we know how we are doing?
If we have a single feature that people are buying it may be fairly simple, but that is rarely the case as most applications have many features. We have to put more thought into how we look at our value data. First, we need value data on how the user is using the new feature.
- Do the users use the feature? Did the
- How often to do they use the feature?
- For how long do they use the feature?
Our value data impacts how we might introduce a feature to our users. Do we offer a couple of free goes before they have to pay or do they pay before they use? Offering a couple of free goes allows us to gather value data (or metrics) on how people use the feature. We want to know how long the user spends on each step of using the feature and whether they stop using the feature before they complete all the steps. This will help us understand whether the user values the output enough to put the effort in to provide all the inputs. The value data will tell us which step result in people dropping out of the process.
Context is everything. Context drives how you use value data. A web site is different to an installed application. Retail is different to enterprise. The value of each customer impacts how much we are prepared to test on them. The golden rule is to get feedback on our business case.
If we have an application that the user can download for free and have a couple of free goes before they pay, we have to build a business model ( or belief system ) around how we will make money.
For example, we assume 1,000,000 download the application. 50% use the first free go with the feature, of which 20% use the second free go, of which 10% then pay for the feature. The feature costs $10.00. This would result in revenue of 1,000,000 * 50% * 20% * 10% * $10.00 = $100,000. We could release an earlier version of the feature which tests to see how many people download and use the free versions. We can look at when people drop out of using the feature.
We aim to deliver value but the value data (or metrics) let us know whether things are going to plan.
January 18th, 2013 at 12:48 am
Loved the article, as someone who plays in the IT Risk market it resonated at a lot of levels. Thanks!
January 21st, 2013 at 3:51 pm
Interesting post. That’s a question I’ve often tried to deal with, without any success.
As you said, a web site is not an installed app, and retail is not enterprise. I’ve always worked in the worst case context regarding lean startup, which is an installed app for enterprise. Which means:
– long sales and integration cycles (thus high difficulty to map features and value)
– few sales with big amounts (thus too few numbers to compute relevant statistics)
– high difficulty to map usage and value to the different stakeholders (business, buyers, end users, admins, integrators, sales,…).
If you have any clue to help me getting started, it’s a pleasure 🙂
March 1st, 2013 at 11:18 am
We can release an earlier version — but do we release restricted functionality? If we make a restricted release, are we learning what we think we’re learning about user drop-off, or just frustration at the restrictions? Are we likely to get feedback as well?