Working with a new client last week we came across an interesting challenge. The teams were just starting out with customer-centric product development, product discovery and OKRs. However, their leadership team, who had invested significantly in the effort to bring these methods into the organization, had to report up to their stakeholders on the return on investment of that work. While not unexpected, it put us and the teams in an interesting position. How do we report on a measurable return on investment for a ways-of-working transformation effort that is literally just getting started? We solved it by scaffolding success metrics into levels. Here’s how we did it.
Level 1: Team-level behavior change
Since the teams are doing none of the new ideas, the first level of measuring and reporting we can provide leadership is evidence of team-level behavior change. For example, if the teams aren’t talking to customers at all right now, we’d like to see them start doing that on a regular level. If the teams aren’t aware of the product strategy or the P&L requirements for the product prior to scoping the work then we’d like to see that be a part of every new initiative kickoff meeting. If teams don’t have customer-centric OKRs set as the goals of each initiative, that will need to change as well. In other words, we want to see teams actually practicing the new ways of working. It’s observable and measurable. It is not, however, an indication of new customer value being delivered but it is an indication that new ideas are being put into use.
Level 2: Decision-making starts to change
Once we’ve identified that new behaviors are taking place consistently we can move on to level 2. We want to see that these new behaviors aren’t just box-ticking exercises but instead are starting to impact how the teams make decisions. Specifically, we want to observe and measure what goes into each prioritization, pivot and product decision. We also want to ensure that the justification for each decision comes from the activities the teams are doing in Level 1. For example, we want to see that customer interviews are the justification for a pivot in the product roadmap or that P&L familiarity combined with broader market appetite for a new idea changed how the work was prioritized. Once again we’re looking at team behavior as an indicator that the investment in new ways of working is changing how the teams deliver value to their customers. While this is a good start, we’re still not at the level that company leadership wants to see. That comes next.
Level 3: Business impact is felt
The final level of measurement for the ROI of this type of transformation is business impact. We want to see that the way teams are now working affects their decision-making in such a way that the products and services they ship to market are better. How do we know they’re better? We are selling more of them. Customers are complaining less. Retention rates go up. Market share goes up. And many other metrics like that. This is why the leadership team originally invested in these new ways of working and it’s finally paying off. And, again, we’re looking at human behavior as an indicator of value only this time it’s customer behavior rather than team behavior.
Leveling works with the right timeline expectations
This type of leveling will work in delivering consistent reporting to the stakeholders of your transformation with one big caveat – you have to set a realistic timeline. You can’t expect a team who starts doing product discovery on Monday to deliver business impact by Friday. This type of measurable ROI takes time. If you’re setting proper expectations you should let your leaders know that Level 1 behavior changes should be observable in the next few weeks. Level 2 metrics that measure decision-making changes will start to become common within a few months. Business impact metrics, level 3, could take as long as a year or even longer especially in large organizations working in the B2B space. This could perhaps be shortened in B2C situations. Regardless, modern ways of working are not silver bullets that work overnight. They take time to practice, improve and ultimately make their way into the customer’s hands. If we can be patient to let them work, the ROI will come but keep an eye out for Level 1 and Level 2 metrics first.
