How do you measure product discovery?

If you’ve read Lean UX and are practicing product discovery, you’re likely being asked regularly for measures of progress. How will we know if the team is “doing it?” How will we know if they’re doing it well? How will we know it’s having an impact on our ways of working? And an impact on our products and services? These are valid questions. If we’re going to ask our companies to invest in training us in product management and discovery we should be able to show return on that investment. 

Is product discovery happening at all? 

If we’re going to measure product discovery we should start at the beginning – is it happening at all? There’s a risk in measuring whether an activity is taking place at all. At the end of the day it’s a vanity metric – a metric that makes you look good to others but doesn’t actually measure whether value is achieved. Just because you spoke to a customer, built a prototype or ran an experiment doesn’t mean that anything of value came out of it. I tend to believe that if a team is not doing any product discovery at the moment then measuring activity is a fine place to start. Even if the activities yield little useful insight, at first, there is value in the teams figuring out how to do discovery work, how to get it prioritized and how to share their work with their colleagues. 

So, yes, start with measuring things like:

• Number of customer interviews held
• Number of experiments run
• Number of hypotheses written

Let’s at least get the team thinking about doing the work. 

Is product discovery being deployed well?

Now that we have the team going through the processes of product discovery, we need to determine if the work is being done in an effective, efficient way. If the teams are doing good product discovery work we should be able to measure insights collected that have had an impact on product work. We should be able to measure:

• Percentage of experiments yielding actionable insights
• Percentage of products and initiatives that have been positively impacted by learnings from discovery work

The teams should be generating insight from their various assumption tests. If those insights are not coming in, there’s an issue with the way teams are executing discovery work. If insights are coming in and not impacting the actual work in flight, the teams aren’t testing the right assumptions. In both cases, insight should be coming in and relevant to the work the teams are doing. 

Is product discovery impacting our ways of working? 

One of the biggest challenges when integrating lean ux and product discovery work into a product development workstream is maintaining momentum over time. There might be time made for learning work at the beginning of an initiative but as the work goes on, all too often the discovery work gets deprioritized in favor of shipping features. If we’ve invested in it, we should be able to see this work taking place consistently over time in the way our teams are working. To that end, consider measuring:

• Average percentage of a sprint spent doing discovery work
• Percentage of sprints that have at least one discovery story
• Percentage of prioritization changes made based on insight gained from discovery

I would also measure who is doing the discovery work. I would expect it to be product management and ux design in the early days but I’d hope to see broader participation from engineering and other disciplines as time goes on. 

Is product discovery helping us build better products and services?

This question should be the easiest to answer if you’ve instrumented your products properly and have clear baselines on where things are today. If you’ve identified a workflow that is underperforming, worked through the product discovery process to understand why customers are behaving that way and then adjusted the user experience to reflect your insights, the subsequent behavior changes are directly attributable to the product discovery work your team has done. There are countless stories of how a deep understanding of the customer and a humble, test and learn attitude have led to massive impact on customer behavior and success rates. If you want to understand how product discovery is affecting your products and services consider measuring:

• Baseline customer behavior in a particular workflow vs behavior after insight was gained and improvements implemented
• How that behavior improvement impacts the top and bottom lines of your organization (Did it reduce complaints to the call center? Did it get customers to purchase more successfully? etc)
• How much did the deployment of unvalidated assumptions and hypotheses cost us in lost revenue, operational costs and profits?

We work in the most accountable medium

Building digital products and services makes understanding how well our work meets the needs of the market simple. Everything is measurable. If we choose to invest in understanding our customers – both quantitatively and qualitatively – we build a baseline understanding of what’s happening now and how anything we change in our ways of working affects that. Product discovery is no exception. As we work to implement a ship, sense and respond cycle in our work we can quite literally measure whether it’s happening, the impact it’s having on our teams and our work and, ultimately and most importantly, our customers. These criteria can then be used to justify further investment in the product discovery process. Everyone will benefit from that.