Years ago, Eric Ries, author of The Lean Startup gave us the term “vanity metrics.” He defined vanity metrics as the metrics you measure that “make you feel good, but they don’t offer clear guidance for what to do.” Eric stated that if you’re measuring something and it doesn’t give you clear, actionable next steps you’re wasting time and effort measuring that thing. I agreed back then and I still do. When it comes to digital products and services typical vanity metrics can be things like number of visits to our website or the number of times our app was downloaded. Just because someone interacted with your website or download page doesn’t mean you’ve actually delivered anything of value. Worse, there’s no clear sense of what to do with these specific metrics – whether good or bad. However, in a recent LinkedIn discussion of the OKRs we set for our recent book launch, it dawned on me that there is value in vanity metrics. Indulge me a bit more in the next paragraph. 

What can we learn from vanity metrics?

Our initial set of metrics for the success of our new book were focused on sales. The argument against those metrics was that these vanity metrics didn’t tell us anything about the quality of the book, the impact it has/would have on those who read it or the change it might create in the organizations who put our ideas into practice. This is true. Sales numbers won’t tell us that. What these vanity metrics will tell us though is whether or not we’ve correctly assessed the problem space. We wrote a book that helps organizations implement OKRs successfully. If that’s not a real problem to solve we won’t sell very many books. 

If this is a real problem for a sizable market (as we’ve assumed) the next thing we can learn from our vanity metrics is whether those who have this issue want to hear from us (Jeff and Josh) on this topic. Did we make enough people with this issue aware of the book’s launch? Furthermore, are we credible on this topic? Will people seek an answer to this problem from us? Sales numbers also help us understand that.

In addition, sales and review metrics tell us how well the material is resonating with those who have bought and read the book. Were they motivated enough by the material to write a review – good or bad? Did they share the book with a friend, colleague or boss? Did they buy a box of books for their teams? Again, these are indeed vanity metrics. The goal of the book, ultimately, is to change how organizations work for the better, to make them more customer-centric. But, for now, in the early days we need to see if we can get the book into people’s hands. There won’t be any change without that. To understand whether we’re achieving that we have to use vanity metrics. 

Vanity metrics measure if an activity is taking place

For years, when working with teams and teaching them to do customer research, I was a steadfast opponent of measuring “the number of times we talked to a customer” as an indicator of growth and improvement in an organization. “That’s a vanity metric!”, I would say. “You could talk to 1000 customers and learn nothing.” And that’s true. Just because an activity took place doesn’t mean you got any value from it. Same as buying a book – just because it was purchased doesn’t mean it had any impact on the buyer. Then, one day, a lightbulb turned on in my head. If an organization is doing exactly zero customer interviews today then setting goals and then measuring the number of interviews ensures that this activity will take place. This might be a vanity metric but it’s an amazing first step from where the organization is today. They may not be getting a ton of value out of these interviews right away but they’re getting practice. They’re building the muscle and the systems to allow this to happen more and better in the future. The value – the “real” metrics – will come later. 

So, yes, vanity metrics aren’t the ultimate measure of your success however, at the beginning of a new product, process or activity they do provide insight as to whether you’re addressing a real need, your ideas are resonating and if the activity is even taking place at all. Value will come later. So will better metrics. For now, vanity metrics do help give us direction.