For this week’s blog post I am going to answer a reader’s 2-part question. I regularly get questions from folks and try to respond to all of them. Sometimes though, the answer benefits all readers. I’ve not used any personally identifiable information to maintain the reader’s privacy. Here are the two questions that came in:
Cost-saving while maintaining the customer experience
My company is trying to reduce costs by changing a core material in our (physical) product that is cheaper to use. The goal is to make the change completely unnoticeable to users. Is there a behavior to measure here to tell us how well the idea is implemented and working?
I think this is a common use case for many companies. Sometimes we’re not necessarily trying to improve the user experience and, instead, improve our bottom line. At the same time, we don’t want to make the customer experience worse. What should we measure to determine if the cost savings we’re implementing are having a negative impact on our customers?
In this case, I would recommend looking for a flat behavior change. In other words, customer behavior should not change at all from what it is today despite the change in the core material. While a positive impact would be nice to have, the customer isn’t supposed to feel a change at all. Therefore the current baseline should stay flat. And it should definitely not worsen. If it does, your customers have figured out you’ve made a change and it’s had a negative impact on their experience.
Enhancing system resilience against cyberattacks
We are enhancing the security around our system to better protect from cyberattacks. While some of this work requires our developers changing their behavior, a big part of it is purely technical. It will focus primarily around new configurations and automated policies. Who is the “customer” in this case? And if we’re trying to not get hacked, what behavior changes should we measure?
In this case I think there are two customers – the developers and the hackers. For the developers we would want to see a reduction in behavior that increases the likelihood of attacks. If these new configs and policies have the desired impact, our developers end up behaving in less risky ways. This should be directly measurable in the way they interact with their development environments.
For the hackers, there are at least two metrics to keep an eye on. One would be the number of attempted attacks they try to commit. If your new policy is effective they should eventually get frustrated and try their attacks somewhere else. The other is the number of successful attacks they perpetrate. You might want to measure this as an absolute number or as a percentage of attempted attacks (or both) but in both cases the increased preventative measures you’re putting in place should reduce both of these behaviors.
A lack of behavior can also be a success metric
The great Canadian poet Neil Peart once said, “If you choose not to decide, you still have made a choice.” This holds true for OKRs as well. If your customers don’t change their behavior, that’s still a reaction worth measuring. Sometimes the work that you’re doing, as in the case of the reader, isn’t designed to change anything noticeable for them. In this case, their lack of behavior change is the signal you’re looking for. Overall, there’s always some signal to pick up on when you’re changing your products and services. It may not be obvious at first but as you work your way through the proposed enhancements and what your customers are doing right now, you can work your way through to the desired change.
P.S. — Our book on OKRs, Who does what by how much? is on super sale until January 1, 2026 at 50% off. Get your copy now.
