In recent training engagements I came across two scenarios where trying to make an OKR work as a measure of human behavior became challenging. Both teams I was working with served live content online at scale. One was serving advertising. The other was serving streaming media content. Both teams clearly understood our position that Objectives and Key Results make a big impact on a team when the key results are outcomes – measures of human behavior that drive business results. However, in both cases, neither team could tell us how their specific metrics fit into our template of “Who does what by how much?” Specifically they were talking about a metric called “cost to serve.”
What is “cost to serve?”
As I understood it from these two teams (and a huge thanks to Sean O’Neill for my crash course in this topic), cost to serve (also known as total cost of ownership) is the overall financial impact to the organization to serve a finite amount of content at a time. For an ad-serving business for example these costs could include:
- Developer, data science, product people, et al costs
- Server and cloud hosting costs
- Vendor licensing costs
- Manual processing costs
The ad-serving team would add up all these costs and divide them by a finite number of ad units, say 1000. The goal is to create a metric that can be measured over time, seasonality, country, global region, etc.
For the streaming media team, the fundamentals are the same. “What does it cost us to serve each show to a certain number of viewers?” They would take into account similar drivers of cost. These metrics become a direct indicator of what it costs and will cost to scale the business and where there might be further opportunities to automate the business. In fact, it would become nearly impossible to assess the profitability of the business without these metrics.
“Cost to serve” is not a human behavior
Reconciling cost and total cost of ownership with OKRs becomes an interesting challenge because these are not measures of human behavior. They are measures of system performance and efficiency. Increasing or decreasing these metrics doesn’t directly impact any specific human inside or outside the business – at least not their behavior. You could make an argument that, at some point, if cost to serve isn’t measured and managed at scale the end customer, be it an advertiser or a tv viewer, would feel the impact of an increase to their fees. But the distance between that ultimate end result and the teams optimizing these largely technical system behaviors is far. Tying the optimization team’s goals to profitability directly or even subscription or advertiser purchases is unfair and unrealistic. There are many steps between the work they do and these human behaviors at the end of the value chaing.
Not all metrics are OKRs
The conclusion I’ve drawn from this experience is that there will always be system performance metrics that require optimization. These system behaviors are a core part of the value delivery mechanism of the business but, at the end of the day, they don’t directly impact any human behavior. In this case, these do not make good candidates for OKRs. That doesn’t mean they shouldn’t be set as goals for the teams that optimize them. Quite the contrary, these are exactly the goals these teams should be working towards. They’re just not going to be OKRs in the classic sense.
Reconciling that in an organization that has adopted OKRs broadly may be a bit tricky but ultimately it’s the right thing to do. Level-setting the team’s goals is critical to their success. Goals should assess work teams can actually influence. In the case where the influence isn’t directly tied to human behavior, as in cost to serve, a team will sit “outside” the OKR framework. It shouldn’t isolate them nor give them a free pass to forget the end user. However, they’re focus is going to be optimizing system behavior. And that’s ok.