Product Metrics: The final problem
So what?
Welcome to the final part of Product Metrics for Platform Product Managers.
The final installment in this three-part series comes a week later than anticipated, but I hope you enjoy reading it nonetheless.
I started this series a few weeks ago with the hope of giving practical examples of how folks can think about Metrics in different contexts.
If you want a short recap, the first two parts are:
I left this bit at the end, but this is also the most important part: so you can measure platform metrics, Speed, and/or Productivity. Now what?
Goodhart's Law
The first thing to lead with is the Goodhart's law:
"When a measure becomes a target, it ceases to be a good measure."
The more I progress in my career, the more I observe that Metrics aren't the magic bullet, and they most certainly don't tell you the whole story.
Metrics can be an excellent tool to help you make prioritisation decisions and help you figure out parts of the product you want to focus on. But these should not be the primary goal.
Full Transparency
Whenever you start an initiative like measuring Speed, or dare I say, Productivity, one of the most important things, in my opinion, is transparency.
This is essential. Share the metrics and the why behind them with your product and engineering teams and involve them in interpreting the data and setting goals.
Ensure everyone understands the importance of these metrics and how they contribute to the team's success.
You should also be able to set out expectations and be clear that the Metrics you've chosen will not be used to measure:
individual or team performance
compare teams
the success of the organisation
Knowing if you ship code 200 times a day doesn't matter if, in the end, you don't deliver any value to the customer or the organisation.
This brings me to my last point: learn to look at the whole picture, not just the dashboards.
Qualitative data matters
Qualitative data provides context and helps explain the "why" behind quantitative metrics(i.e.why does it matter you can ship code 200 times a day?).
Looking at this data enables you to understand the factors influencing the numbers, helping you make better decisions.
For example, if code review speed has decreased, qualitative data can reveal whether this is due to increased code complexity, team communication issues, or other factors.
Usually, this type of data includes feedback from team members, stakeholders, and customers. This feedback is essential for making improvements in the development process.
It can highlight pain points, usability issues, and user experience concerns that quantitative metrics may not capture.
Most importantly, this can tell you whether or not your engineers are happy with the culture, team, processes, and collaboration.
Having the whole picture will enable you to truly build developer-centric products.
The Simple Loop
I know from experience that this is hard. Really hard. It also requires a lot of thinking and time spent on collaborating and making good decisions.
Sometimes, you won't make the best product decisions, and sometimes, you won't communicate as effectively. That's fine, too, and it will allow you to learn.
However, the more I think about this, the more I can summarise it into a simple feedback-type loop, represented below:
As long as we don't treat Metrics as the ultimate target and communicate transparently with teams, things should be as simple as that.
More content on this:
A better way to measure Developer Productivity by
- and take on measuring Developer Productivity
How to measure and improve Developer Productivity by Dr Nicole Forsgren
See ya next Friday!