Product Metrics for Platform Product Managers
Part One: It's all about the definitions
I have been recently slacking at writing content relevant to Platform Product Managers because I was in a bit of a funk. I wrote about PM'ing, Product Vision, Strategy, Running Developer Survey initiatives, and even Positioning. A break was much needed. Or was it?
Anyway, enter my community of subscribers to the rescue - as I recently received many messages to write about specific Platform Product Metrics.
And because I listen to feedback, I am kicking off another three-part series on Product Metrics. Exciting, I hear you say!
In today's piece, we will dive into:
Defining Platform Product Metrics
When to start thinking about them
Different categories of Metrics we might want to think about, depending on the context
Defining Platform Product Metrics
If you've read my Newsletter, you know that I always keep talking about context. This is the first thing to consider when you want to start thinking about Metrics for your Platform.
A boilerplate definition of Platform Metrics can sound like this:
Metrics for Internal Developer Platforms are a way to measure the scalability, reliability, and overall health of a platform or Infrastructure that supports various products, services, or applications within an organisation. These metrics can help platform teams better understand their performance and continuously improve their functionality and reliability.
As excellent and as scientific as this sounds, some questions to answer when you want to start defining these metrics are:
What is your Product?
Is it Infrastructure?
Is it Tools?
Is it data?
Is it THE golden-path?
All the above?
What problems are you trying to solve?
Are these problems related to the scalability of the org?
Are these problems related to Speed to market?
Are these problems related to docs and processes?
Are these problems related to people?
What is your Strategy to solve these problems, and what data do you need to help you make decisions?
Answering these questions above should give you a good start in defining your Platform Metrics.
Although these might seem like a lot of questions, it's something that you want to be on top of because this is a significant area of impact.
As usual, the last part of my series includes examples to tackle this together.
When to start thinking about Metrics?
Think about it this way: you never want to reach the dialogue below.
Senior Person: You need to move faster!
Product Manager: Can you please define fast?
Senior Person: McKinsey said we should measure code velocity!!! How many times are we shipping the code to production?
Product Manager: I don't know. You never agreed to pay for that tool to help us measure it.
Joke aside, the lack of clear metrics has been a problem I have constantly observed in my roles, which I am passionate about in my day-to-day role. I am the annoying person who asked you at one point how you plan to measure that thing on your roadmap.
I am not advocating for metrics over everything, and I know that some are qualitative. Even I am struggling with this from time to time. However, as PMs, we should find a way to measure the impact of our initiatives.
A boilerplate answer to this question would be:
The best time to start thinking about product metrics is during the early stages of product development, ideally before you even begin building the product.
When you know you want to kick off an initiative, even a Discovery piece, you need a way to figure out if you succeeded or failed. And most importantly, when to stop.
Think you are going through two of these phases:
Ideation and Planning Phase:
Conceptualisation: Begin by defining your developer platform's purpose and value proposition. Consider what problems developers are facing and how your Platform can address them.
Developer Needs: Understand Developers' needs and pain points by conducting surveys, interviews, or engaging with developer communities. Identify metrics that align with developer satisfaction or happiness.
Design Phase:
Feature Definition: Determine which platform features, patterns, or APIs will be provided to developers. Define metrics that track developer adoption, usage, and the effectiveness of these features.
Documentation and Onboarding: Plan for comprehensive documentation and onboarding materials. Metrics related to onboarding completion and developer feedback can be valuable.
Whatever way you decide to go about this, make sure it's relevant to the current problems that exist in your organisation.
We should consider Different categories of Metrics, depending on context.
I have included some concrete examples below and I hope this will give you a good start.
→ Developer Engagement Metrics: These are Metrics related to the way Developers interact with the Platform
Examples: Active users, session duration, feature adoption rates, API usage, and developer engagement.
Note: DataDog is excellent at giving you out-of-the-box basic application usage metrics like the ones mentioned above.
→ Developer Happiness Metrics: These Metrics measure Developer Happiness and, if you ask me, the most important.
Examples: Net Promoter Score (NPS), Satisfaction Scores, Sentiment Analysis.
Note: These can be measured by running developer survey initiatives and some CSAT scores you have internally, like Peakon.
→ Performance and Reliability Metrics: These Metrics assess the Platform'sPlatform's Speed, reliability, and uptime.
Examples: Uptime percentage, response time, error rate, and latency.
Note: Depending on your Observability, DataDog is an excellent tool for this.
→ Developer Adoption Metrics: These are Metrics that track how extensively the Platform is used and adopted.
Examples: User growth rate, API call volume, number of integrations, and user onboarding completion rate.
Note: I don't have a specific tool suggestion here as there are quite a few, and it depends on your tech stack.
→ Security and Compliance Metrics: These Metrics evaluate the Platform's security and adherence to compliance standards.
Examples: Number of security incidents, compliance audit results, and vulnerability response times.
Note: Vanta is perfect for this.
→ Cost and Resource Utilisation Metrics: These are Metrics that help manage the cost-effectiveness of the Platform
Examples: Infrastructure cost, resource utilization efficiency, and cost per user.
Note: Very recently, I discovered kubecost for this, but it comes with its limitations.
Though these can include more examples, I want to talk about some of them in a bit more detail. Part Two of this series is coming up next Friday, where I will talk about Speed at all costs and Scale.
Before I part ways, here is a good podcast related to this from
:See ya next Friday!