Imagine yourself as a company looking to develop a new API of your product, or you could be a Product/Project Manager of the said product. How would you rationalise the acceptance and success of the work done? The crux of the calculation falls on three primary metrics at launch — Adoption/Engagement/Retention (AER).
Background to AER –
To begin with, you list the methods of experimentation to start the adoption of your new API. Once you’ve set yourself on a few different methods to ensure adoption, you need to start measuring it. With adoption, comes engagement. Engagement is primarily an indicator of how useful you find a given product. And once users start to engage with the product, the next question that you need to answer is retention. Retention tells you exactly why users keep coming back to your product and who are your users. It tracks the decay function of that engagement overtime for a cohort of users. You need to keep shifting between these three stages and focus on doubling down on efforts to improve the metrics in each stage. This in turn helps you ensure that you’re API is meeting the larger level goals set by the business of serving the customers and help decide sales and marketing budgets.
Let’s start with –
Examining API Adoption Rate (API AR)
API Adoption Rate tracking is similar to tracking the adoption rate of any other SaaS tools. It is a step by step process. Adoption of API is measuring the conversation rate between each step and also the time is taken to move to the next step.
In AR journey, the steps are broken down in three majors parts –
- The User, generates the API key
- User makes their first API call in a Sandbox environment, or also popularly known as Time to First Hello World (TTFHW)
- The final step is deploying their app in production
We track the user in via this flow while they are in the Adoption Phase and measure AR. We measure the number of hours spent in each phase and then average it out to get a trend. This in turn allows us to identify the areas that require improvements.
API Engagement Phase
With the adoption metrics setting the user’s adoption of APIs in place, the next step is to understand the user’s engagement post-adoption. Engagement Rate (ER) focuses on high user engagement and ensuring high usage of the APIs.
When it comes to ER, it’s important to focus on metrics that determine the success of the APIs. These include infrastructure metrics that allow the product and engineering team to monitor the usage of the APIs. There are two valid metrics that help us understand ER.
I. Distinct Token Access (DTA) defined the number of unique tokens accessing your API per week. By tracking this number, we understand the active number of users per week and their background so the product can be improved according to the feedback from the regularly engaging users who had come on board in the on-adoption phase. This also creates extra insights for the sales and the customer service team, so they can focus on the key users.
II. Endpoint Tracking (ET) entails tracking the features of your APIs that engage the most number of users. This brings forth the key areas of your API that are driving engagement and the not so well used parts of your API. This allows the product and engineering team to improve the product by ensuring the current engagement levels to sustain and not so well used APIs are either deprecated or updates to meet the user’s need.
However, for APIS, there are a few ERs that can be misleading –
- Request per Minute – while having high requests per minutes for your API indicate a high usage post-adoption, it can be misleading as it might be led by bot traffic or poorly designed APIs. Hence it’s worth avoiding unless you’re absolutely sure about your API design and the traffic accessing it.
- Average Latency – having high latency / slow APIs are acceptable with partners consuming API because it’s predictable hence it is not an indicator of the engagement with the APIs.
Retention Rate (RR)
With ER rate being tracked and feedback into the loop of product development, the next step is to ensure the APIs have a high retention rate. However, we also need to track the APIs with a lower consumption/retention rate.
- Monitoring API RR – from the moment users sign consuming APIs, certain APIs will have a higher RR while others will have a lower RR. APIs with the high number of users consuming it showcases that the purpose of the API has been successful. However, APIs with a low retention rate indicates that we need to first get our product right before investing time and effort in the AR & ER.
This first version of this blog focuses entirely on PMs improving the product. In the next post, the focus will be more on the metrics that help Engineering teams improve the API offerings.