It's time to learn the missing metric

Over the past years I have been looking for a metric that could indicate the agility of an organization. After a study of the more common metrics used for products and management reports, I couldn’t really find a metric that indicates the level of agility. For example in the time-to-market metric I missed the feedback of the actual usage of the product or service. So for the past few months I’ve worked to create this missing metric. The new metric is “Time-to-Learn” or T2L for short. It’s the total time between the moment that the work is started until we learn through feedback from our users. So this includes releasing it to market, gathering feedback and studying it in order to learn. The T2L is a good indication of the agility of the organization because it is all about the interaction time with the market. It provides clarity on the speed a company is able to react to new market opportunities, as well as how responsive it is to real customer feedback.

Example of a T2L of 9 months

Let’s have a look at an example. In the beginning of January a team starts to refine, work out the first details and make the first implementation of a new feature. It’s deployed to customers in May. The gathering of usage data and customer feedback takes 3 months. At the end of September the lessons learned are presented, new ideas are added to the backlog, priorities are re-assessed and some existing Product Backlog Items are removed. The total T2L is 9 months: from beginning of January until the end of September. That does not mean that the team did not work on anything else until they got the customer feedback, but without that feedback or learning the team might not be working on the right, highest value work. In this example, the organization took 9 months to learn.

How to start tracking the T2L

There is a relatively easy way to get a good measurement. Every sprint the Product Owner picks one or two important Product Backlog Items (PBIs) that are tracked throughout the next months. The date the work is started is recorded. Every sprint the team also spends time on learning from actual work it has done several sprints ago. The moment the learning is done, the date can be used to calculate the T2L of that individual Product Backlog Item. Using the data of multiple PBIs the average T2L is found. Tracking the development of the T2L is handy to see if the improvements really result in a better T2L. Start small and start simple, but start thinking about how to instrument your backlog items to drive learning and capture when that learning happens.

3 benefits of an improved T2L

1. Upfront discussion on what we want to achieve

When the team routinely starts applying T2L, it’s more likely to be focused on what they really need to achieve as a team. They ask questions of the work such as: when is a feature successful? When 1.000 users use it daily? When the conversion on the website has increased by 5%? How are we going to measure success? When the team starts thinking about T2L it is often easier to focus on value with both the work undertaken and the solutions selected.

2. Focus on learning more, and not on doing more

Presenting this T2L during the sprint review often removes the focus on the velocity and the details of the planning. Of course how effective development is important, but only in the context of doing the right thing or delivering the right learning. When stakeholders get transparency on the outcome and impact of the features delivered several sprints ago, the mindset of the meeting shifts. It shifts more to long term goals based on value such as usage, market penetration and customer satisfaction. This often results in quicker experiments, learning from actual data and finding improvements more rapidly.

3. Faster learning is faster value

With a short T2L the organization can learn a lot faster on many fronts. The team starts to understand the customer demands better, because it sees if the new functionality or added features are used or not. The Product Owner learns what delivers actual value and what does not. With a long and slow learning loop, so much new functionality is added that it’s hard to deduce what actually improved it. With a short T2L, it’s much easier to know what led to the improvement and therefore what is of real value and what is not.

Additionally, the team gets quick feedback on the quality of their work so they know more quickly what to improve on their craftsmanship. Stakeholders also start to see which ideas were actually good ideas and what the customers really need.

Conclusion

The T2L is the missing metric to measure the agility of the organization. It’s the total time between doing the work and learning from actual customer usage. I hope you start to track the T2L. Tracking the T2L for only a few items per sprint is already good enough to know your T2L. It will give you several benefits like focus on the goal, not working harder but working smarter, and will help develop a learning organization. The T2L metric and the mindset behind it can be used for a wide variety of work. From making an internal memo, updating a report, developing internal software for colleagues to actionable outcomes of the retrospectives and management decisions.

By thinking about T2L you will get insights into your true agility and what you can do to improve the responsiveness of the organization. In a world that’s changing faster and faster, the urgency to accelerate within that world increases. The T2L is the missing measurement to see if you are moving fast enough to keep pace with your own complex market.

About the author

Peter Koning has over 12 years of experience with Agile and Scrum. He is a professional Scrum trainer and gives monthly Product Owner trainings.

For many years he has been the Agile leader for several Scrum Teams. Together with severall companies he is developing an Agile Leadership framework to really support Agile leaders in their complex job.