A common management adage goes, “What gets measured gets managed”. This rule, attributed to V.F. Ridgway, communicates more than just the importance of measurement. But there’s more to the story. Simon Caulkin summarized Ridgway’s study of performance management this way:
“What gets measured gets managed – even when it’s pointless to measure and manage it, and even if it harms the purpose of the organization to do so.”
When measuring and evaluating performance is of the utmost importance, how can product teams ensure they measure and manage the most important variables? Furthermore, how can measurement benefit the organization?
When scrum teams measure the most important metrics, they can evaluate their performance over time. This helps teams improve effectiveness and pivot quickly when needed.
What is scrum?
Scrum is a product management framework that helps teams iterate product solutions. In scrum, timelines are scoped in 1-4 week sprint increments. This helps teams structure and estimate work. The scope of work for any sprint is defined by an estimate of how much the team can complete within that sprint. At the end of each sprint, the delivered work should be ready to be delivered to customers.
Scrum frameworks are based on agile project management philosophy. Under agile, teams prioritize collaboration and iteration over processes and tools. This helps teams to adapt quickly to changing needs and priorities, and ultimately deliver solutions iteratively.
What are scrum metrics?
Scrum metrics are specific data points that teams gather and track to improve their efficiency and effectiveness. Tracking scrum metrics helps teams estimate workload and project completion timelines, set goals, and improve performance over time.
How can teams gather scrum metrics?
Scrum has different recurring events that can help teams gather data. These events are:
- Sprint planning: A meeting at the beginning of a sprint to break larger priorities into workable tasks. This provides a detailed estimate of the scope of the work and the amount of time it will take to complete that work.
- Daily scrum: A daily meeting where team members share progress updates and roadblocks they are facing. This also recaps the hours remaining for sprint tasks, which helps teams calculate sprint burndown.
- Sprint retrospective: A meeting at the end of a sprint to recap what went well, discuss what didn’t go well, and share ideas for improvement.
Why are scrum metrics important?
Although scrum events are pre-set milestones for teams, the events alone don’t guarantee progress or success. These are simply checkpoints throughout a project. Events don’t indicate the quality of the work being delivered, or how efficiently it was completed.
Tracking scrum metrics enables teams to estimate workload and project completion timelines. Over time, teams can benchmark their performance and guide future projects. Scrum metrics help the team can also help the team craft a compelling story about their work. Over time, these metrics may approximate the satisfaction and effectiveness of the team overall.
Are scrum metrics KPIs?
Although the terms “metric” and “KPI (key performance indicator)” are often used interchangeably, they are not the same concept. Metrics are units of measurement that can be used to measure KPIs. On the other hand, KPIs are metrics with a set, time-bound goal.
Scrum metrics can be used to set KPIs. For agile product teams, KPIs should demonstrate how well the team satisfies customer and market needs, and overall supports company goals.
Scrum metrics to track
Here are 9 metrics that teams can track to evaluate performance and set KPIs of their own. This is not a comprehensive list of all scrum metrics. Product teams should decide which metrics are tracked, how often they are tracked, and when and how those metrics are communicated with the larger team.
1. Story Completion Ratio
Story completion rate is the ratio of actual stories completed vs the number of stories the team committed to. Product teams commit to stories during sprint planning and review the number of stories completed during the sprint review. If a team commits to 10 stories during sprint planning and delivers 8 by sprint review, their story completion ratio is 80%.
Story Completion Ratio = # of Stories Delivered / # Number of Stores Committed
2. Technical Debt Index
In software product development, technical debt refers to the work that builds up when developers implement quick, short-term solutions instead of working through better, more labor-intensive solutions. Essentially, technical debt is the cost of slapping a band-aid on a product to make a quick fix or complete an MVP.
Technical debt can be measured using a debt index. This is calculated by dividing the total technical debt by another metric that represents the full scale of the project or system. For example, you can calculate the debt index by dividing the total technical debt (measured by estimated hours of work or story points) by the codebase size (measured by lines of code). The resulting ratio approximates a scaled picture of the remaining technical debt.
Technical Debt Index = Technical Debt / Codebase Size
3. Velocity
Velocity is one of the most important metrics in product management. This represents the consistency of the teams’ estimates over time.
Actual velocity is the sum of units (often story points) delivered in a sprint. This is calculated as the number of delivered units divided by the total number of sprints.
For example, if a team delivers 15 story points in 1 sprint, then their actual velocity is 15.
Actual Velocity = 15 story points / 1 sprint = 15
If a team delivers 100 story points in 4 sprints, then their actual velocity is 25.
Actual Velocity = 100 story points / 4 sprints = 25
Expected velocity is the total estimated story points divided by the total number of planned sprints.
For example, if a team estimates they will complete 45 story points in 3 sprints, then their expected velocity is 15.
Expected Velocity = 45 story points / 3 sprints = 15
Actual velocity isn’t a measure of productivity, it simply represents an average unit of completion based on past data. Actual velocity may change due to outside factors like holidays, team changes, and technical issues. For this reason, actual velocity should be used as a planning tool. Over time, actual velocity can be compared to estimated velocity to see how team output differs against expectations.
4. Burndown
Sprint burndown helps teams measure the work remaining during the execution of the sprint. This is a visual representation, usually a line chart over time, that represents the rate at which work is completed against how much work still needs to be done. On a burndown chart, the x-axis represents increments of time, while the y-axis represents the number of tasks to be completed.
5. Escaped Defects
The number of escaped defects helps product teams evaluate the quality of the product they are delivering. This metric shows how many bugs, or defects, “escaped” testing and were experienced by users in production. In an ideal world, scrum teams can fully test stories to avoid escaped defects. In reality, bugs are sometimes deployed to production.
Defect density can also help teams understand their product quality. This measures the number of defects per software size, such as lines of code.
Tracking the number of escaped defects over time, as well as the defect density over time, can help your team understand the quality of releases and whether that quality improves or degrades over time.
6. Sprint Goal Success
Sprint goals are 1-2 sentence statements that outline the goals of a sprint. They are an optional part of the scrum framework, but tracking this metric over time provides insight into whether teams are meeting their goals. If a team is consistently missing goals, this data can help them investigate whether their goals are realistic. Furthermore, this can help them identify roadblocks to goal success.
7. Workload Distribution
A strong product team is essential to creating a strong product. Monitoring workload distribution helps ensure that all team members are taking on what they can handle. Without visibility into workload distribution, some team members may be overworked, while others may be underutilized. Ultimately, both issues can contribute to low employee satisfaction.
Workload distribution can be used as a proxy variable for productivity, which can create stress for employees. Be sure to create an environment of trust and psychological safety when tracking workload over time.
8. Team Satisfaction
Team satisfaction is a major component of a successful scrum team. If your team members don’t feel that their needs are met and aren’t enthusiastic about their work, then no process or methodology can fix your team’s problems. Checking in on workload, work/life balance, happiness, and motivation not only helps you gather data on your team’s satisfaction – it also shows that you care about your team as people.
9. Customer Satisfaction (NPS)
While there are several ways to measure customer satisfaction, Net Promoter Score (NPS) is one of the most common. NPS measures whether users would recommend the product to others. Strong, consistent NPS scores mean that the product team is delivering valuable products to customers.
In conclusion…
Monitoring and tracking scrum metrics can support agile product management in several ways. By recording and visualizing this data, scrum teams can enhance their efficiency, effectiveness, and their ability to adapt quickly to changing needs or priorities. Tracking metrics also helps ensure that product teams are satisfied and connected to the organization’s broader goals of serving customers.
