Conversations around metrics

Most people and teams view metrics on any project team as a measure of productivity. This is the reason why metrics are rarely spoken about in most teams (agile or otherwise). Think of the last time your team metrics were spoken about in your retrospective.

Let's just clarify something at the beginning of this post. "Productivity" isn't a measure of the team's success; instead, it is delivering excellent value to the stakeholders is. If a team is super productive and is delivering many stories every sprint, but isn't delivering value, I would say that team needs to take a step back and look at what needs to change.

Metrics is the start of a conversation. It is not something that needs to go in a report to be analyzed by a manager. It is instead a starting point for a discussion a team needs to have.

This means that the agile team must be actively collecting and talking about these metrics regularly. It should be a part of their regular daily lives, and a part of all of their agile meetings as well.

How to discuss a metric:

Metrics are not something that you put in a presentation of a google sheet, but it is something that you then discuss. For example, story cycle times measure how long it takes for a story to move from 'Start work' to 'Done.' Now, these metrics can give the team great insights into what is happening daily. If the average cycle time of the team suddenly increases, that is something to discuss in the team retrospective. If you notice a developer(or pair) working on a story is taking quite some time to get the story done, maybe it is time to talk about what is happening and the blockers the developer is facing. You cannot have that conversation unless you are actively tracking and talking about cycle time.

Here are some of the agile metrics, and how they can be used in your teams' day-to-day functions;

Velocity: This is probably a metric every agile team tracks pretty regularly. Basically, at the end of a sprint, add up the number of points that the team is done with and that is your velocity.

However, in most agile teams, velocity seems to be used as a measure of performance and how "productive" a team is in a sprint. But in most cases, the team's velocity has very little to do with the team's capability and focus. Many reasons contribute to a team's velocity, and it is essential to understand those reasons.

The main goal of velocity is to help teams understand and benchmark for planning future sprints. How many points can we expect for the next sprint? That is the central question that the team's average velocity can answer.

Velocity also has nothing to do with the quality of the software or the business value the team is creating. That is why just speaking about velocity, all on its own, isn't the right conversation.

For example, if the team has an average velocity of 30, in the past three sprints, and the current sprint, the team got 26 points done, that is a pretty standard deviation in velocity, and I probably would not read to much into it.

However, if the team's current sprint velocity is 18, that means something has changed, and it is essential for us to discuss what that is. Did the team have a lot of sick days? Were they waiting on assets from the design team? Did they discover a series of production bugs that needed their immediate attention? These are conversations that you need to have with your team.

Recidivism: This is the rate at which stories move back into development after they have been moved into QA, usually due to some QA tests failing. This can provide great insights into your team's processes. How many times in the sprint, was a story moved back into development? Did that hamper the team's rhythm or cause frustration? How can we bring this metric down in the next sprint? Could we have a quick dev check before moving a story? Could we have more conversations with the testers? All great conversations to come out of this metric.

Defect count: Number of bugs per sprint. This is great to track alongside velocity. If the team's velocity is increasing, but their bug count is increasing too, that means they need to slow down and focus on quality instead of just getting a story done. Defect count can lead to other great questions also. Is there a part of the code for which the defects are high? Maybe we need to refactor there? Or has the defect count suddenly dropped? What changed?

Story completion ratio: This is the ratio of stories completed in a sprint v/s the stories that were picked up in the sprint. If this ration is 7/10, that means of the 10 stories picked up; only 7 were completed in the sprint. This metric can provide great insights to the team. Were there any blockers that the team faced? Did we underestimate a story? If so, why? Are the stories sizes too large? Do we need to start breaking them up into smaller chunks? All right questions to be discussed in a retrospective or regular team conversation!

Note: I just wanted to throw in here that it is crucial NOT to attack the developer here who has not finished that story. It is more important to address the problem that the developer might be facing.

Time blocked per work item: I always encourage my team to move a story card into the 'Blocked' swim lane when blocked. This gives us the amount of time each story is blocked. We can later analyze this by figuring out what caused these blockers? How can we avoid such blockers in the future? Again, you don't want to blame a particular person for causing those blockers. Instead, you want to come together as a team to understand how the blocker happened and what you can do to avoid such blockers in the future.

For example, it is very easy for the team to blame a designer for a blocker. Maybe they were waiting on a design for a screen. Instead of doing this, though, let's come together as a team to figure out how we can avoid this happening in the future.

Story cycle time: This is the amount of time it takes for a story to move from 'Start work' to 'Done.' Ideally, the average cycle time on a team should be somewhere between a day and one half of a sprint. If that is not the case, we want to be examining why stories are taking longer than a few days to get done? Should we be splitting the stories into smaller chunks? Do we have an element of the code that requires maintenance? What can we do to reduce this cycle time is a great conversation to have with your team.

Test coverage: A much discussed and controversial topic defines the percentage of your code covered by automated tests.

What the test coverage does not tell you is how functional those tests are? Have the developers just written these tests to get the coverage up? This isn't to say that test coverage isn't an important metric. It's just that test coverage should not be looked at all on its own. You want to look at how many of those tests are failing in every build. How long is it taking for people to update and maintain the test suite? How are these automated tests run? Is there a test blind spot where an entire code area isn't tested at all? All necessary conversations to have around your team's test suite.

These are just a few of the examples of the metrics you can use to connect and understand your team better and to figure out ways to improve.

Remember, metrics are not something that just sits in a spreadsheet to be analyzed by a manager. It is the beginning of useful and vital discussions with everyone on the team.

I hope you have found this useful. Please let me know if there are any questions.