Technical Debt: A Practical Approach

Today, I'm gonna step away from the Amazon tutorials, I think I have spent enough words on it, so it's time to move to other areas, and as you can see from the title, this post is going to be all about technical debt.

For the last couple of years, this topic has been a rollercoaster (take a look at the google trends chart here), but rather than just writing about what is it, as many people already do, I will write about how it can be combined in a regular scrum development process. 

What's Technical Debt?

Before actually diving into the main topic, I need to give a little bit of background. If you are not familiar with the concept, this section might interest you, otherwise, it's still a good read.

The term technical debt is not something that started appearing in the web recently, nor is something I created. It was coined by Ward Cunningham around 1992 in his article titled "The WyCash portfolio management system" [1]. 

If I need to summarize the definition, I would say that it's the result of any decision that you make in software development that has a visible result and a secondary not-so-visible effect.

In the financial counterpart, any debt generates an interest that makes the debt increase over time. At some point, you might pay the debt, but not after incurring in a lot of expenses. In this scenario, the decision was to ask for a loan, and the secondary effect was the interest that made you pay more and more over time.

If we come back to software development, a perfect example is when you are comparing two solutions, the first one is a quick and dirty solution, and the other one takes more time but provide a cleaner result. If you choose the first one because it reduces the time to publish a product, or there is a hard deadline, then you are starting a loan and the interest starts to run. The interest can appear in many forms, for example, it could be the "Spaguetti code" that is created, or the high complexity of a method that will require extra time when doing any support activity, among other possible outcomes.

Incurring in technical debt is not bad by its definition, the bad part is when the debt is not handled properly. So, how can we handle it?

Managing Technical Debt

In 2014, a group of researchers performed a simulation to determine what's the best strategy to manage technical debt in an agile environment. They came up with a few strategies and decided that the best approach is to list the technical aspects that are most important to the developers, monitor those aspects using automated tools and remediate any failure based on thresholds limits. Sounds like a lot, I know, and that's why they explicitly state that, even though this is the best approach, it would require resources and time that are not always available. If you want to read the full article, you can purchase it from here or download it if you have an IEEE account.

Last year, to complete my master's degree, I had to lead a project and decided to focus it on managing technical debt. I already had a strategy in place, based on what these researchers simulated, I just had to prove that it works.

So, I started by doing some research to see if there were other efforts of people doing the same as me and chose a static code analysis tool among the ones in the market. We opted to use SonarQube to execute that part in a continuous way, and created some custom alerts whenever a metric failed based on some threshold limits. Sonarqube, in my opinion, leads the market not only because it is free and open source, but because the number of functionalities and languages t supports exceed any of its competitors.

After that, we altered our development process. Added weekly code review meetings to check the progress on the technical debt metrics, allocated time (no higher than 20% of 1 resource per week) to remediate failing metrics and established a more standardized code review strategy. 

We evaluated these changes during 5 sprints of 1 week, where we added new features and worked on remediating failing metrics. At the end of those sprints, we obtained the following results:

  1. Defects reduced in 30%, according to Sonarqube.
  2. Days of technical debt reduced in 25%.
  3. Security vulnerabilities reduced in 86%.
  4. Reduced cyclomatic complexity in 129 possible flows accross the application, resulting in simpler classes and functions.

Also, 4 months after the changes were applied, we noticed a reduction in the number of support tickets by 60% and the resolution time was improved in 50%, saving time in support activities and reducing costs. 

Even though the whole project took other areas into consideration, like process engineering and automated testing, we ended up adding technical debt management activities to our development process effectively. So, if you want to read the complete document about this project, in spanish although, it's publicly available from here.

If you reached here, I hope you enjoyed this post and if you are interested in applying something like this in your organization, don't hesitate in contacting me for more information.