Managing Technical Debt
Technical debt is a term coined by Ward Cunningham to describe the cumulative consequences of corners being cut throughout a software project’s design and development. According to James Shore, it is the cumulative total of less than perfect design and implementations in your project. Tom Poppendick defines technical debt as everything that makes your code harder to change.
During the course of a project, the development team cuts corners due to a variety of reasons such as lack of skill, pressure of meeting deadlines or sheer laziness. Over time, these result in accumulating a large backlog of technical inefficiencies that needs to be serviced in future. These issues could be related to code base, the development environment, platform, libraries, design, test coverage, test automation etc – resulting in high defect rates, reduced velocity, poor code maintenance and low productivity. The key to managing technical debt is to be constantly vigilant, avoid using shortcuts, use of simple design and re-factor relentlessly. Unchecked technical debt makes the software more expensive to modify than to re-implement.
Classifying Technical Debt
Martin Fowler’s Technical debt quadrant gives us a clear picture of what Technical debt is.
Figure 1 : Technical Debt Grid Quadrant
This could be used in organizations to decide which sources of debt are acceptable and which are not, and use the information to establish tactics for preventing unacceptable behaviour…
- 1. Prudent & Deliberate
This quadrant is reserved for business drivers that have a compelling ROI for why a product has to ship immediately – i.e. responding to a threat to the business bottom line, or exploiting an opportunity that positively affects the bottom line. Examples where this could be justified could be :
- Entry into new market – first to market may mean that less quality or features with more work-arounds might be acceptable
- Regulatory or Legal Requirement – compliance projects where not meeting dates could have severe financial and operational impact to business
- Peak-period opportunity – within retail, the Christmas period is increasingly becoming the most critical part of the annual cycle and failure to launch prior to Christmas could have a significant negative impact on the company’s performance.
- 2. Reckless & Deliberate
This quadrant reflects poor management where usually corners are being cut in order to hit a deadline that is related to perceived operational needs rather than an underlying clear business case. This is a very common cause of Technical Debt.
- Rushing the project to completion because there’s lots more projects to deliver in the pipeline.
- Pushing the project through because the client wants it on a set date, no one has built a relationship with the client to discuss the details, nor has the client been informed of the affect on quality if the delivery is rushed
- Cutting corners because teams have been incentivized to deliver projects into production by a specified date.
- 3. Reckless & Inadvertent
This happens to typically programmers who are incompetent and unaware of the implications of adding / removing a piece of code – thus incurring a huge technical debt. Technical debt in this quadrant needs to be carefully managed by using processes and tools – through pair programming, code reviews, continuous integration, automated testing etc.
- 4. Prudent & Inadvertent
This is a natural occurrence. Teams would like to improve upon whatever has been done after gaining experience and relevant knowledge.
Short Term and Long Term Debt
Technical debt can also be short term and long term debt. A short term debt is reactive and a tactical measure (pushing a release with known Severity 3 bugs) where as a long term one is a proactive and strategic one (not providing support for a platform which may not exist after 2-3 years). The implication is that short-term debt should be paid off quickly, preferably as part of the next release cycle, whereas long-term debt can be carried on for a few years or longer.
Technical debt in legacy projects
According to James Shore, one of the biggest problems facing legacy projects is excessive technical debt due to which velocity takes a huge hit. Typically in a legacy system, so much work goes into keeping a production system running (i.e., “servicing the debt”) that there is little time left over to add new capabilities to the system. The “bleeding” has to be stopped to prevent more technical debt from occurring.
Besides the use of practices sucgh as Test Driven Development and Continuous Integration, it would be preferable to set apart a certain amount of time as “slack” during each iteration to service technical debt. Though these measures may not be fruitful for the first few iterations, one would be able to see improvements in quality and productivity over a period of time.
Attitudes Toward Technical Debt
Like financial debt, different companies have different philosophies about the usefulness of debt. Some companies want to avoid taking on any debt at all, others see debt as a useful tool and just want to know how to use debt wisely.
Few companies track debt vs. team velocity. Once a team’s velocity begins to drop as a result of servicing its technical debt, the team focuses on reducing its debt until its velocity recovers. Other approaches include tracking rework, and use that as a measure of how much debt a team is accumulating.
Servicing the Technical Debt
The first step in servicing the debt is to identify any such issues and register them – making it visible for the team. These could be made part of the Product backlog or even have a separate Technical Debt backlog for purposes of tracking. The team can then evaluate and prioritize taking into account the effort required and the “pain” caused by the technical debt and its frequency. The Product owner would need to take a conscious decision whether the “economics” justifies the cost of the technical debt It has to be kept in mind that not all technical debt need not be repaid. E,g. one can live with issues relating to a Product nearing its end of its life. However it is important during this time that technical debt is not accumulated since that would only cripple the system
Similar to payment of financial debt, it is prudent to repay the “high interest” technical debt first. It is preferable that technical debt is repaid incrementally – like making a monthly mortage payment, through use of practises such as peer reviews, TDD, refactoring, continuous integration and automated testing.
Steve Garnett in his blog on Strategies for Technical Debt prioritization has an interesting take on Technical Debt Portfolio Prioritization. Based on the degree of business value which the application currently provides against the degree of business value which the application fulfils the future needs of business, he has come out with Technical Debt prioritization quadrant. The remedial action for each of these four situations summarizes how technical debt is to be managed.
It is important that organizations take cognizance of technical debt and find ways and means to manage them efficiently to prevent a system collapse.
- Art of Agile Development – James Shore
- Succeeding with Agile – Mike Cohn
- Managing Technical Debt – Presentation by Kenny Rubin at Mile High Agile 2013, Denver, Colorado – http://www.innolution.com/resources/presentations/managing-technical-debt