Tackling Technical Debt

Technical debt can be defined as the sum of technical work that your team has chosen, either actively or passively, to not do now, and instead, defer until later.

Didn't have time to include unit tests in a feature release? Technical debt. Compile times that have crept from a few seconds to a few minutes? Technical debt. That big list of project dependencies that are approaching end-of-life? Technical debt.

Some technical debt is innocuous, but large amounts of it will almost inevitably start to have some seriously deleterious effects, e.g., codebases becoming harder to maintain, more bugs making it to production, and tests breaking randomly. These debts begin to "accrue interest" to the point where the effort required to fix them can exceed what it would have been six months ago. Therefore, chronic failure to pay down technical debt with some immediacy can result in development being brought to a complete standstill as teams scramble to fix what has become a cascade of compounding issues.

This scenario sounds dramatic, however, technical debt in commercial software is almost always an expected part of the work. Maintaining competitiveness and delivering value to users quickly means that development teams are constantly negotiating time against what's the best and most elegant solution. Plus, the fast pace of the SaaS market and tech in general means that often what might have been the "perfect solution" last year is now irrelevant.

Part of the job of software developers is to act as the brokers of technical debt. They catalogue what needs to be paid promptly and what can wait until later—which includes deciding when exactly "later" will be, and advocating (perhaps sometimes even fighting!) for the time and resources to pay down the debt pile.

Strategies for identifying technical debt

From a product and a user standpoint, one of the particularly insidious qualities of technical debt is that it's often invisible. Even for developers it can be hard to spot—particularly if you've been working with a codebase for a long time!

Here are some strategies to find and track your technical debt load:

Use metrics

It's beyond the scope of this article to cover all of the tools measuring tools for tech debt out there! At a minimum we recommend tracking your software's:
⚠️ Error levels - Are they trending up or down?
📊 Build statistics - How long does it take for your app to build? How big is the resulting bundle? What are the trends?
🌟 App performance - The key metrics will vary depending on the use case of your product.

Use static analysis tooling

Code linters identify common pitfalls and bad practices in your code, and they exist for pretty much every modern language. Type checkers are also available if you're using a typed language. These tools should run as part of your code-deployment pipeline. Similar tools exist specifically for checking security concerns. On the Interfaces here at Float team we’re using a pretty standard suite of analysis tools: ESlint, Prettier, and the TypeScript compiler’s built-in type checker.

Write unit tests and measure your coverage levels

Unit tests verify that a piece of code does what it's supposed to do. A safety net for updating old code by ensuring that any new code is correct, unit tests encourage good practices.💡 Note: Code that's hard to test is often hard to maintain.

Most modern unit testing tools will also check the coverage levels of your codebase. Exactly what percentage of your code should be covered by tests is hotly debated, so it's best to discuss and agree with your team what makes sense for your use case. We've recently doubled down on test coverage in our interfaces team, with our coverage levels incrementally increasing every new cycle.

Be mindful of dependency versions

Pretty much all modern software depends on a lot of external dependencies, e.g., frameworks, libraries, and tools. These dependencies will change as new features are added and bugs are fixed. Neglecting to update a major version of a critical framework can cause chaos down the line! We recommend keeping a list of your key dependencies and defining a specific schedule for important updates. This helps to keep your team on top of these changes and any codebase updates— which is essential for maintaining a quality app.

Leverage new hires

It's easy to become accustomed the problems and deficiencies of your codebase when you've been working in it for some time, e.g., reflexively working around a known pain point. New hires provide a fresh eyes which is great way for pointing things out that don't make sense or that are uncomfortable or inefficient—make sure to ask for their feedback!

Empower a culture of ownership

It's your developers who hold the most knowledge, as they're the ones who are actively working with your technical debt every day! Your team will have the best insights on what the scope of the current debt is, what needs to be tackled first, and what the trade-offs are. Utilize a project management system for developers to log tasks of specific issues that need to be tackled, and incorporate this into your prioritization process. Implement a tag system for tracking tech debt issues, or create a dedicated technical debt project.

Prioritizing technical debt

How technical debt is addressed and prioritized will look different for each team, however, the essential need for maintaining visiblity and awareness is universal. This ensures that your technical debt can be addressed in a structured manner.

By keeping an issues list (in your project management tool), you can assign and track prioritization in close to real time. This helps to keep development leads informed so that they can plan and align how tech debt is addressed in the businesss' engineering or product roadmap.

Here's how we catalogue technical debt at Float to determine what should be prioritized:

• User impact: Our commitment to customers means issues that directly affect the user experience or product usability typically take precedence.
• Business impact: High-impact consequences like customer churn or revenue loss due to technical debt.
• Security concerns: Addresses security vulnerabilities introduced by technical debt. We always seek to prioritize reducing any security risk for the business and customers.
• Interference with shipping: Swiftly addressing technical debt that obstructs new feature development and deployment. This is important for maintaining team agility and competitiveness.
• Quick wins: Some technical debt can be resolved relatively quickly, yielding significant value with minimal disruption.

Another key consideration is our team's capacity and resources. Sometimes we might need specialized skills to resolve specific issues,  or our team might already be fully scheduled on other work. Aligning our technical debt needs with resource capacity informs how we prioritize work and also influence our future hiring plans.

Addressing back end technical debt: A real life example

Imagine a website or app as an iceberg—the part above the water is what you see and interact with (the user interface or front end), while the hidden part below the water is the back end. Back-end services are like the engines that run behind the scenes, handling tasks like storing information, processing requests, and making sure everything works properly. They're like the gears and machinery that make the visible part of the website or app function smoothly, even though you can't see them directly.

That's why addressing technical debt in the back end plays such a critical part in the smooth operation of software systems.

When we kicked off designing and developing a robust notification distribution system, the initial technical approach suggested the creation of individual services. Each would be tied to a repository to support distinct delivery channels like email and Slack. The aim was to achieve scalable autonomy for each channel—determined by factors like user notification preferences and the rate of notification-worthy events.

Given the availability of a boilerplate repository and a corresponding architecture, we opted to adopt this as the foundation for each service. The possibility of encountering code duplication across these services was a consideration, but we consciously accepted this technical debt to expedite the delivery of what should be a solid solution. This approach allowed us to swiftly deliver results while assimilating valuable insights enroute to our subsequent iterations.

As these services evolved independently over time, they presented a set of challenges:

• Code duplication: Maintaining and updating code across multiple repositories led to redundant efforts and introduced potential errors.
• Inconsistent dependencies: The two repositories relied on different versions of common libraries, leading to compatibility issues and complicating maintenance.
• Testing and deployment complexity: Coordinating testing and deployment across dual repositories added intricacy that increased the risk of deployment inconsistencies.
• Version control overhead: Managing separate repositories necessitated overseeing distinct version histories, which as a result, increased the administrative complexity.

As a consequence, we decided to address the accrued technical debt by consolidating the repositories into a monorepo.

Here's how we did it:

• Establishment of a monorepo: We set up a new repository to serve as the monorepo, and selected a monorepo management tool, Turborepo, to facilitate its administration.
• Code refactoring: Through the identification of shared components, utilities, and modules, we restructured the codebase for improved efficiency.
• Dependency standardization: We harmonized dependencies across both projects to mitigate version conflicts and simplify maintenance.
• Unified build and testing pipeline: A consistent build and testing pipeline was established for the monorepo, streamlining the development process.
• Coherent versioning strategy: We embraced a unified versioning strategy and streamlined the release process, ensuring synchronized updates for shared packages.
• Comprehensive documentation: The monorepo's structure, development procedures, and contribution guidelines were documented for transparency and contributor clarity.

We saw positive outcomes immediately from implementing these measures. Maintenance efforts were notably reduced, collaboration became more efficient, testing and deployment procedures were streamlined, and our confidence in the uniform delivery of new features surged. This pivotal shift laid the groundwork for the seamless incorporation of additional delivery channels, accelerating our feature deployment rate.

How to embrace good debt, and prevent bad debt

The first step is to accept that technical debt is an everyday facet in the operations of an engineering team—and you're not alone in navigating this terrain! Secondly, it's important to distinguish between beneficial and detrimental technical debt—both definitions that hinge on the business context and its associated requisites.

Beneficial technical debt is like a strategic loan where you temporarily cut corners to get a product out fast, knowing you'll pay it back later with improvements. It helps you seize opportunities and adapt quickly.

Detrimental technical debt, on the other hand, is like reckless borrowing - you rush without a plan, and the shortcuts create long-term issues.

Engineers need to be able to identify when an excessive accrual of technical debt over time becomes untenable.

A good technical debt prevention strategy can go a long way to future proof the health of your engineering team and the business. And although an intricate endeavor, it is attainable.

A good strategy involves revisiting fundamentals and addressing the issue at a foundational level:

• Revisit the code review process: There might be some gaps in this process that allows code quality to be compromised or code not conforming to standards slipping through the cracks
• Encourage continuous improvement:  Foster a culture of accountability in the pursuit of excellence by empowering developers to take pride in their work. Refactoring should not be misconstrued as weakness, but rather as a testament to seeking continuous improvement.
• Knowledge sharing: Regular knowledge sharing sessions spread awareness of best practices and approaches.
• Education and training: By offering training opportunities for engineers, they can learn more about new technologies, tools and best practices that are likely to reduce the introduction of technical debt and produce quality code

These our top four, however, additional important points to consider in your prevention strategy includes:

• Visibility: Choose transparent tools and systems to track and categorize technical debt. Regularly review and prioritize the identified debt.
• Planning:  Dedicate time for addressing technical debt in your sprint planning. Balancing feature development with debt reduction can prevent debt from piling up.
• Avoid taking shortcuts: Despite pressures to expedite delivery, resist the temptation to compromise code quality through shortcuts. Short-term benefits can lead to long-lasting technical problems.
• Higher management support: Rally support from higher management for underscoring the significance of addressing technical debt. By doing so, you can win the fair allocation of time and resources to manage debt effectively.

In the ever-evolving world of software engineering, addressing technical debt isn't just another chore—it's a strategic move for success. Embrace it with savvy, tackle it head-on, and empower your team to use it as a catalyst for innovation. This will keep the health of your software's technology intact and supercharge your ability to deliver value to customers faster and more effectively.