Why Test Coverage Can Mislead

Long-lived software systems rarely stay tidy. Over time they accumulate decisions, workarounds, and layers of history that can make even simple changes feel risky. For engineers responsible for maintaining those systems, the challenge often becomes less about writing new code and more about understanding what already exists.

In this episode of Maintainable, Robby Russell speaks with Joel Oliveira, Engineering Manager at ezCater, about what helps software remain understandable and adaptable as it evolves.

Joel starts with a principle that often gets overlooked: predictability. When patterns are consistent and code is organized in familiar ways, engineers can navigate a codebase with confidence. Frameworks like Ruby on Rails reinforce this by encouraging shared conventions, making it easier for developers to orient themselves when working in a new application.

The conversation also explores how common metrics can be misleading. Test coverage is often treated as a proxy for quality, but Joel explains that it can create a false sense of confidence. Instead, he values testing most as a thinking tool. Practices like test-driven development can help engineers clarify interfaces and better understand the problem before committing to an implementation.

Joel also shares a story from ezCater about replacing an aging image-cropping service that had become difficult to maintain and required frequent restarts due to a memory leak. Rather than patch the system again, the team introduced ImageProxy, an open source image processing tool created by Evil Martians.

Because the image URLs flowed through a single object in their GraphQL layer, the team could introduce an adapter and gradually route traffic to the new service using feature flags. This allowed them to migrate safely and incrementally instead of relying on a risky “big bang” change.

Robby and Joel also discuss how engineers’ perspectives shift over time. Early in a career it is easy to look at legacy code and label it as bad. Joel now sees older systems as layers of decisions shaped by real constraints. Approaching them with empathy makes it easier to improve them thoughtfully.

The episode closes with advice for engineers maintaining complex systems: frame problems as opportunities. By documenting impact and proposing incremental improvements, teams can steadily move their software toward a healthier future.

Maintainable software rarely comes from one heroic refactor. More often, it’s the result of many small improvements made by teams who understand their systems and care about how they evolve.

Episode Highlights

00:02:18 – Predictability as a Maintainability Feature
Joel explains why predictable patterns and conventions make large codebases easier to navigate.

00:07:41 – When Test Coverage Misleads
Why high coverage can give a false sense of quality.

00:12:05 – Consulting vs. Product Engineering
How switching environments shaped Joel’s approach to code.

00:16:32 – Replacing a Legacy Image Service
ezCater’s migration away from a failing Node-based image service.

00:21:14 – Migrating with Adapters and Feature Flags
How the team gradually moved traffic to ImageProxy.

00:26:03 – Developing Empathy for Legacy Code
Why older systems deserve understanding, not blame.

00:30:47 – The Shift to Engineering Management
Joel reflects on moving from IC work to leading teams.

00:34:52 – Advice for Improving Complex Systems
Small, consistent improvements matter more than big rewrites.

Thanks to Our Sponsor!

Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks.

It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications.

Keep your coding cool and error-free, one line at a time! 

Use the code maintainable to get a 10% discount for your first year. Check them out!

Links / References

• Joel Oliveira — LinkedIn
• Joel Oliveira — Website
• Joel Oliveira — Mastodon (@jayroh)
• ezCater
• ImageProxy

Subscribe to Maintainable on:

• Apple Podcasts
• Spotify

Or search "Maintainable" wherever you stream your podcasts.

Keep up to date with the Maintainable Podcast by joining the newsletter.