Case Study: Saving Time With Testing

Introduction

A client of mine realized his team had hit a plateau and needed to shake things up. This was to hopefully spark further improvements to cycle time. The question he had was, "Will test-driven-development help?"

Background

A growing company in the private security space built an engineering team that realized that if they could ship fast enough, they could respond to any change and issue. Benchmarks against DORA showed they were already high-performing in almost every way.

My client wanted to see that improve even further. They asked if teaching the team test-driven development would reduce their cycle time even more. I said that it would. So with the goal of reducing cycle time, I taught the team test-driven development.

Approach

The approach was designed to take approximately six months. This was due to the fact that an intrusive change like TDD often needs a certain amount of established credibility and rapport. Building rapport would be the first three months, and execution the last three.

The method to introduce TDD focused on a workshop followed by one-on-one pairing to help people approach the technique in their actual work instead of a classroom.

At the same time, measurements were collected and showed throughout the process to demonstrate the impact to cycle time and address the hypothesis that, "Testing takes too long."

Results

After the initial six-months, there was a 20% drop in cycle time. More than that, approximately half of the engineering team believed that TDD improved their code. Half were unsure. This split was wider than I expected.

As a part of tracking metrics, I tracked what I call a Defect Rate. This is a ratio of work completed to bugs and defects created. The group started at 60% and dropped to 40% within three months. This result was the most surprising. Mainly because they were unaware how much waste they had when 60% of the time they were creating more work for themselves as bugs.

Benchmarks at other clients ranged from 80%-120%. This means that every time someone completed a story or task, there was that percent chance of creating a bug or defect.

Conclusion

My client and I agreed that while we had the desired impact, there was more to do. I stayed with the team longer to help more of the team with TDD to give it the best shot it had. There were still numerous debates within the organization, but the impact of TDD on cycle time and quality was established firmly.