�Trade-offs between Productivity and Quality in Selecting Software Development Practices�

http://www.pitt.edu/~ckemerer/CK%20research%20papers/TradeoffProductivityQuality_MacCormackKemererCusumanoCrandall03.pdf

p84 paragraph beginning “Overall, our results suggest…”

On the other hand, the authors start their conclusions with, “Our experiences and distilled lessons learned, all point to the fact that TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team.” Which sounds like an overall win.

steve.freeman > but, again, I don’t see the case for it being a trade-off

Given the authors state “only one practice—releasing an early prototype during development—was associated with both higher productivity and a lower defect rate” perhaps you should read the paper again.

“Integration or regression testing at check-in” DID NOT contribute to higher productivity.
“Integration or regression testing at check-in” DID contribute to lower defect rate.

There’s also another level of value that most of the studies don’t take into account which is the extent to which writing a test first encourages people to think about what they really want, rather than building in speculative or cool ideas.

If the initial value to the business is greater than the cost of the 15-35% increase in development time, then this is the real pressure forcing management and thereby IT teams to focus on delivery and thus push TDD out of the picture? ie: Business are happy to pay the higher defect cost, because they get value earlier (assuming the initial user acceptance testing phase meets their initial expectations).

I don’t have the full second paper but, again, I don’t see the case for it being a trade-off.

It’s a trade-off.

“Trade-offs between Productivity and Quality in Selecting Software Development Practices”

Also, see “Product-Development Practices That Work: How Internet Companies Build Software”

Like Agent Mulder, I want to believe – I really do – but I’m afraid my search for statistically compelling evidence continues…

My suspicion is there would be a substantial divergence between subjective increased development time and actual development time. There are three factors I’d expect to cause overestimates:

First, people notice the new thing more. Second, they’d have a hard time noticing the time spent not debugging. And third, software projects are always under time pressure, so many managers habitually find and presents reasons why the team isn’t going as fast as people want.

The decrease in defect rates matches my experience, though. For one team where we mainly coached them on TDD, we cut bug rates by 50% on the first project, and by 50% again on the second project. For an experienced team doing full XP, I’d expect 90%+.