I started writing a reply to this post but it got kind of long, by the time I started thinking about adding pictures it dawned on me that it’s probably a good idea to write my own post on the subject.

The discussion here is based on an idea from lean development. Traditionally people thought that quality had a price. Removing defects has a cost so more quality means higher cost. Lean production tells us this is not true. If you remove the defects sooner you can remove them cheaper. By removing the cause of the defects you can prevent them from being added. This will not only improve the quality but actually remove the cost of fixing the defects. Agile methodologies like XP show us that this works for software too. Up to a point.

There are a couple of ways you can prevent defects from appearing in your software. The two most efective things you can do are TDD or test first development and pair programming. Both give you early feedback about defects and prevent you from introducing them. Both have their own types of defects they will prevent. Test first programming seems suited for removing defects related to low level behaviour of components in your software. Pair programming has a more wide range of defects it will prevent, having two pairs of eyes and two brains look at the same piece of code. I’ve found that most errors it prevents are maintenance related. High level interaction problems are usually too complex to find this way.

The biggest improvement in quality you will get from these methods is not so obvious though. I’ve found both methods force you to think about low level specifications before implementing them. Writing a test forces you to think about how the code you’re testing is going to behave. You’ll have to specify this behaviour in your test before implementing it. Writing down executable specifications like this is very powerful. I’ve found the dynamics of TDD don’t feel like testing at all. It focusses the development effort but TDD will not replace testing. Simply because you won’t find all classes of errors.

I’ve found two places where developer testing breaks down.

The most obvious place is in testing your requirements. Requirements are notoriously imprecise. Problems in writing down the requirements or in interpreting the requirements will often only be found in an aditional inspection phase. Software might be working correctly but is it actually doing the right thing. Usability issues also fall in this category.

Integration of components is also a source of defects that won’t be found with TDD or pair programming. In simple cases where components just call each other integration issues are often found by the compiler. But in more complex cases, for example when components should work together in a multithreaded application defects are almost inevitable.

I don’t think you can elliminate inspection. Software is just too complex. I don’t think zero defects is achievable either. But when looking at common development practices today I think there is a lot of low hanging fruit. Most development organizations can indeed save on inspection and defect-removal by increasing quality using these practices. But even then they’ll be a long way from zero defects.

The original fallacy was to think the cost of removing defects was a straight line that went up. But assuming the cost per defect is a straight line with a downward slope is too simplistic too. Here’s the picture I promised:

Quality will start to cost money when all the easy bugs are gone. But until then we should invest in it as much as we can.