2.3 Factors affecting defect "detection"

Mib Software
High Reuse Software
Development and Consulting
We extend and customize
reusable and open software.

Some Implications of Bazaar Size

Third Draft. Aug 11, 1998
Copyright 1997-1998, Forrest J. Cavalier, III Mib Software
All rights reserved. Comments welcome!

2.3 Factors affecting defect "detection"

Assuming that the preceding sequential requirements are met: the code is installed, reachable and it runs, it is quite possible that erroneous behavior goes unnoticed. Not all defects cause core dumps or service unavailability when activated. This is influenced by technical factors, but is also influenced by social/cultural factors (which may be more important.)

Since this step is sequential with defect triggering, it is important to note that the effective working size for this activity is not larger than the preceding step in the defect reporting sequence.

Factors which decrease defect detection

There could be a "benign" defect which computes the correct result, but in a very inefficient manner.
The erroneous behavior may not be critical, perhaps only causing an inappropriate message or formatting in a log file.
System redundancy may mask errors. (For example, a Usenet article which is not propagated by one server due to defective code could be provided by another server at a later time.)
How the defect manifests itself. Is it buried in a long report file that no one reads?
If a program fault causes abrupt program termination, the user will probably notice. But even in some cases of chronic repeated abrupt program termination, the human will not notice: for practical reasons they have installed monitoring "agents" which will automatically restart if a service is not running.
User "apathy"
As long as needs are being met acceptably, a need-driven user will not be diligently inspecting daily reports and error logs looking for erroneous behavior.

Strategies to counter low effective size

Make failures "spectacular" - likely to get attention and abort execution at the first detection of failure, not run for several hours with a corruption that later causes an abort (after the trail is cold.) Someone is sure to notice spectacular failures.
Motivate users to look for defects. (This will be hard to do for "need driven" users, and may result in a lot of "false positives" which will require time to reject.)
Alternate defect detection strategies (See 3. Alternate Debugging methods) including the use of automated test cases.
Remove the human factor. Detect defects automatically: log scanning, verification software running in parallel, etc.

Next: 2.4 Factors affecting defect "reporting"

Up to: 2. Debugging by testing dramatically lowers effective size
Up to: Some Implications of Bazaar Size