What’s the point of code coverage?

2011-08-25

Testing has to satisfy two objectives: it has to be effective, and it has to be cost-effective. Testing is effective if it can distinguish a correct product from one that is incorrect. Testing is cost-effective if it can achieve all it needs to do at the lowest cost (which usually means the fewest tests, least amount of effort and shortest amount of time).

Measuring code coverage during testing can help to meet these two objectives by:

Assessing effectiveness by demonstrating that the tests exercise all relevant features of the code during testing. Example features are branches, statements, function calls or decisions.
Improving cost-effectiveness by allowing the developers of tests to determine whether any tests are redundant, in other words where multiple tests exercise the same features without demonstrating anything new.

Clearly there is a balance between cost-effectiveness and effectiveness of testing. Both can be improved to a certain point, after which there is a trade-off. Different coverage criteria allow this balance to deliver a level of rigor in the testing that is appropriate for the level of reliability, or the software integrity level (SIL) of the system.