Dispelling Common Code Coverage Myths and Dodging Pitfalls
Posted on Thu, Aug 11, 2011
Your tests ensure the quality of your code, but how do you determine the quality of your tests?
These days, a comprehensive suite of unit tests is a standard component of pretty much every professional software development project. Developers rely on unit tests to assess the threat of bugs in the code. A test suite that passes without any failures is generally seen as an indicator of high-quality code and properly-functioning software. But how much of your code is really tested by your test suite? And how can you be sure that there are no significant gaps?
A popular measure for test suite quality is the the code coverage achieved by tests. In simplified terms, this is the overall percentage of code that is exercised by tests. Many project teams don’t know the actual code coverage that their test suites achieve. Often, there is a good deal of surprise when a development team is confronted with the actual code coverage numbers that were measured for their tests. Also, many developers are unaware that even when a code coverage tool reports 100% code coverage and all tests are passing (and supposedly correct), there can still be undiscovered bugs in the code.
To effectively gauge the quality of your test suite and the code being tested, it is essential to understand the various code coverage criteria and the different levels of trust they should inspire.
The next few blog entries introduce common pitfalls and misconceptions regarding code coverage granularity, code coverage criteria, code coverage density, and test overlap, then explain what you need to know to choose the best code coverage metric for your goals. We'll be using Java code examples to illustrate these code coverage concepts, but most of the presented ideas apply just as well to other programming languages.
It's important to note that code coverage is just one aspect of software quality. A test suite might fully cover the tested code but inadvertently assert incorrect behavior. This series of posts assumes that all test cases are accurate and assert correct behavior.
***
To explore code coverage white papers, articles, and videos/webinars, visit our Code Coverage Resource Center. Or, visit Parasoft's Code Coverage Analysis page.
Image credit: Steelparadox