ALM Best Practices

Code Coverage Granularity

  

 Email Article  

Tweet  

  

Code coverage granularity is an important aspect of automated tools that measure code coverage metrics for testing. To determine the supported level of  granularity, ask yourself “What is the smallest unit of code whose coverage status I can determine unambiguously?”

For many available tools, the granularity is limited to individual lines. This can be a bigger limitation than it seems at first glance. The key problem is that Java, like most popular programming languages, is a free-form language. A developer may write a complete Java class as one very long line of code. If a coverage tool that offers a line-based coverage granularity reports that this one-liner class is “covered,” what exactly does that mean? Does it mean that every single expression in that line was covered, or is 100% coverage reported if at least one expression in the line was covered?

Admittedly, the example of a Java class that is written as a single line of code is somewhat contrived, and such a programming style would already be a cause for concern. However, line-based granularity can reach its limits for some pretty common idioms. Consider the method in Listing 1.

public class Listing1
{
    static int minOrMax(boolean minimum, int a, int b)
    {
        return minimum? Math.min(a, b):Math.max(a, b);
    }
}

Listing 1: A class demonstrating the limits of line-based coverage granularity

A single test is necessary to partially cover the line containing the return statement, but at least two test cases are needed to cover both alternatives of the conditional expression. Even without using the conditional operator, it is not difficult to create situations where a line of code is only partially covered. Exceptions during program execution can always leave parts of a line without coverage. Line-based coverage granularity is sufficient for most cases—as long as the coverage tool does not report complete coverage for lines that are, in reality, only partially covered. Also, it may be tricky to determine why a particular line is not fully covered and what tests need to be added to achieve full coverage.

Coverage tools that report coverage for individual expressions make it easier to identify missing test cases. Visualization of expression-based coverage granularity is a little more intrusive. Line-based coverage can easily be displayed in a side ruler of the source editor, whereas expression-based coverage requires markers (like coloring or underlining) in the source code itself.

In rare cases, a seemingly atomic expression translates to multiple bytecode instructions—some of which may not get covered under certain circumstances. Ideally, coverage granularity reaches down to the level of individual bytecode instructions. In fact, many coverage tools collect information about whether or not individual bytecode instructions are executed. However, the granularity of the collected data may be reduced to expression-based or line-based coverage when reported to the user.

***

To explore code coverage white papers, articles, and videos/webinars, visit our Code Coverage Resource Center. Or, visit Parasoft's Code Coverage Analysis page.

Image credit: NeilsPhotography