Code coverage

Author: Kasper B. Graversen, 05/12/07 Keywords: ANT, Build script Abstract: This article introduces the term "code coverage" and show how you will get started using the free code coverage tool "Emma".

subscribe to my RSS feed

Code coverage

1. Testing should be a terse specification of the behavioral requirements of the code you are about to write.
2. Testing should be applied in the dosage that enables you to feel confident of the code you have been producing.
3. Testing should act as a form of documentation on how to use a piece of code.

Why not 100% code coverage

• Accessor methods / value objects.
• Overloaded constructors
• User interface code

Code coverage with EMMA and ANT

• Setup paths for the code coverage report, the destination of the instrumented bytecode files and finally where Emma is installed.
• Then emma.filter is set to denote the classes to cover (in this case org.supercsv)
• Build all the classes with debug options into the coverage dir
• Run the emma instrumentation
• Run JUnit
• Generate the code coverage report

    <target name="testcoverage" depends="-init_test">
         <!-- directory which emma coverage reports will be written to -->
        <property name="dir.coveragereport" value="${basedir}/web/emmareports/" />
        <mkdir dir="${dir.coveragereport}"/>
        <delete file="${dir.coveragereport}/coverage.emma" />
        <delete file="${dir.coveragereport}/metadata.emma" />
        <!-- directory that contains emma.jar and emma_ant.jar -->
        <property name="emma.dir" value="${dir.lib}" />

        <!-- Set emma.lib to refer to the list of EMMA jar files -->
        <path id="emma.lib" >
            <fileset dir="${emma.dir}">
                <include name="*.jar"/>
            </fileset>
        </path>

        <!-- Load <emma> and <emmajava> custom tasks so that they can be used in ANT -->
        <taskdef resource="emma_ant.properties" classpathref="emma.lib" />
        <property name="emma.enabled" value="true" />


        <!-- this property, if overriden via -Demma.filter=<list of filter specs>
         on ANT's command line, will set the coverage filter; by default,
         all classes found in 'emma.coverage.classes' pathref will be instrumented:
        -->
        <!-- must use filtering -->
        <property name="emma.filter" value="org.*" />

        <!-- compile files -->
        <antcall target="-compile"/>


        <!-- Define which classes will be reported in the coverage reports (by default, we will include all classes and assume -->
        <!-- that your project's output folder is target/classes -->
        <path id="emma.coverage.classes" >
            <pathelement location="${dir.build}" />
        </path>

        <!-- EMMA INSTRUMENTATION -->
        <!-- Instrument the classes, this takes the normal classes, and makes instrumented classes instead -->
        <emma enabled="${emma.enabled}" >
            <instr instrpathref="emma.coverage.classes"
                 metadatafile="${dir.coveragereport}/metadata.emma"
                 merge="true"
                mode="overwrite" verbosity="verbose"
            >
                 <!-- exclude classes with a "Test" in the name: -->
                 <filter excludes="*Test" />
                 <filter value="${emma.filter}" />
            </instr>
        </emma>

        <!-- before test run, copy test files and compile them -->
        <antcall target="-compile_tests"/>

        <!-- run test -->
        <junit haltonfailure="true" printsummary="true" fork="true" forkmode="once">
            <batchtest>
                <fileset dir="${dir.build}" includes="**/*Test.class"/>
            </batchtest>
            <formatter type="brief" usefile="false" />
            <classpath>
                <path refid="classpath" />
                <pathelement location="${dir.build}" />
            </classpath>

            <jvmarg value="-Demma.coverage.out.file=${dir.coveragereport}coverage.emma" />
            <jvmarg value="-Demma.coverage.out.merge=false" />
        </junit>


        <!-- generate test coverage report -->
        <!-- if enabled, generate coverage report(s): -->
        <emma enabled="${emma.enabled}" verbosity="verbose">
            <report
                  sort="+name,+block,+method,+class"
                  metrics="method:80,block:80,line:80,class:100"
            >
                <sourcepath >
                      <dirset dir="${dir.tmp_src}" >
                      </dirset>
                </sourcepath>

            <!-- collect all EMMA data dumps (metadata and runtime)
                 [this can be done via nested <fileset> fileset elements
                 or <file> elements pointing to a single file]:
            -->
            <fileset dir="${dir.coveragereport}" >
                <include name="*.emma" />
            </fileset>

            <!-- for every type of report desired, configure a nested
                 element; various report parameters
                 can be inherited from the parent <report>
                 and individually overridden for each report type:
            -->
                <html outfile="${dir.coveragereport}/index.html"
                depth="method"
                columns="name,class,method,block,line"
            />
            </report>
        </emma>

        <antcall target="-post_clean"/>
    </target>

Aftermath

• It does not record your test-coverage over time. Hence you cannot automatically trace general improvements of your test coverage
• There is no instantanious feedback in your development tool. Instead you have to run this script and consult your browser for a report. For Eclipse, there is a nice free tool (based on EMMA) called ???.
• There is no integration with ANT such that the coverage-task fails if the coverage is too low. This would be useful in ensuring a company test-coverage policy. However, too strict rules may easily get more in the way than do good. Another tool Cubertura has such a feature, but seems (at present) to be lacking block coverage.

Comments

Help spread the word