////

    Copyright © 2019 Christian Kaltepoth

    This program and the accompanying materials are made available under the
    terms of the Eclipse Public License v. 2.0, which is available at
    http://www.eclipse.org/legal/epl-2.0.

    This Source Code may also be made available under the following Secondary
    Licenses when the conditions for such availability set forth in the
    Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
    version 2 with the GNU Classpath Exception, which is available at
    https://www.gnu.org/software/classpath/license.html.

    SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0

////
== Coverage report

A specification can be distilled into a collection of assertions that define the behavior of the software.
This section introduces the MVC 1.0 TCK coverage report, which documents the relationship between the assertions
that have been identified in the MVC 1.0 specification document and the tests in the TCK test suite.

The structure of this report is controlled by the assertion document, so we’ll start there.

=== MVC 1.0 TCK Assertions

The MVC 1.0 TCK developers have analyzed the MVC 1.0 specification document and identified the assertions that
are present in each chapter.

The assertions are listed in an XML file which is generated from the specification document.
Each assertion is identified by the section of the specification document in which it resides and
a unique paragraph identifier to narrow down the location of the assertion further.

See the following example of an example:

[source,xml]
----
<section id="controllers" title="Controllers" level="2">
    <!-- ... -->
    <assertion id="ctrl-method">
        <text>An MVC controller is a JAX-RS resource method decorated by @Controller</text>
    </assertion>
    <!-- ... -->
</section>
----

The strategy of the MVC 1.0 TCK is to write a test which validates this assertion when run against an implementation.
A test case (a method annotated with `@Test`) is correlated with an assertion using the
`@org.jboss.test.audit.annotations.SpecAssertion` annotation as follows:

[source,java]
----
@RunWith(Arquillian.class)
@SpecVersion(spec = "mvc", version = "1.0")
public class ControllerAnnotationTest {

    /* more tests */

    @Test
    @SpecAssertion(section = Sections.MVC_CONTROLLERS, id = "ctrl-method")
    public void controllerMethod() throws IOException {
        // test implementation
    }

    /* more tests */

}
----

To help evaluate the distribution of coverage for these assertions, the TCK provides a detailed coverage report.
This report is also useful to help implementors match tests with the language in the specification that supports the behavior being tested.


=== The Coverage Report

The coverage report is an HTML report generated as part of the TCK project build.
Specifically, it is generated by an annotation processor that attaches to the compilation of the classes in the TCK test suite.
The report is written to the directory `tests/target/coverage-report/`.

The report itself has three main sections:

Chapter Summary::
List the chapters in the specification document along with total assertions, tests and coverage percentage.

Section Summary::
Lists the sections in the specification document along with total assertions, tests and coverage percentage.

Coverage Detail::
Each assertion and the test that covers it, if any.

The coverage report is color coded to indicate the status of an assertion, or group of assertions.
The status codes are as follows:

Covered::
Test exists for this assertion

Not covered::
No test exists for this assertion

Unimplemented::
A test exists, but is unimplemented

Untestable::
the assertion has been deemed untestable, a note, explaining why, is normally provided

For reasons provided in the `tck-audit.xml` document and presented in the coverage report, some assertions are not testable.
