[[configuration]]
== Running the TCK test suite

This chapter lays out how to run and configure the TCK harness against
a given Jakarta Validation provider in a given Jakarta EE container. If you have
not by now made yourself familiar with the https://docs.jboss.org/author/display/ARQ/Reference+Guide[Arquillian
documentation], this is a good time to do it. It will give you a
deeper understanding of the different parts described in the following
sections.

=== Setup examples

The TCK distribution comes with a directory
`setup-examples` which contains an example maven project
for running the TCK. If you followed the instructions in <<installation>> you find the directory under
`validation-tck-dist-{tck-version}/setup-examples/maven`. The example is using
Hibernate Validator as Jakarta Validation Provider and Eclipse GlassFish 7.0+ as the Jakarta EE
constainer.

The example comes with a `readme.md` containing
the prerequisites for using this setup, how to run the TCK against
Hibernate Validator and Eclipse GlassFish. The readme in
`setup-examples` itself contains information about what
needs to be changed to use a different Jakarta Validation provider and Jakarta EE
container.

The following chapters contain some more information about the
general structure of the TCK which will give you a deeper understanding
above the simple readme files.

=== Configuring TestNG to execute the TCK

The Jakarta Validation test harness is built atop TestNG, and it is
TestNG that is responsible for selecting the tests to execute, the order
of execution, and reporting the results. Detailed TestNG documentation can
be found at http://testng.org/doc/documentation-main.html[testng.org].

The `tck-tests.xml` artifact provided in the TCK
distribution must be run by TestNG (described by the TestNG documentation
as "with a `testng.xml` file") unmodified for an
implementation to pass the TCK. For testing purposes it is of course ok to
modify the file (see also the TestNG http://testng.org/doc/documentation-main.html#testng-xml[documentation])

[source,xml]
----
<suite name="Jakarta-Validation-TCK" verbose="1">
    <test name="Jakarta-Validation-TCK">

        <method-selectors>
            <method-selector>
                <selector-class name="org.hibernate.beanvalidation.tck.util.IntegrationTestsMethodSelector"/>
            </method-selector>
        </method-selectors>

        <packages>
            <package name="org.hibernate.beanvalidation.tck.tests"/>
        </packages>
    </test>
</suite>
----

TestNG provides extensive reporting information. Depending on the
build tool or IDE you use, the reporting will take a different format.
Please consult the TestNG documentation and the tool documentation for
more information.

=== Selecting the `ValidationProvider`

The most important configuration you have make in order to run the
Jakarta Validation TCK is to specify your
`ValidationProvider` you want to run your tests
against. To do so you need to set the Java system property
`validation.provider` to the fully specified class name
of your `ValidationProvider`. In Maven this is done
via the `systemProperties` configuration option of the
maven-surefire-plugin. This
system property will be picked up by
`org.hibernate.beanvalidation.tck.util.TestUtil`
which will instantiate the `Validator` under test.
This means the test harness does not rely on the service provider
mechanism to instantiate the Jakarta Validation provider under test, partly
because this selection mechanism is under test as well.

[[configuration-deployable-container]]
=== Selecting the `DeployableContainer`

After setting the `ValidationProvider` you have
to make a choice on the right `DeployableContainer`.
Arquillian picks which container it is going to use to deploy the test
archive and negotiate test execution using Java's service provider
mechanism. Concretely Arquillian is looking for an implementation of the
`DeployableContainer` SPI on the classpath. The setup
examples use a remote Eclipse GlassFish container adapter, which means that
Arquillian tries to deploy the test artifacts onto a specified remote
Eclipse GlassFish instance, run the tests remotely and report the results back to
the current JVM. The installation directory of the remote container is
specified via the `container.home` property in the
example build files.
To make it easier to develop, debug or test the TCK, an in JVM
adapter is provided as part of the distribution
(`beanvalidation-standalone-container-adapter-{tckVersion}.jar`).
Using this adapter the tests are not executed in a remote Jakarta EE
container, but in the current JVM. This allows for easy and fast
debugging. Some tests, however, are only runnable in a Jakarta EE container
and will fail in this in JVM execution mode. By setting the property
`excludeIntegrationTests` to
true these tests can be excluded.

The adapter is also available as Maven artifact under the GAV
`org.hibernate.beanvalidation.tck:beanvalidation-standalone-container-adapter:{tckVersion}.`
You can refer to `pom-local.xml` in the setup-examples/maven module
of Hibernate Validator TCK (in the directory
`validation-tck/setup-examples/maven`, if you followed the
instruction in <<installation>>) to see how it is used.

[[configuration-arquillian-xml]]
=== arquillian.xml

The next piece in the configuration puzzle is
`arquillian.xml`. This xml file needs to be in the root
of the classpath and is used to pass additional options to the selected
container. Let's look at an example:

[source, xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
        http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

    <engine>
        <property name="deploymentExportPath">target/artifacts</property>
    </engine>

    <!-- Run tests in the GlassFish container -->
    <container qualifier="incontainer">
        <protocol type="Servlet 5.0" />
        <configuration>
            <property name="glassFishHome">${container.home}</property>
            <property name="adminHost">localhost</property>
            <property name="adminPort">4848</property>
            <property name="debug">true</property>
            <property name="outputToConsole">true</property>
        </configuration>
    </container>

    <container qualifier="remote">
        <protocol type="Servlet 5.0" />
        <configuration>
            <property name="adminHost">localhost</property>
            <property name="adminPort">4848</property>
            <property name="debugRequests">false</property>
        </configuration>
    </container>

    <!-- Run tests locally -->
    <container qualifier="local">
        <protocol type="Local" />
        <configuration />
    </container>
</arquillian>

----

The most important container configuration option is the protocol
type which determines how Arquillian communicates with the selected
container. The most popular types are `Servlet 5.0` and
`Local`. The former is used when connecting to a remote
container whereas the latter is used for the in JVM mode.

Another interesting property is
`deploymentExportPath` which is optional and instructs
Arquillian to dump the test artifacts to the specified directory on disk.
Inspection of the deployed artifacts can be very useful when debugging
test failures.

[[configuration-javafx]]

The Jakarta Validation specification mandates a support of JavaFX if JavaFX is available in the classpath.

While JavaFX is included in the Oracle JDK and some other JDKs include https://wiki.openjdk.java.net/display/OpenJFX/Main[OpenJFX],
it might not be included in all JDKs.

Having JavaFX available in a container environment might also not be straightforward.

For these reasons, the JavaFX tests are disabled when running the TCK with the default options.

It is highly recommended to run the TCK in at least one configuration that allows to test the support of JavaFX.

Using this configuration, the JavaFX tests can be enabled by passing the `-DincludeJavaFXTests=true` option to the TCK.
