Managed container doesn't generate JaCoCo's coverage report

[dudaMeneses]

Arquillian runs nicely but integration tests aren't covered by report when managed container is used. Follow below my configurations:

<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<dependencies>
    <dependency>
        <groupId>org.jboss.arquillian.extension</groupId>
        <artifactId>arquillian-jacoco</artifactId>
        <version>1.0.0.Alpha8</version>
    </dependency>
    <dependency>
        <groupId>org.jacoco</groupId>
        <artifactId>org.jacoco.core</artifactId>
        <version>${jacoco.version}</version>
    </dependency>
</dependencies>
<configuration>
    <append>true</append>
</configuration>
<executions>
    <execution>
        <id>pre-unit-tests</id>
        <phase>process-test-classes</phase>
        <goals>
            <goal>instrument</goal>
        </goals>
        <configuration>
            <destFile>${sonar.jacoco.reportPath}</destFile>
            <propertyName>jacoco.agent.ut.arg</propertyName>
            <append>true</append>
        </configuration>
    </execution>
    <execution>
        <id>post-unit-tests</id>
        <goals>
            <goal>restore-instrumented-classes</goal>
        </goals>
    </execution>
    <execution>
        <id>pre-integration-test</id>
        <phase>package</phase>
        <goals>
            <goal>prepare-agent</goal>
        </goals>
        <configuration>
            <destFile>${sonar.jacoco.itReportPath}</destFile>
            <propertyName>jacoco.agent.it.arg</propertyName>
            <append>true</append>
        </configuration>
    </execution>
    <execution>
        <id>jacoco-merge</id>
        <phase>verify</phase>
        <goals>
            <goal>merge</goal>
        </goals>
        <configuration>
            <fileSets>
                <fileSet implementation="org.apache.maven.shared.model.fileset.FileSet">
                    <directory>target/jacoco</directory>
                    <includes>
                        <include>*.exec</include>
                    </includes>
                </fileSet>
            </fileSets>
            <destFile>target/jacoco.exec</destFile>
        </configuration>
    </execution>
    <execution>
        <id>jacoco-site</id>
        <phase>verify</phase>
        <goals>
            <goal>report</goal>
        </goals>
    </execution>
   </executions>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<configuration>
    <environmentVariables>
        <JBOSS_HOME>arquillian.jbossHome</JBOSS_HOME>
    </environmentVariables>
    <argLine>${jacoco.agent.it.arg}</argLine>
    <includes>
        <include>integration/*</include>
    </includes>
    <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
    <systemPropertyVariables>
        <arquillian.launch>jboss-managed</arquillian.launch>
    </systemPropertyVariables>
</configuration>
<executions>
    <execution>
        <id>integration-tests</id>
        <phase>integration-test</phase>
        <goals>
            <goal>integration-test</goal>
            <goal>verify</goal>
        </goals>
    </execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
    <skipTests>true</skipTests>
    <systemPropertyVariables>
        <jacoco-agent.destfile>target/jacoco/jacoco-ut.exec</jacoco-agent.destfile>
    </systemPropertyVariables>
</configuration>
<executions>
    <execution>
        <id>unit-tests</id>
        <phase>test</phase>
        <goals>
            <goal>test</goal>
        </goals>
        <configuration>
            <skipTests>false</skipTests>
            <argLine>${jacoco.agent.ut.arg}</argLine>
            <excludes>
                <exclude>integration/*IT.java</exclude>
            </excludes>
        </configuration>
    </execution>
</executions>
</plugin>

[aslakknutsen]

Could you provide a complete example with some java code?

How do you run this?

[CSchulz]

I have the same issue and found a cause. If you set the argline property of the failsafe-plugin it doesn't work anymore otherwise the coverage is fully available.

@aslakknutsen If you want I can upload my showcase project.

[sventorben]

My team is also facing a similar issue.
@CSchulz If you can upload your showcase, I am happy to support further investigations.

[CSchulz]

@sventorben Finally I have found a solution to get it working. See my linked repository.

https://github.com/CSchulz/arquillian-jacoco-showcase

It seems that it isn't supported to change the datafile for integration tests (yet).
Mistake from my side, using wrong property name.

[CSchulz]

@aslakknutsen @sventorben I think it works like expected, if you don't overwrite the argline of the failsafe-plugin.

If you want to pass anything to the container (vm) you have to use the arquillian.xml and create there a property javaVmArguments.

[dudaMeneses]

@CSchulz I've done what you said but it doesn't filled any coverage anyway :(

@aslakknutsen follow below an quick example I created

https://github.com/dudaMeneses/arquillian-jacoco-example

[rankinc]

I'm also seeing something like this with a managed Tomcat 7 container. Can someone tell me how Arquillian and JaCoCo are supposed to integrate please? It looks like Arquillian is instrumenting everything offline before packaging everything up into my WebArchive (perhaps courtesy of the

<extension qualifier="jacoco">
    <property name="includes">com.my.stuff.*</property>
</extension>

section in my arquillian.xml?). I'm certainly not seeing the JaCoCo Java agent included on my managed Tomcat's command line, and if I try to add it in manually via ${argLine} then I get lots of "Class already instrumented" errors.

I'm also noticing that my jacoco.exec file is 68K after the unit tests run and 224K after the integration tests, but the coverage report generated in the verify phase always looks exactly the same, regardless of whether I include the integration tests or not.

Frankly, I have no idea what is going on here.

[rankinc]

Hmm, it looks like the jacoco.exec file from my integration tests only contains the tiny amount of extra coverage gained from executing the *IT classes themselves. I don't believe that the coverage data from the managed Tomcat container is being dumped at all, because when I configured the JaCoCo agent to dump to a totally separate file, that file was 0K long.

[rankinc]

OK, so I managed to get my coverage data after all. The steps were:

• Configure the container in arquillian.xml:

<property name="javaVmArguments">${argLine},jmx=true,includes=com.my.stuff.*</property>

• Configure the arquillian extension in arquillian.xml:

<extension qualifier="jacoco">
    <property name="excludes">*</property>
</extension>

• Dump the Tomcat JVM's coverage data using JMX from an @AfterTest method:

JMXServiceURL jmxURL = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:8089/jmxrmi");
try (JMXConnector jmxc = JMXConnectorFactory.connect(jmxURL)) {
    MBeanServerConnection connection = jmxc.getMBeanServerConnection();
    ObjectName objName = new ObjectName("org.jacoco:type=Runtime");
    connection.invoke(objName, "dump",
        new Object[]{ true },
        new String[]{ Boolean.TYPE.getName() }
    );
}

My Coverage Report in Jenkins still mentions classes that I believe I've excluded - although they all show 0% coverage now - but this must be close to the correct solution.

[mejdahl]

I have the same problem, I got a jacoco.exec but no jacoco-it.exec"
My setup is more or less the same as your arquillian-jacoco-example.

The difference to our project is:

• Wildfly is using embedded container
• Arquillian-bom 1.1.11.Final
• arquillian-transaction 1.0.3.Final
• shrinkwrap-bom 1.2.3
• shrinkwrap-resolver-bom 2.2.2
• arquillian-persistence-dbunit 1.0.0.Alpha7

Could it be the use of embedded wildfly container or is it because I use core 1.1.11 or....
Any suggestion?

Please note that I made the arquillian-jacoco-example work nicely for Wildfly 8.2.1 managed!

[rankinc]

Speaking as a user who has also beaten my head against the table to get this working...

I have the same problem, I got a jacoco.exec but no jacoco-it.exec

What matters is the coverage data that has been dumped into your jacoco.exe. For example, my build contains this:

<properties>
    <argLine></argLine>
</properties>

...

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>${jacoco.version}</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>verify</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

My integrations tests also pass ${argLine} as part of the container's javaVmArgument property in arquillian.xml. This results in both my unit tests and my integration tests writing into jacoco.exec, because the JaCoCo agent uses append=true by default.
Finally, I create the report in the verify phase, which comes after the integration-test phase in the Maven build lifecycle; you might want to check that you're not creating your report in the package phase instead.

[mejdahl]

Well I created my Jacoco look like this:

<plugin> 
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>${version.jacoco}</version>
    <executions>
        <execution>
            <id>Jacoco-prepare-agent</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>Jacoco-prepare-agent-integration</id>
            <goals>
                <goal>prepare-agent-integration</goal>
            </goals>
            <configuration>
                <destFile>${project.build.directory}/jacoco-it.exec</destFile>
            </configuration>
        </execution>
    </executions>               
</plugin>

I expected to find a jacoco.exec and a jacoco-it.exec in the target/ folder but only found the jacoco.exec!
The maven output look as expected, both excecutions is shown at the right lifecycle and with the right filenames!

Note that the "javaVmArguments" only works when using managed container..

I have tried your suggestion, but when looking in the generated "site" it is clear that no "IT" classes is covered.

[rankinc]

So where do you execute the report goal for jacoco-maven-plugin?

[mejdahl]

I tried a lot of combination but I stopped lokking for the right spot for the report generation when I found that the file wasn't genereated!

<execution>
    <id>jacoco-site</id>
    <phase>verify</phase>
    <goals>
        <goal>report</goal>
        <goal>report-integration</goal>
    </goals>
</execution>

Here is a snip form the maven output

[INFO] argLine set to -javaagent:C:\\Users\\sihs\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.6.201602180812\\org.jacoco.agent-0.7.6.201602180812-runtime.jar=destfile=C:\\src\\domain\\geodata\\geodata\\target\\jacoco-it.exec
but it failed with
[INFO] Skipping JaCoCo execution due to missing execution data file:C:\src\domain\geodata\geodata\target\jacoco-it.exec

[rankinc]

I think this implies that JaCoCo isn't instrumenting your "IT" classes, which would suggest that your maven-failsafe-plugin isn't configured correctly with your integration test argLine.
Perhaps this document is relevant?