Name	Name	Last commit message	Last commit date
parent directory ..
res	res
README.md	README.md

Java Code Coverage

The following document goes through running code coverage for Java, automation with Jenkins and integration with SonarQube. The code coverage tool to be used for Java is JaCoCo.

⚔ Note: This document currently covers coverage reporting for Maven based projects, for a wider Project Management support please contact us.

This document is separated into 3 parts:

Manual Coverage - section will present you the supported method of running code coverage on Java.
Jenkins Automation - section will introduce the basic steps in order to automate the coverage process using the web UI.
SonarQube Integration - section will teach you how to publish your results to SonarQube using the Jenkins web UI as well as manually.

⚔ Note: Additional advanced topics such as JJB configuration are covered at the bottom of the document under the Advanced Topics section.

Java Code Coverage
Table of Contents
Manual Coverage
- Prerequisites
- Running code coverage manually
  - Example
Jenkins Automation
SonarQube Integration

Manual Coverage

Prerequisites

⚔ Note: notice there are several methods and tools to run code coverage such as JCov, Clover, Cobertura and EMMA, we will only introduce one way of going about it which we have concluded to be the most suitable and straightforward for this process.

Running code coverage manually

in order to run coverage on your maven project's tests, simple run
```
mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true
```
⚔ Note: The parameters above requires the JaCoCo plugin to install it's run-time agent for your tests and continue testing if either of your tests fail.

⚔ Note: For advanced configuration options, see Maven Plug-in but note that the default configuration is being propagated automatically based on your POM.
coverage reports can be found at target/jacoco.exec in your project's root directory

Example

The following example includes encountering a failure and a successful run.

⚔ Note: This example uses the standardized JUnit testing framework.

Let's assume you have the following 2 Java files and an additional POM file:

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.testingjava.app</groupId>
  <artifactId>maven-test-project</artifactId>
  <packaging>jar</packaging>
  <version>1.0</version>
  <name>Maven Test Project</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

src/main/.../App.java

package com.testingjava.app;

/**
 * our simple app class
 */
public class App
{
    public static String printIt(String name)
    {
      return "Hello " + name;
    }
}

src/test/.../AppTest.java

package com.testingjava.app;

import junit.framework.TestCase;

/**
 * Unit test for our simple App.
 */
public class AppTest
    extends TestCase
{
    /**
     * Create the test case
     *
     * @param testName name of the test case
     */
    public AppTest( String testName )
    {
        super( testName );
    }

    /**
     * Making a simple test
     */
    public void testApp()
    {
        assertEquals(App.printIt("Bob"), "Welcome Bob");
    }
}

⚔ Note: The following files were generated using a basic maven project archetype.

we attempt to run coverage for our test using

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true

but apparently we hit an error

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.testingjava.app.AppTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.033 sec <<< FAILURE!
testApp(com.testingjava.app.AppTest)  Time elapsed: 0.017 sec  <<< FAILURE!
junit.framework.ComparisonFailure: expected:<Welcome...> but was:<Hello...>
    at junit.framework.Assert.assertEquals(Assert.java:81)
    at junit.framework.Assert.assertEquals(Assert.java:87)
    at com.testingjava.app.AppTest.testApp(AppTest.java:26)
Results :
Failed tests:   testApp(com.testingjava.app.AppTest): expected:<Welcome...> but was:<Hello...>
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[ERROR] There are test failures.
Please refer to ./javaTest/target/surefire-reports for the individual test results.

as you can see, we've encountered an error due to an issue in our tests! Now that we've fixed the problem, you can see our tests running successfully!

----------------------------------------
 T E S T S
----------------------------------------
Running com.testingjava.app.AppTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ maven-test-project ---
[INFO] Building jar: javaTest/target/maven-test-project-1.0.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ maven-test-project ---
[INFO] Installing javaTest/target/maven-test-project-1.0.jar to .m2/repository/com/testingjava/app/maven-test-project/1.0/maven-test-project-1.0.jar
[INFO] Installing javaTest/pom.xml to .m2/repository/com/testingjava/app/maven-test-project/1.0/maven-test-project-1.0.pom
[INFO] -------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------
[INFO] Total time: 2.645 s
[INFO] Finished at: 2017-04-13T15:08:27+03:00
[INFO] Final Memory: 17M/185M
[INFO] -------------------------------------------

you can now see the generated report file jacoco.exec in the target directory in the project's root directory and we are done! (:

⚔ Note: The jacoco.exec file is a binary file and can only be viewed by either the Jenkins plugin or SonarQube, further details on publishing these results appear in the following chapters.

Jenkins Automation

Prerequisites

Jenkins v1.653+
Jenkins JaCoCo plugin v2.2.0+
testing environment running:
- fedora v23
- Java v8+
- git
⚔ Note: you could either use a provisioning system such as docker daemon, OpenShift, OpenStack, Kubernetes, etc. or use a local environment.

⚔ Note: notice there are several methods and tools to manage CI and automation such as Gump, Hudson, Bamboo, Travis and more. We will only introduce and support Jenkins for this end as it is the standardized CI tool in RedHat.

Automating using the web UI

Continuing from the previous chapter, assuming our project files are held on a remote github repository https://github.com/RedHatQE/CodeQuality/tree/master/examples/java-test-repo.

Example

in the main Jenkins page, click to New Item button to create a new job
name your job, select the Freestyle Project radio button and save the new job
on the newly opened screen, set the Source Code Management to git and fill in our repository
scroll down, click Add build step and select Execute shell to create a new bash script build step
paste the following deployment script onto the bash text editor
```
# install dependencies
dnf install -y maven

cd examples/java-test-repo

# run the tests with coverage
mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true
```
⚔ Note: we are using Fedora v23+, if you are using a older version or a different redhat distribution, you might want to try 'yum' instead of 'dnf'

⚔ Note: the -y parameter in the dnf command approves installation prompts which is mandatory for automation purposes.

let's have a look for a moment at our script, we can see it's divided into 2 main parts:
- installation of prerequisites
- running our tests with coverage to create a report (as seen on the previous chapter)
  
  ⚔ Note: in most cases, each of these parts will be more complicated and it's a good habit to break each part into it's own bash build step to ease troubleshooting
run a build of our newly created job

And we're done! on the next chapter you will learn how to publish your generated results into SonarQube to view them.

Uploading coverage results to Jenkins

Sometimes it's useful to have your coverage results uploaded to your Jenkins job which could ease troubleshooting procceses in case of large scale development efforts which might require several independant coverage jobs.

For that purpose, we will use the Jenkins JaCoCo plugin in order to preview this results in our job's web UI.

Example

Continuing from the previous section, assuming our newly created job has generated a coverage report into ${WORKSPACE}/target.

in the job's configuration screen, add a post-build action to record the JaCoCo coverage report
input the following relative paths in the actions configuration fields save the job
- Path to exec files: **/target/**.exec
- Path to class directories: **/target/classes
- Path to source directories: **/src/main/java
after rerunning our job you will be able to view the report's preview in your specific build's page

SonarQube Integration

Prerequisites

Integrating SonarQube through the Jenkins web UI

As a direct continuation of the previous chapter, building on the same Jenkins job, we'll now add the SonarQube integration.

Example

add the sonar configuration options to your pom.xml file

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

<!-- language (string): project language(java) -->
<sonar.language>java</sonar.language>

<!-- host.url (string): SonarQube server URL(test server URL) -->
<sonar.host.url>sonar_server_address</sonar.host.url>

<!-- projectKey (string): SonarQube project identification key (unique) -->
<sonar.projectKey>maven-test-project_full-analysis</sonar.projectKey>

    <!-- inclusions (string): file inclusion pattern, used to exclude non-java files -->
  <sonar.inclusions>**/*.java</sonar.inclusions>

<!-- ws.timeout (int): optional connection timeout parameter -->
  <sonar.ws.timeout>180</sonar.ws.timeout>
</properties>

⚔ Note: for further details on SonarQube analysis parameters, see Analysis Parameters.

in the job configuration, add another execute shell build step for our Maven sonar report

paste in the sonar target command

# publish results to sonar
mvn sonar:sonar

run a build again to view the reported results

you'd now be able to locate your project's dashboard on the SonarQube server.

And we are done! In your project's dashboard you'll be able to view your coverage results.

Publishing to SonarQube manually

Sometimes it's useful to be able to publish our coverage report to SonarQube manually. Although it is not a recommended methodology, we will allaborate upon the needed steps for those ends.

⚔ Note: in this section we assume you are running an up-to-date RedHat distribution(Fedora, CentOS, RHEL)

Example

add the sonar configuration options to your pom.xml file

<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

<!-- language (string): project language(java) -->
<sonar.language>java</sonar.language>

<!-- host.url (string): SonarQube server URL(test server URL) -->
<sonar.host.url>sonar_server_address</sonar.host.url>

<!-- projectKey (string): SonarQube project identification key (unique) -->
<sonar.projectKey>maven-test-project_full-analysis</sonar.projectKey>

    <!-- inclusions (string): file inclusion pattern, used to exclude non-java files -->
  <sonar.inclusions>**/*.java</sonar.inclusions>

<!-- ws.timeout (int): optional connection timeout parameter -->
  <sonar.ws.timeout>180</sonar.ws.timeout>
</properties>

⚔ Note: for further details on SonarQube analysis parameters, see Analysis Parameters.

run your sonar maven target
```
mvn sonar:sonar
```

finally, you should be able to see a success prompt with a link to your published coverage report dashboard such as this one:

DEBUG: Upload report
DEBUG: POST 200 http://sonar_server_address/api/ce/submit?projectKey=some-project&projectName=Some%20Project | time=43ms
INFO: Analysis report uploaded in 52ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://sonar_server_address/dashboard/index/some-project
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://sonar_server_address/api/ce/task?id=AVrR-YHSEXNZ6r-PQPEx
DEBUG: Report metadata written to /root/ruby_coverage_testfiles/.sonar/report-task.txt
DEBUG: Post-jobs :
INFO: ---------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ---------------------------------------------
INFO: Total time: 13.570s
INFO: Final Memory: 53M/215M
INFO: ---------------------------------------------

and your results have been published! (:

Advanced Topics

Covering Sevices

In order to run coverage over a service or a process not launched manually from it's entry-point, i.e multi-service products, API integration tests, multi-host integration tests, etc, we are proposing the following solution, which inserts a pipeline hook to each Java process..

Prerequisites

Implementation Steps

Download the jacocoagent-runtime.jar using the following command:

wget http://repo1.maven.org/maven2/org/jacoco/org.jacoco.agent/0.8.0/org.jacoco.agent-0.8.0-runtime.jar

This will download the jar to your current directory, which we will call .

Set the JAVA_TOOL_OPTIONS environment variable with the correct java agent configuration for jacoco.

export JAVA_TOOL_OPTIONSS="-javaagent:<path to agent>/jacocoagent-runtime.jar= \
    # A colon separated list of compiled class files, e.g. Foo.class:Bar.class:Bazz.class
    includes=<LIST OF CLASSES>, \
    # The id that the coverage run will be identified by in the coverage report.
    sessionid=<SESSION ID>, \
    # The target directory you wish the coverage report to be written to.
    destfile=<TARGET DIRECTORY>/jacoco.exec"

⚔ Note: The sessionid parameter is optional, and jacoco will generate a random identification string if it is not set. However, it is recommended that the SESSION ID be set to the same string across all services being tested at the same time. For example if the test is run by Jenkins use the job name and number.

⚔ Note: The includes parameter is optional, if it is left out the coverage report will be generated for all files including dependencies.

Start the service. This will start the JVM with the java agent enabled.
Run the tests.
Stop the service, this will cause the jacoco agent to dump the coverage results to the jacoco.exec file in your target directory.

Example

This example runs against the java_coverage_testfiles example project from above. For the purposes of this example org.jacoco.agent-0.8.0-runtime.jar has been downloaded into the root of the project.

Clone the example project.

git clone https://github.com/RedHatQE/CodeQuality

Change into the cloned project.

cd ~/CodeQuality/examples/java-test-repo

Get a copy of the org.jacoco.agent-0.8.0-runtime.jar.

wget http://repo1.maven.org/maven2/org/jacoco/org.jacoco.agent/0.8.0/org.jacoco.agent-0.8.0-runtime.jar

Set the JAVA_TOOL_OPTIONS environment variable.

export JAVA_TOOL_OPTIONS="-javaagent:$(pwd)/org.jacoco.agent-0.8.0-runtime.jar=includes=com.testingjava.app.AppTest,sessionid=TestSession,destfile=$(pwd)/target/jacoco.exec"

Run the tests.

mvn test

After the tests are complete there will be a jacoco.exec file in the target directory.

⚔ Note: The jacoco.exec file is a binary file and can only be viewed by either the Jenkins plugin or SonarQube.

Jenkinsfile

Starting with Jenkins 2, automation configuration can mainitained using a Jenkinsfile which levrages the power of Grooveyscript to describe a jenkins job.

Prerequisites

⚔ Note: For more details on the Jenkinsfile format, see Using a Jenkinsfile

Example

The following file illustrates a possible Jenkinsfile configuration

pipeline {
    agent { node { label 'sonarqube-upshift' } }
    options {
      skipDefaultCheckout true
    }
    triggers {
      cron('0 8 * * *')
    }
    stages {
        stage('Deploy') {
            steps {
                // clone project
                git url: 'https://github.com/RedHatQE/CodeQuality.git'
            }
        }
        stage('Analyse') {
            steps {
                // coverage tests initialization script
                dir('examples/java-test-repo'){
                sh '''mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent \
                  install -Dmaven.test.failure.ignore=true || true'''
                }
            }
        }
        stage('Report') {
            steps {
              // initite pre-configured sonar scanner tool on project
              // 'sonarqube_prod' is our cnfigured tool name, see yours
              // in the Jenkins tool configuration
              // NOTE: pay attention we are using maven under sonar env
              withSonarQubeEnv('sonarqube_prod') {
                dir('examples/java-test-repo'){

                sh "mvn sonar:sonar"
                }
              }
            }
        }
    }
}

Jenkins Job Builder

Jenkins automation configuration can also be done by using the Jenkins Job builder (JJB) which takes simple descriptions of Jenkins jobs in YAML or JSON format and uses them to configure Jenkins.

Prerequisites

⚔ Note: For more details on installing and configuring JJB, see the Quick Start Guide

Example

The following file illustrates a possible JJB configuration

- job:
    name: sonarqube_java_analysis


    #######################################################
    ############## SonarQube Parameters ###################
    #######################################################

    # sonarqube project parameters, set before build
    parameters:
      - string:
          name: SONAR_KEY
          default: sonarqube_java_analysis
          description: "SonarQube unique project key"
      - string:
          name: SONAR_NAME
          default: Ruby-Plugin Java Analysis
          description: "SonarQube project name"
      - string:
          name: SONAR_PROJECT_VERSION
          default: "1.0"
          description: "SonarQube project version"

    #######################################################
    ############### Logging Aggregation ###################
    #######################################################

    # define how many days to kee build information
    properties:
      - build-discarder:
          days-to-keep: 60
          num-to-keep: 200
          artifact-days-to-keep: 60
          artifact-num-to-keep: 200

    #######################################################
    ################### Slave Image #######################
    #######################################################

    node: sonarqube-upshift

    #######################################################
    ################ Git Trigger Config ###################
    #######################################################

    # git repo to follow, skip-tag to not require auth
    scm:
      - git:
          url: https://github.com/RedHatQE/CodeQuality.git
          skip-tag: true

    # git polling trigger set to once an hour
    triggers:
      - pollscm:
          cron: "0 0 * * 0"
          ignore-post-commit-hooks: True

    #######################################################
    ################### Build Steps #######################
    #######################################################

    builders:

      # coverage tests initialization script
      - shell: |
          cd examples/java-test-repo
          mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent \
            install -Dmaven.test.failure.ignore=true || true

      # sonar runner parameters, set sources and baseDir to project home
      # projectKey (string): SonarQube project identification key (unique)
      # projectName (string): SonarQube project name (NOT unique)
      # projectVersion (string): SonarQube project version (unique)
      # sources (string): source code home directory
      # projectBaseDir (string): project home directory (same as sources)
      # language (string): project language(ruby)
      # inclusions (string): file inclusion pattern
      # exclusions (string): file exclusion pattern
      # login (string): SonarQube server user name
      # password (string): SonarQube server user password
      - sonar:
          sonar-name: sonarqube_prod
          properties: |
            sonar.projectKey=$SONAR_KEY
            sonar.projectName=$SONAR_NAME
            sonar.projectVersion=$SONAR_PROJECT_VERSION
            sonar.sources=${WORKSPACE}/examples/java-test-repo
            sonar.projectBaseDir=${WORKSPACE}/examples/java-test-repo
            sonar.language=java
            sonar.inclusions=**/*.java
            sonar.exclusions=src/test/**/*.java
            sonar.login=test
            sonar.password=test
            sonar.ws.timeout=180
            sonar.java.binaries=**/target/classes

Job DSL

Example

def jobName = 'java-coverage-dsl-sample'
def giturl = 'https://github.com/RedHatQE/CodeQuality.git'
def sonarProperties = '''
    sonar.projectKey=sonarqube_java_analysis
    sonar.projectName=Java Analysis
    sonar.projectVersion=1.0
    sonar.sources=${WORKSPACE}/examples/java-test-repo
    sonar.projectBaseDir=${WORKSPACE}/examples/java-test-repo
    sonar.language=java
    sonar.inclusions=**/*.java
    sonar.exclusions=tests/**/*.java
    sonar.login=test
    sonar.password=test
    sonar.ws.timeout=180
    sonar.java.binaries=target/maven-test-project-1.0.jar
       '''.stripIndent()


job(jobName) {
    label('sonarqube-upshift')
    scm {
        git(giturl)
    }
    triggers {
        cron '0 8 * * *'
    }
    steps {
        shell '''
           cd examples/java-test-repo
           mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true || true
        '''
    }
    configure {
        it / 'builders' << 'hudson.plugins.sonar.SonarRunnerBuilder' {
            properties ("$sonarProperties")
    }
  }
}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java

java

README.md

Java Code Coverage

Table of Contents

Manual Coverage

Prerequisites

Running code coverage manually

Example

Jenkins Automation

Prerequisites

Automating using the web UI

Example

Uploading coverage results to Jenkins

Example

SonarQube Integration

Prerequisites

Integrating SonarQube through the Jenkins web UI

Example

Publishing to SonarQube manually

Example

Advanced Topics

Covering Sevices

Prerequisites

Implementation Steps

Example

Jenkinsfile

Prerequisites

Example

Jenkins Job Builder

Prerequisites

Example

Job DSL

Example

Files

java

Directory actions

More options

Directory actions

More options

Latest commit

History

java

Folders and files

parent directory

README.md

Java Code Coverage

Table of Contents

Manual Coverage

Prerequisites

Running code coverage manually

Example

Jenkins Automation

Prerequisites

Automating using the web UI

Example

Uploading coverage results to Jenkins

Example

SonarQube Integration

Prerequisites

Integrating SonarQube through the Jenkins web UI

Example

Publishing to SonarQube manually

Example

Advanced Topics

Covering Sevices

Prerequisites

Implementation Steps

Example

Jenkinsfile

Prerequisites

Example

Jenkins Job Builder

Prerequisites

Example

Job DSL

Example