Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Name

...

Sonar Pitest Plugin

...

License

...

LGPL v3

...

Authors

...

Alexandre Victoor

...

Jira

...

http://jira.codehaus.org/browse/SONARPLUGINS/component/15475

...

Sources

Wiki Markup
{iframe:src=http://

...

update.

...

sonarsource.org/

...

plugins/

...

pitest

...

Latest version

...

0.1

...

Download

...

http://repository.codehaus.org/org/codehaus/sonar-plugins/sonar-pitest-plugin/0.1/sonar-pitest-plugin-0.1.jar

...

-confluence.html|width=700|height=350|frameborder=0}
Your browser does not support iframes.
{iframe}

 

Description / Features

PIT is a mutation testing tool for java. You can check out the official pitest web site  for site for more details on mutation testing and PIT. 

...

The goal of this plugin is to bring PIT results to sonarSonarQube. Right now the integration of these result results is quite simple, "survived mutants" on code covered by tests are seen as sonar violationsSonarQube issues.  

Info
titleNote
Even if PIT detects "survived mutants" on uncovered lines of code, these mutants are simply ignored by the plugin.

 

Usage & Installation

Include Page
Include - Plugin Installation
Include - Plugin Installation

Usage

Limitations of mutation testing

...

  • Mutation testing is very CPU time expensive. It is really important to control the scope of mutation testing in order to keep acceptable sonar analysis SonarQube analysis times. See below for tips on analysis time.
  • Mutation testing works on true unit tests. Do not try to use it on integration tests, you might mess up your database, file system, whatever external system used by your integration tests. 

 

Tip

Henri Coles, creator of PIT, gave very useful tips on the PIT mailing list to help reduce PIT execution time:

  • Target only specific portions of your codebase (using the class
    filters)
  • Limit the mutation operators used
  • Limit the number of mutations per class
  • Tweak the number of threads

Check out the "Advanced configuration properties" at the bottom of this page to address the above points.

 

Sonar server installation 

On the sonar side, just copy the sonar-pitest-plugin jar to the extensions/plugins directory just like any regular sonar plugin. 

Configuration

Since mutation testing is not (yet) officially supported by sonarSonarQube, this plugin acts as a "single rule" rule engine... This rule, named "Survived mutant", is disabled by default and hence needs to be activated when pitest is used. 

Project build setup

You have two options in order to produce PIT reports for sonar. You can either let the sonar plugin run PIT or you can run PIT prior to the sonar analysis.

PIT launched by Sonar 

Warning
There is a compatibility issue with maven 3.X

This is the recommended way to go if your build is based on maven 2. Most mandatory PIT configuration entries will be set using information provided by the pom.xml file of your project. You just need to activate PIT execution and specify which classes to mutate and tests to execute. Below an example:

Code Block
<properties>
  <sonar.pitest.mode>active</sonar.pitest.mode>
  <sonar.pit.target.classes>com.acme.tools.commons*</sonar.pit.target.classes>
  <sonar.pit.target.tests>com.acme.tools.commons*</sonar.pit.target.tests>
</properties>

Note: you will find at the bottom of this page an exhaustive description of all the configuration parameters.

The example above tells PIT to mutate classes from packages starting with "com.acme.tools.commons". It also specifies that test classes are located in packages starting with "com.acme.tools.commons".

...

PIT needs to be launched before SonarQube

You can launch PIT using the PIT maven plugin or the command line runner. PIT execution must be done before sonar SonarQube analysis. You also need to specify the "reuseReport" mode of the PIT sonar SonarQube plugin. 

Pit needs to be configured in order to generate XML reports. Be aware that PIT default behavior is to generate HTML reports.  Below a simple configuration example for maven :

...

Last but not least, you need to run a sonar SonarQube analysis with the PIT plugin activated in "reuseReport" mode. The following command would do the job :
"mvn sonar:sonar -Dsonar.pitest.mode=reuseReport"

By default sonar will SonarQube will search the latest PIT report in "target/pit-reports". You can specify another location using property "sonar.pitest.reportsDirectory".

...

Below the exhaustive list of configuration properties of the sonar SonarQube pitest plugin.

Name

Key

Default value

Description

Pitest activation mode

sonar.pitest.mode

skip

Possible values : 'skip' , 'active' (not supported yet) and 'reuseReport'

Path to the pitest reports

sonar.pitest.reportsDirectory

target/pit-reports

Path used when mode "reuseReport" is activate to locate pitest xml reports.
Pitest creates a new subfolder "timestamp" at each shot. The sonar SonarQube plugin will explore these subfolders and find the newest xml reports generated.

Classes to include in mutation testssonar.pit.target.classesgroupId*The classes to be mutated. This is expressed as a comma seperated list of globs. For example com.mycompany.* or com.mycompany.package.*, com.mycompany.packageB.Foo, com.partner.*. When maven is used, the default value is 'groupId*'
Tests to runsonar.pit.target.testssonar.pit.target.classesA comma seperated list of globs can be supplied to this parameter to limit the tests available to be run. If not specified, the value of "sonar.pit.target.classes" is used (not recommended)

Advanced configuration properties

Tip
titleAbout advanced configuration

The properties below mimic the configuration option of the official PIT maven plugin. You can check out the official documentation of this maven plugin to get more detailed information on these options.

...

Name

...

Key

...

Default value

...

Description

...

 

...

Whether to throw error when no mutations found.
Defaults to true.

...

Maximum distance to look from test to class. Relevant when mutating static initializers

...

Calls excluded from mutations

...

sonar.pit.avoid.calls.to

...

major logging framework APIs

...

List of packages and classes which are to be considered outside the scope of mutation

...

Classes not to mutate or run tests from

...

sonar.pit.excluded.classes

...

 

...

List of globs to match against class names. Matching classes will be excluded from mutation. Matching test classes will not be run (note if a suite includes an excluded class, then it will “leak” back in).

...

Not useful when maven is used.
Classpath that contains either JUnit or TestNG as well as your code, tests and any dependencies. When maven is used, this classpath is build using maven classpaths.

You can check out the quickstart section of the official pitest web site for detailed instructions.