Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Table of Contents

Unit Tests and Code Coverage

There are two different ways to feed the SonarQube platform with information on unit tests execution and code coverage:

  • Letting the SonarQube platform drive the execution of the unit tests and the code coverage tool
  • Reusing reports that have been generated by external tools prior to the SonarQube analysis

These two ways are handled by Gallio. So you first have to install it on the machine(s) running the SonarQube analysis. You also have to install the code coverage tool of your choice that will be driven by Gallio. Supported tools are dotCover, NCover, OpenCover and PartCover.

Configuring Gallio

Log in as a System administrator and go to Settings > Configuration > General Settings > .NET Gallio.

  • Set the path to the Gallio installation directory through the sonar.gallio.installDirectory property.
  • Define which tool to use to compute the code coverage through the sonar.gallio.coverage.tool property. Then, set the installation directory of this code coverage tool through the sonar.dotcover.installDirectory or sonar.opencover.installDirectory or sonar.partcover.installDirectory property. Note that there is not such a property for NCover as the installation path of NCover is automatically discovered by Gallio.

Letting the SonarQube platform drive the execution of the unit tests and the code coverage tool

This is the default mode.

The first and unique step is to tell SonarQube which assemblies are unit test assemblies. To do so, set the sonar.donet.visualstudio.testProjectPattern property.

Then, you just have to run a SonarQube analysis and you'll get data on unit tests and code coverage. Indeed, the paths to the compiled unit test assemblies are automatically retrieved from the Visual Studio "csproj" files. And the execution of unit tests and the driving of the coverage tool is automatically performed by Gallio.

Reusing existing reports

To activate this mode, add the following line to your project configuration file:

Then, you just need to provide SonarQube with the following reports: unit tests execution and code coverage.

The following command lines are launched while letting the SonarQube platform drive the execution of the unit tests and the code coverage tool. The reuseReport mode has been tested with those command lines. That's why, it is highly recommend to generate your reports by running similar command lines:

Deactivating Unit Tests and Code Coverage

Add the following line to your project configuration file:

Integration Tests and Code Coverage

As the principles are very similar to the unit tests. please read the Unit Tests and Code Coverage before.

Letting the SonarQube platform drive the execution of the unit tests and the code coverage tool

To activate this mode, add the following line to your project configuration file:

The first and unique step is to tell SonarQube which assemblies are integration test assemblies. To do so, set the sonar.donet.visualstudio.itProjectPattern property.

Then, you just have to run a SonarQube analysis and you'll get data on integration tests and code coverage. Indeed, the paths to the compiled integration test assemblies are automatically retrieved from the Visual Studio "csproj" files. And the execution of integration tests and the driving of the coverage tool is automatically performed by Gallio.

Reusing existing reports

To activate this mode, add the following line to your project configuration file:

Then, you just need to provide SonarQube with the following reports: integration tests execution report and code coverage report.

Deactivating Integration Tests and Code Coverage

This is the default mode.

FAQ

My unit tests and integration tests are defined within the same assemblies, how can I run my unit tests only?

Define the sonar.gallio.filter property (or the sonar.gallio.it.filter property for integration tests):

How can I exclude some specific assemblies and/or namespaces from code coverage computation?

Define the sonar.gallio.coverage.excludes property. Note that this property is shared between unit and integration tests.

SonarQube cannot retrieve the compiled unit test assemblies

If for some reasons, compiled unit test assemblies are moved after the build (no longer available in directories defined in "csproj" files), you can tell SonarQube where to find them thanks to the sonar.dotnet.test.assemblies property (sonar.dotnet.it.assemblies property for integration tests):

0% code coverage in reuseReport mode

This might happen if you have generated the reports then moved the files of your Visual Studio solution. Indeed, in the XML coverage files, the paths to the C# source files must be the same as the paths specified in the csproj files of the analyzed solution, otherwise the coverage date are simply not taken into account.

OpenCover is activated and Gallio fails without any explanation

OpenCover needs .net framework 4 in order to run. If you try to use OpenCover without .net4 installed, the analysis will crash.

When using OpenCover with Gallio, I get 0% code coverage whereas I do have test classes that are covered by tests

Most probably, you should use the following property to run the tests in an isolated process:

As describe in the Gallio documentation, possible values are IsolatedAppDomain (PartCover default runner type), IsolatedProcess and Local. Does not apply to NCover.

Gallio execution fails with following error message

This error happens when one of the dotnet config files (app.config, web.config, ...) of your Visual Studio solution specifies a runtime version not supported by the instance of Gallio. This is often the case when the following XML fragment appears in an app.config file:

The "sku" attribute is not something understandable by Gallio... If you remove the above XML fragment by the one below the issue is solved:

NCover, the analysis succeeded but only one test assembly is taken into account

The NCover Complete Edition allows to run tests from multiple assemblies and merge the coverage data, the NCover Classic Edition does not.

The workaround is to activate the Gallio safe mode (sonar.gallio.safe.mode=true). Gallio will then be launched for each test assembly and the generated XML reports will be merged.

How to set Gallio's timeout?

Set the sonar.gallio.timeoutMinutes property (default value is 30).

How can I run Gallio and PartCover on a 64-bit Windows?

Gallio and PartCover work perfectly out-of-the-box on x86 Windows, but not on 64-bit versions. To make them work:

  1. Download the latest Gallio x64 installer and install it in a folder, but not under "Program Files".
  2. Download the latest PartCover installer and install it in a folder, but not under "Program Files".
  3. Run corflags.exe on Gallio and PartCover executables:

See http://www.planetgeek.ch/2009/10/15/get-partcover-running-on-x64-windows/

Gallio execution fails on a Windows 64-bit system

If in the logs you get something such as:

that might be because some of your assemblies have been compiled with a x86 profile (32 bits) while Gallio is configured to run in an "Any CPU" mode, hence 64bits on your system.
This problem might be solved using corflags tool to force Gallio to run in a 32 bits mode as shown below:

Some tests fail unexpectedly using a legacy .net2 library

If you get this kind of error:

It is likely that you are facing the issue described here

The problem might be solved if you modify your installation of Gallio. Add the following XML fragment in files Gallio.Host.x86.exe.config and/or Gallio.Host.x64.exe.config:

  • No labels