This page documents versions 2.1 and below of the C# plugin. For newer versions, consult this page.

If you reached this page from a link, please send a note to dev@sonar.codehaus.org (you'll have to join first.)

If you reached this page from a bookmark, please update it.

Requirements

Unit Tests and Code Coverage

There are two different ways to feed the SonarQube platform with information on unit test execution and code coverage:

  1. Let the SonarQube platform drive the execution of the unit tests and the code coverage tool
  2. Reuse reports that have been generated by external tools prior to the SonarQube analysis

Both ways use are handled by Gallio, so you first have to install Gallio on the machine(s) running the SonarQube analysis. You also have to install the code coverage tool of your choice that will be driven by Gallio. Supported tools are dotCover, NCover, OpenCover and PartCover.

Configuring Gallio

Log in as a System administrator and go to Settings > Configuration > General Settings > .NET Gallio.

Letting the SonarQube platform drive the execution of the unit tests and the code coverage tool

This is the default mode.

The first step is to tell SonarQube which projects are unit test projects. To do so, set the sonar.dotnet.visualstudio.testProjectPattern property.

# Semicolon-separated list of patterns to identify unit test projects
# Default value is *.Tests
sonar.dotnet.visualstudio.testProjectPattern=*.Tests;*.UnitTests

Then, you just have to run a SonarQube analysis and you'll get data on unit tests and code coverage. The paths to the unit test assemblies are automatically retrieved from the Visual Studio ".csproj" files, and the execution of unit tests and the driving of the coverage tool is automatically performed by Gallio.

Reusing existing reports

To activate this mode, add the following line to your project configuration file:

sonar.gallio.mode=reuseReport

Then configure the paths to the unit test execution and code coverage reports.

# Paths are relative to the Solution base directory

# Path to the unit tests execution report
sonar.gallio.reports.path=reports/gallio-report.xml

# Path to the code coverage report
sonar.gallio.coverage.reports.path=reports/coverage-report.xml

 

The reuseReport mode has been tested with the output of the following commands, which are the ones used when the SonarQube platform drives the execution of the unit tests and the code coverage tool. It is highly recommend that you generate your reports using similar command lines:

# Based on the following solution sample: https://github.com/SonarCommunity/sonar-dotnet/tree/master/sonar/dotnet/sonar-dotnet-plugin/src/test/resources/solution/Example

# dotCover
C:\Program Files\JetBrains\dotCover\v2.2\Bin\dotCover.exe a /TargetExecutable=C:\Program Files\Gallio\bin\Gallio.Echo.exe /TargetWorkingDir=C:\Example\target\sonar "/TargetArguments=\"/r:Local\" \"/report-directory:C:\Example\target\sonar\" \"/report-name-format:gallio-report\" \"/report-type:Xml\" \"C:\Example\Example.Core.Tests\bin\Debug\Example.Core.Tests.dll\"" /Filters=+:module=Example.Application;class=*;function=*;+:module=Example.Core;class=*;function=*; /ReportType=TeamCityXML /Output=C:\Example\target\sonar\coverage-report.xml

# NCover
C:\Program Files\Gallio\bin\Gallio.Echo.exe /r:NCover3 /report-directory:C:\Example\target\sonar /report-name-format:gallio-report /report-type:Xml C:\Example\Example.Core.Tests\bin\Debug\Example.Core.Tests.dll /runner-property:NCoverCoverageFile=C:\Example\target\sonar\coverage-report.xml /runner-property:NCoverArguments=//ias Example.Application;Example.Core

# OpenCover
C:\Program Files\OpenCover\OpenCover.Console.exe -register:user -target:C:\Program Files\Gallio\bin\Gallio.Echo.exe -targetdir:C:\Example\target\sonar "-targetargs:\"/r:IsolatedAppDomain\" \"/report-directory:C:\Example\target\sonar\" \"/report-name-format:gallio-report\" \"/report-type:Xml\" \"C:\Example\Example.Core.Tests\bin\Debug\Example.Core.Tests.dll\"" "-filter:+[Example.Application]* +[Example.Core]* " -mergebyhash -output:C:\Example\target\sonar\coverage-report.xml

# PartCover
C:\Program Files\PartCover\PartCover .NET 4.0\PartCover.exe --target C:\Program Files\Gallio\bin\Gallio.Echo.exe --target-work-dir C:\Example\target\sonar --target-args \"/r:IsolatedAppDomain\" \"/report-directory:C:\Example\target\sonar\" \"/report-name-format:gallio-report\" \"/report-type:Xml\" \"C:\Example\Example.Core.Tests\bin\Debug\Example.Core.Tests.dll\" --include [Example.Application]* --include [Example.Core]* --output C:\Example\target\sonar\coverage-report.xml

Deactivating Unit Tests and Code Coverage

Add the following line to your analysis configuration file:

sonar.gallio.mode=skip

Integration Tests and Code Coverage

Because the principles are very similar to those for unit tests, please read Unit Tests and Code Coverage.

Letting the SonarQube platform drive the execution of the unit tests and the code coverage tool

To activate this mode, add the following line to your analysis configuration file:

sonar.gallio.it.mode=active

The first step is to tell SonarQube which projects are integration test projects. To do so, set the sonar.dotnet.visualstudio.itProjectPattern property.

# Semicolon-separated list of patterns to identify integration test projects
sonar.dotnet.visualstudio.itProjectPattern=*.Integration.Tests;*.IntegrationTests

Then you just have to run a SonarQube analysis and you'll get data on integration tests and code coverage. The paths to the integration test assemblies are automatically retrieved from the Visual Studio ".csproj" files, and the execution of integration tests and the driving of the coverage tool is automatically performed by Gallio.

Reusing existing reports

To activate this mode, add the following line to your project configuration file:

sonar.gallio.it.mode=reuseReport

Then configure the paths to the integration tests execution report and the code coverage report.

# Paths are relative to the Solution base directory

# Path to the integration tests execution report
sonar.gallio.it.reports.path=reports/it-gallio-report.xml

# Path to the code coverage report
sonar.gallio.it.coverage.reports.path=reports/it-coverage-report.xml

Due to SONAR-4101, execution data for integration tests are not currently stored as such. Thus, the C# plugin merges integration test data with unit test data.

Deactivating Integration Tests and Code Coverage

This is the default mode.

FAQ

My unit tests and integration tests are defined within the same project, how can I run my unit tests only?

Define the sonar.gallio.filter property (or the sonar.gallio.it.filter property for integration tests):

# To run test from the "unit" category only
sonar.gallio.filter=CategoryName:unit

Note that Gallio does not support MSTest categories. Thus it is useless to specify this property when MSTest is used.

How can I exclude some specific projects and/or namespaces from code coverage computation?

Define the sonar.gallio.coverage.excludes property. Note that this property is shared between unit and integration tests.

# List of components to exclude. Syntax is different from one tool to another.

# dotCover filter format
# Exclusions can be defined at any levels: project, class, function
# Example: Excludes class Example.Core.MoneyBag and function Multiply defined in class Example.Core.Money
sonar.gallio.coverage.excludes=-:module=Example.Core;class=Example.Core.MoneyBag;function=*;-:module=Example.Core;class=Example.Core.Money;function=Multiply;

# NCover
# Exclusions can be defined at project level only
# Comma-separated list of projects to exclude
# Example: Excludes all components from projects Example.Core and Example.Model
sonar.gallio.coverage.excludes=Example.Core,Example.Model

# OpenCover
# Exclusions can be defined at project level only
# Comma-separated list of projects to exclude
# Example: Excludes all components from projects Example.Core and Example.Model
sonar.gallio.coverage.excludes=Example.Core,Example.Model

# PartCover
# Exclusions can be defined at project level only
# Comma-separated list of projects to exclude. Syntax: [project]*
# Example: Excludes all components from projects Example.Core and Example.Model 
sonar.gallio.coverage.excludes=[Example.Core]*,[Example.Model]*

SonarQube cannot retrieve the unit test assemblies

See Running an Analysis on a C# Solution, "SonarQube cannot retrieve the assemblies" section.

# If, for each test project, the test assembly is moved to a "bin/package" directory inside the project directory
sonar.dotnet.test.assemblies=bin/package/$(AssemblyName).$(OutputType)
 
# If all the assemblies are copied into a "package" directory located in the solution directory
sonar.dotnet.test.assemblies=$(SolutionDir)/package/$(AssemblyName).$(OutputType)

0% code coverage in reuseReport mode

This might happen if you have generated the reports then moved your solution's files. In the XML coverage files, the paths to the C# source files must be the same as the paths specified in the ".csproj" files, otherwise the coverage data are simply not taken into account.

When using OpenCover with Gallio, I get 0% code coverage, but I do have test classes that are covered by tests

Most probably, you should use the following property to run the tests in an isolated process:

sonar.gallio.runner=IsolatedProcess

As describe in the Gallio documentation, possible values are IsolatedAppDomain (PartCover default runner type), IsolatedProcess and Local. Does not apply to NCover.

Gallio execution fails with following error message

Host process exited with code: -2146232576
A fatal exception occurred while running tests. Possible causes include invalid test runner parameters and stack overflows

This error happens when one of the dotnet config files (app.config, web.config, ...) in your Visual Studio solution specifies a runtime version not supported by the instance of Gallio. This is often the case when the following XML fragment appears in an app.config file:

<startup>
   <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>

The "sku" attribute is not something Gallio understands. If you replace the above XML fragment with the one below, the issue is solved:

<startup>
    <supportedRuntime version="v4.0.30319"/>
</startup>

NCover, the analysis succeeded but only one test project is taken into account

The NCover Complete Edition allows you to run tests from multiple projects and merge the coverage data, the NCover Classic Edition does not.

The workaround is to activate the Gallio safe mode (sonar.gallio.safe.mode=true). Gallio will then be launched for each test project and the generated XML reports will be merged.

How to set Gallio's timeout?

Set the sonar.gallio.timeoutMinutes property (default value is 30).

How can I run Gallio and PartCover on 64-bit Windows?

Gallio and PartCover work perfectly out-of-the-box on x86 Windows, but not on 64-bit versions. To make them work:

  1. Download the latest Gallio x64 installer and install it in a folder, but not under "Program Files".
  2. Download the latest PartCover installer and install it in a folder, but not under "Program Files".
  3. Run corflags.exe on Gallio and PartCover executables:

    # Go to the PartCover folder
    corflags /32bit+ /force PartCover.exe
    corflags /32bit+ /force PartCover.Browser.exe
    
    # Go to the Gallio bin folder
    corflags /32bit+ /force Gallio.Echo.exe
    

See http://www.planetgeek.ch/2009/10/15/get-partcover-running-on-x64-windows/

Gallio execution fails on a Windows 64-bit system

If in the logs you get something like:

[INFO] [23:30:05.643] A fatal exception occurred while running tests.  Possible causes include invalid test runner parameters and stack overflows.
[INFO] [23:30:05.727]   Gallio.Model.ModelException: An exception occurred while invoking a test driver. ---> Gallio.Model.ModelException: Gallio.Model.ModelException: Could not load test assembly from 'C:\whatever\MyTestAssembly.Test.dll'. ---> System.BadImageFormatException: Could not load file...

it might be because some of your assemblies have been compiled with a x86 profile (32-bit) while Gallio is configured to run in an "Any CPU" mode, hence 64-bit on your system.
This problem might be solved using corflags tool to force Gallio to run in a 32-bit mode as shown below:

corFlags.exe Gallio.Echo.exe /32BIT+ /Force

Some tests fail unexpectedly using a legacy .net2 library

If you get this kind of error:

System.IO.FileLoadException: Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information.

It is likely that you are facing the issue described here

The problem might be solved if you modify your installation of Gallio. Add the following XML fragment in files Gallio.Host.x86.exe.config and/or Gallio.Host.x64.exe.config:

<startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0"/>
</startup>