Review of Plugin Testing Strategies

Able to stage local repository state

Able to compile custom sources

Able to add custom dependencies

Supports pre-build hook

Supports verification

Supports post-build hook for tearDown

Beanshell support

Groovy support

Capture Maven exit code and exceptions

Supports ANSI output

Can ignore failures

Can skip execution

Can stage or use a pre-staged local repository

Runs with Surefire

Generates JUnit XML

Can measure coverage using standard techniques

Can be used for contextual integration testing

Run tests against multiple versions of Maven

Run tests against multiple versions of the plugin

Forked Maven

Embedded Maven

Does not pollute source tree on execution

Cleans up anything stored outside target directory

Includes local repository and system temporary directory

Uses unique storage location for all execution files

This is a concurrency issue if it uses the same location.

Isolated from user's environment/settings by default

Can be achieved by passing a separate settings file for those that support it.

Able to run from the IDE regular test functionality

Able to run offline (unless plugin requires it)

Offline by default

Able to run a single test

Able to run wildcard/regex set of tests

Able to run a suite of tests

Able to run in a separate Maven project

Able to run in the same Maven project

Fast execution

Maven home is configurable

Configurable Maven settings file

Configurable individual Maven settings

Configurable working directory

Configurable input/output handlers for Maven

Configurable Maven logger

Can select which projects to run (include/exclude)

Able to run Maven in batch mode

Able to run Maven in debug mode

Able to inherit shell environment

Pass goals to run Maven with

Pass goals to run Maven with in a file

Interpolate goals to run Maven with

Pass properties to run Maven with

Pass properties to run Maven with in a file

Interpolate properties to run Maven with

Pass profiles to run Maven with

Pass profiles to run Maven with in a file

Pass flags to run Maven with

Pass flags to run Maven with in a file

Able to run tests in parallel

Able to timeout tests run in parallel