Currently, Maven only supports unit testing out of the box. What we want is something flexible enough to work without having to make test definitions arbitrary.
Types of Tests
- run before package
- want 100% success before moving on generally
- require the artifact, so run after package but before install.
- generally want 100% before moving on.
- May use junit or other plugin like cactus, including deploying to a server
- also called functional or regression tests.
- not run every time, probably a report and/or part of a profile.
- run after install, before deploy
- Regression tests (acceptance tests from previous milestones) should be at 100%
- acceptance (also called functional) tests only need to get to 100% at release time.
- performance/stress tests - another form of acceptance test.
- plugin integration tests (c.f. eclipse plugin)
- report integration tests (see Vincent S's work - htmlunit)
- cactus plugin
- geronimo itest plugin
- need to be able to compile additional sources, have own dependencies (basically reapply the whole test lifecycle again - given that this also matches the main one, perhaps that can be reusable/forked)
- we should have a setup and teardown phase around the actual test goal
- may want to reapply test stuff on other test sets (eg junit report, clover)
separate project is ideal, but
- a bit unnatural, more strung out
- doesn't fit IDE well
- you can still use it if you support in project tests, so better to do that
While a separate project might be something we recommend, I think we need to support them within the same project.