Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

Currently, Maven only supports unit testing out of the box. What we want is something flexible enough to work without having to make test definitions arbitrary.

Types of Tests

Unit Tests

  • run before package
  • want 100% success before moving on generally

Integration Tests

  • require the artifact, so run after package but before install.
  • generally want 100% before moving on.
  • May use junit or other plugin like cactus, including deploying to a server

Acceptance Tests

  • also called functional or regression tests.
  • not run every time, probably a report and/or part of a profile.
  • run after install, before deploy
  • Regression tests (acceptance tests from previous milestones) should be at 100%
  • acceptance (also called functional) tests only need to get to 100% at release time.
  • performance/stress tests - another form of acceptance test.

Use cases

  • plugin integration tests (c.f. eclipse plugin)
  • report integration tests (see Vincent S's work - htmlunit)
  • cactus plugin
  • geronimo itest plugin
  • need to be able to compile additional sources, have own dependencies (basically reapply the whole test lifecycle again - given that this also matches the main one, perhaps that can be reusable/forked)
  • we should have a setup and teardown phase around the actual test goal
  • may want to reapply test stuff on other test sets (eg junit report, clover)

separate project is ideal, but

  • a bit unnatural, more strung out
  • doesn't fit IDE well
  • you can still use it if you support in project tests, so better to do that

While a separate project might be something we recommend, I think we need to support them within the same project.

...