Added by Jacques Morel, last edited by Jacques Morel on Nov 04, 2008  (view change)

Labels:

Enter labels to add to this page:
Wait Image 
Looking for a label? Just start typing.

Comparaison

Similarities

Feature JUnit4 TestNG
test annotation @Test @Test
test method setUp equivalent @Before @BeforeTest
test method tearDown equivalent @After @AfterTest
test class setUp @BeforeClass @BeforeClass
test class tearDown equivalent @AfterClass @AfterClass
ignore test @Ignore @Test(enabled=false)

Differences

The major difference between JUnit4 and TestNG is that

  1. JUnit4 is a drop in replacement of JUnit3: run mixed JUnit3 & JUnit4 tests in the same test run. TestNG requires source migration (See below).
  2. TestNG allows state sharing between test in the same class: i.e. only one instance of a test class will be used to run all test methods of that class. JUnit enforces absolutely no sharing of instance fields between tests.

TestNG fixture sharing makes significantly easier sharing of expensive fixtures like spring application context, database, web servers...
Combined with TestNG's multiple test grouping levels with associated setup and teardown methods, it provides a very powerful base for building longer than straight unit tests (integration, system, functional):

Group level Defined in Set up/Tear down annotation When run
test testng.xml @BeforeTest/@AfterTest Run before and after a test is run. In TestNG a test is not a test method. A test is a set of test classes/methodes defined in the testng.xml. It may span multiple classes and may not include all methods of these classes
group annotation @Test(groups={"mygroup"}) @BeforeGroup/@AfterGroup Before the first test method of the group is run and after the last is run. Note: Here is the order of calls:@BeforeClass=>@BeforeGroups=>@BeforeMethod=>@Test=>@AfterMethod=>@AfterGroups=> @AfterClass
suite testng.xml @BeforeSuite/@AfterSuite Run before the first and after the last test methods of a suite.

JUnit4

  • Backward compatible. Run your JUnit3 test through JUnit4 and run JUnit4 as JUnit3 tests in all tools (IDE, ant...)
  • Support for parametrized tests at the class level (JUnit4 supports only one set of parameters and values specified to the tests through a custom constructor. This means that all test methods are run for each value set)
  • Flexible Runner framework to allow custom runner to run a test without forcing a base class to insert custom behavior (Spring, JMock uses it). However it only works one time. Could not find a way to combine runners (like using spring di with jmocks)
  • Slightly better IDEA integration: TestNG plugin loses exception stacks after navigating the test tree. JUnit plugin has a memory delta report.

TestNG

  • Suites are optionally defined in an xml file testng.xml
  • Grouping of test: categorize your test through annotation. You can then have in one test class tests that run in different suites like check-in, db, functional... This is nice to keep all your unit and integration tests pertaining to one class in the same test class. For example TestUserStory could contain
    • tests that test the hibernate mapping and the db queries. These would run in the db or integration suites
    • tests that test the domain logic of the class. These would run in the unit or checkin suites
  • Run last failures
  • 2 additional setUp/tearDown level: suite and group (@Before/AfterSuite, @Before/AfterGroup). See more details here
  • Support for parametrized tests at the test level (Unlike JUnit4, TestNG allows to supply a different source (or none) for each test method. Parametrized test methods receives naturally their parameters through... parameters. Rather cleaner than JUnit4)
    I am still not sure that the introduction of an XML definition of suites and test is a good thing. I understand the fact that you don't have to recompile to change a suite but how much pain does compiling one file cause?
  • @Test may be put on a class and all public method will be considered test methods.
  • Can run tests in parallel with a configurable number of threads (while forcing some test class tests to run serially) (from testng.xml)
  • Dependencies between tests
    • hard: stop dependent tests if dependency failed,
    • soft: always run even though dependency may have failed)
  • Automatic tool to convert junit test to testng.
  • IDEA7 has a nice built-in support for testNG including an intention to convert junit test to testng.
  • maven support
  • Backward compatible with junit test through the testng.xml (<test junit="true">). Question: will it run TestSetUp properly? However IDEA does not honor this property and only run true TestNG tests only.

References