Watir (Web Application Testing in Ruby)

Watir

Licence

OpenSource

How it works

Watir uses Ruby, object oriented scripting language, to drive the Internet Explorer web browser.
In fact, Watir is a Ruby library that wraps the COM interface to Internet Explorer. Like most languages, Ruby can access COM interfaces. The COM interface to IE allows access to the browser's document object model (DOM).
It works with pure html, and in limited way with JavaScript and AJAX
It will not work with ActiveX plugin components, Java Applets, Macromedia Flash, or other plugin applications

Pros:

• Runs test from the browser site. Watir drives the Internet Explorer browser the same way people do.
• We can interact with the system directly, using Ruby (for instance we can access database)
• Ruby has an xUnit framework called test::unit that we can use with Watir. We can use unit convention to create test methods and test cases. We can use unit assertion as well

Cons:

• It works only with IE, on Windows. There is no possibility to check if the application is supported by other browsers
• It uses Ruby, yet another language developers have to learn.
• Have limited support for JavaScript. From the Watir User Guide:

...There has been some experimental code packaged with Watir to deal with popups with limited success. A good popup handling solution is being worked on and will be included in a future Watir release...

Sample code

Has been checked into SVN: https://svn.codehaus.org/xplanner/trunk/spikes/watirExample. The suite contains two tests. The first one is simple XPlanner project creation, the second test shows interaction with an AJAX page

Selenium

Selenuim

Licence

Apache 2.0 license

How it works

Selenium tests run directly in a browser, just as real users do. Installed with your application webserver, Selenium automatically deploys its JavaScript automation engine the Browser Bot to your browser when you point it at the Selenium install point on your webserver. Selenium uses JavaScript and Iframes to embed a test automation engine in your browser. Because different browsers handle JavaScript somewhat differently, they usually have to tweak the engine to support a wide range of browsers on Windows, Mac OS X and Linux. Tests can take one of two forms. They can be embedded in an HTML table, which will be read in by the browser bot and then executed. You can also use programming language, taking advantage of language-specific drivers that communicate in Selenese (Selenium command language) to the browser bot. So far Java and Ruby drivers have been released.

Pros:

• According to documentation Selenium-driven tool seems to be great in terms of testing application behaviors against different browsers like IE, Mozilla or Firefox on different platform. The Selenium-html works.
• As for Ajax testing the waitForValue and waitForCondition actions are useful - see articles below.

Cons:

• There are a lot of problems with writing Selenium Java test under Selenium-driven. The documentation is very poor in that field and the developer forums don't give any reasonable answers.
• We had to go through Selenium-driven integration tests to find out how to set-up and run Selenium-driven test. We had hard time trying to run XPlanner and XPlanner test in that environment. The same tests in Selenium-html works ok but the Selenium-driven binaries that you can download from Selenium website seems not to work. The first problem we got the Tomcat server couldn't be instantiated as we got timeout exception constantly. To fix the problem with instantiating Tomcat from the Selenium process we replaced cargo-0.4.jar with newer version (cargo-core-uberjar-0.7.jar and cargo-ant-0.7.jar) and downloaded the Selenium-driven sources to change the implementation of ServletContainer class (OutbeddedTomcat.java) accordingly. The above let us instantiate Tomcat with XPlanner and Selenium-driven applications in it. However, the actions like selenium.type("id","text") keep throwing Internal Server Exception (500) and stay in some loop for ever even if we can do the same action manually in Selenium-driven XPlanner frame!
• The release the you can download from Selenium download page is pretty old - 9/25/2005

Sum up:

To sum up Selenium-driven for Java developers seems not to be stable and ready for functional test using yet.

Sample code

Please see the implemented code in https://svn.codehaus.org/xplanner/trunk/spikes/seleniumExample. The XplannerSimpleTest.java class defines Selenium-driven setup and two tests for creating and deleting a simple Xplanner project. The OutbeddedTomcat.java class was refactored but the previous version was stored as OutbeddedTomcatOld.java.

Interesting articles

Using Selenuim to test AJAX calls
Selenium driven by java movie
Watir and Selenium, Bret Pettichord's blog, February 2005