About STP / 877.257.9531
Log In Join Now

Author



Rating

7


Published

Thursday July 14th 2011 1pm

Eclipse for Testers

Technology Software Agile Performance Services Development Functional Integration Open Source Process Testing Unit Mobile Web Tools

Eclipse has been one of the best IDEs for unit testing ever since its early days. Not too great a surprise, as until a few months ago, JUnit co-founder Erich Gamma was also lead engineer for Eclipse Java Development Tools and related IBM products like Rational Jazz. From Version 3.2 on, JUnit was directly built into Eclipse after a separate plug-in has already been available.

JUnit


Figure 1 – Eclipse JUnit Runner

Once JUnit was supported out of the box by Eclipse this lowered the barrier to give Test Driven Development a try in many projects and teams. Thanks to a JUnit Runner view (see Figure 1) allowing both execution and debugging of Unit Tests at the click of a button.

Soon other test tools were added to the Eclipse Foundation.

Test & Performance Tools Platform

TPTP, the Test & Performance Tools Platform, earlier code-named Hyades started soon after Eclipse had started to gain industry momentum. Release 1.0.3 was published in September 2003. The main focus was on Profiling, but over the years it spread across these sub-projects:
  • Platform
  • Testing Tools
  • Trace & Profiling Tools
  • Monitoring Tools
Commercial products by contributors like IBM were based on TPTP, especially Profiler tools for Java and WebSphere. Over the years TPTP matured and parts of it were gradually included with the free Eclipse distributions. Allowing a Java application to be almost as easily profiled as to run or debug it.

However, given decreased contributions by project members and little improvements in new versions, Eclipse Foundation has decided to archive TPTP after its Helios Release Train last year. TPTP 4.7 was the last version before active development was stopped. TPTP also wasn’t included with this year’s Indigo Release Train any more. It may still be installed manually where required, and given the backward compatibility of most Eclipse API it should work for some time, at least in the 3.x releases.

IBM has handed TPTP leadership to verit Informationssysteme Gmbh, a German company specialized in QA, test automation and test management. Whether or not this brings parts of TPTP back into official Eclipse packages in years to come remains to be seen, but it means, existing users of TPTP can be hopeful not to lose support and may even find more support even if it is at the cost of a commercial license or subscription.

Especially for test and QA professionals this is a development to keep an eye on, and we’re likely to see more on future occasions, either EclipseCon or special events like Eclipse Testing Day, September 7th 2011 in Germany.

I smell a RAT

While test automation and testing tools under the TPTP umbrella are likely to be continued by the new project lead, the area of performance testing, monitoring or profiling at Eclipse received a contribution by Google in form of RAT, the Runtime Analysis Tool.

The project has just been created at Eclipse, so there still is little documentation beside the project proposal. The basis for RAT is a product named CodePro Profiler™, by Instantiations Inc., a vendor Google recently purchased. The company existed since the early days of Java and the project or product now contributed to RAT has also been around since 2006, so once published at Eclipse some maturity level can be assumed. Eventually, like Google’s other main contribution, WindowBuilder it can be expected to become part of release trains like Juno a year from now.

Jubula

While some projects were archived and are no longer available by default, this year’s Eclipse Indigo Release Train contains a great gem for test professionals, a dedicated Eclipse for Testers package. This isn’t a traditional Eclipse distribution as most are commonly known. Beside Mylyn for Test and Project Planning there are only a few perspectives familiar from SDK style Eclipse. Beside those, Jubula introduces 3 perspectives:
  • Functional Test Specification
  • Functional Test Execution
  • Functional Test Reporting
Although loading an Eclipse workspace tests aren’t stored there at all. Instead Jubula comes with an in-memory H2 database located in a folder .jubula/databasexpe under the user’s “HOME”.

Eclipse default usability and standard menu structure is somewhat broken by introducing a separate “Test” menu under which all relevant operations can be found, including that for a new test project.


Figure 2 – New Test Project

Figure 2 shows the first step for creating a new test project. Using Eclipse and Java i18n, the content for e.g. selecting project languages is fully localized. Presenting German terms on a German language OS. The remaining UI is English, unless one applied Eclipse Babel NLS packages where available to translate the majority of Eclipse elements into languages like German, too.


Figure 3 – Application Under Test

Figure 3 shows the next step: defining an Application Under Test (AUT) for the test project. AUT is a variation of SUT (System Under Test) known by many test systems.

An important choice on this page is one of the following (UI) Toolkits:
  • HTML
  • SWT
  • RCP
  • Swing

Figure 4 – AUT Details

The page shown in Figure 4 allows specifying the type of AUT and as in this example for HTML, further details like the browser to use. Some of these may sound familiar to users of test automation frameworks like Selenium or WebDriver.

Figure 5 shows the main parts of the Functional Test Specification Perspective with a test suite and test job defined.

Due to licensing terms the Jubula team promised to address by Indigo SR1 after this summer the “AUT Agent” Service needs to be installed separately into Indigo Testing Package. This service plugs itself into the target application somewhat similar to a remote debugger.

Note, at least on Windows you might have to start the agent as Administrator. A precondition, users of Selenium or Concordion testing frameworks also know on Windows OS.


Figure 5 – Functional Test Specification

Once the AUT Agent has been started and the AUT is accessible, the test suite can be executed, shown in Figure 6.


Figure 6 – Test Execution

As seen in Figure 7 test results are represented in a view somewhat similar to that of plug-ins like for JUnit mentioned earlier. This makes using functional tests from Eclipse for Testing comparable to running other types of tests within different Eclipse packages.

This package is clearly aimed at test managers or test engineers creating tests in a black box or TDD manner without having to know more than a functional specification or wireframes for an AUT.

While some users may find having to switch to a different Eclipse package inconvenient, this separation of roles and the reduced download size of the Testing Package compared to other pre-packaged Eclipse editions justify the scope of the features included. Using Update Sites or Eclipse Marketplace anybody who needs e.g. a Java EE, Spring or RCP perspective can add the necessary bundles to this Eclipse package. Google’s other main contribution; Eclipse WindowBuilder also finally adds improved Swing GUI support for this type of AUT.


Figure 7 – Test Result

WindowTester

Beside TPTP, Google not only contributed WindowBuilder to Eclipse, it also provided a few other products by purchased vendors as Open Source projects. Complimentary to WindowBuilder, but instead of “pure” HTML aimed more at Google’s GAE and GWT, is WindowTester (Pro). Currently the “Pro” version is free and no obvious difference visible between editions, but there might be an extended “Pro” version for Google subscribers and Enterprise customers at some point soon. Plans in this direction were mentioned for its GAE or GWT products already, so it seems logical to follow in other areas like GAE and GWT Test Tools. Whether or not parts of WindowTester get contributed to Eclipse is unclear. There might be parts following WindowBuilder, but even that was split into a Swing/SWT side now with Eclipse and a GWT side Google kept under its own umbrella. Jubula and Eclipse for testing are a great step in the right direction, but in some areas, it is not just about UI and usability, there’s room for improvement. So Google might be welcome to either participate there or even provide additional plug-ins to future versions of Eclipse for testers.

Other Testing Tools and Integration

There are various other testing tools available either as Eclipse.org projects or features and products on top of Eclipse. Mobile developers and testers should find additional support through specialized parts of Eclipse Sequoyah or derived products like MotoDevStudio by Motorola.

For OSGi developers other tools are provided in support of RT projects like Equinox or Virgo, as well as special Embedded testing frameworks aimed at OSGi’s original target groups like Embedded Devices or Automotive.

Most of these allow integration with build systems like Ant or Maven, and common CI tools like Hudson or Jenkins. The fact, that Hudson has just been created as Eclipse project following Oracle’s code contribution makes it the CI server of choice, but most tools and frameworks will be open to more than just one alternative, both inside and outside Eclipse.org.

About the Author:
Werner Keil Werner Keil is an Agile Coach and Principal Consultant at emergn, a company specialized in Value, Flow and Quality using Agile, Lean Principles. Helping Global 500 Enterprises across industries like Mobile/Telco, Web 2.0, Finance, Travel/Logistics, Automotive, Healthcare, Environment & Public Services, as well as IT vendors like Oracle or IBM. He designed and implemented micro-format based tags for the online music portal of a media world leader.

He has worked for more than 20 years as a project manager, software architect, analyst and consultant on leading-edge technologies for Banking, Insurance, Telco/Mobile, Media and Public sector. Werner develops enterprise systems using Java, JEE, Oracle, IBM or Microsoft, does Web design and development using Adobe, Ajax/JavaScript, other dynamic or functional languages. Besides working for major companies, he runs his own creative and talent agency Creative Arts & Technologies and in his spare time runs and supports Open Source Projects, writes song lyrics, novels, screenplays and technical articles.

Werner is a committing member of the Eclipse Foundation, Babel Language Champion (German), UOMo Project Lead and active member of the Java Community Process, including his membership in JSRs like 321 (Trusted Java), 331 (CP), 333 (JCR), 342 (Java EE 7), 348 (JCP.next) and the Executive Committee (SE/EE).


Comments

You must be logged in to comment.
Retrieving Comments...


Advertisement



Friend SoftwareTestPro on Facebook
Follow @SoftwareTestPro on Twitter
Create or Join a Crew

Explore STP