Tuesday August 30th 2011 12am
How to Successfully Build an Automation Program
Software Test Professionals Conference
To effectively implement an automation program, the same approach and similar
effort is needed when building complex software. You will be integrating a system
with many parts as well as software (i.e. scripts and glue software). Additionally,
to complicate things the automation software will be interfacing with another
application (application under test). Therefore the same lessons learned in building
systems and software should be applied to your automation project.
Goals and Objectives
It is vital to define the high level goals of the automation system. The most important
questions to answer are:
Once these questions are answered the next set of items to be identified are the features
you require in the automation system. These include:
- What type of application features are we trying to test - is it performance testing,
unit testing, or functionality testing?
- What is the extent of automation that you are attempting and what is the business need?
- How do you propose to support this automation once implemented?
- Logging capabilities
- Graphical automation scenario creator
- API for remote procedure calls
- Resource management of test/lab resources
- Concurrent scripts
- Capability to prepare or “clean up” the test environment
- Versioning of test scripts
- Documentation for administration/test script creation
Designing and Documenting Automation Architecture
Once these items are identified then you can design the automation framework.
The automation framework lays the foundation and rules for the automation toolset.
For example, a shared software library for automation scripts should be implemented
and used by all automation script writers. Similar points are listed below:
- Extensibility: Automation requirements will grow in step with the
requirements of the software under test, and the automation system must
be able to cope with that growth.
- Instrumentation for debugging purposes: Running scripts is the easy
part. Debugging scripts is what takes the most time. There should be a
system in place that quickly summarizes all the problems from a test
- Scripts should be modularized.
- Maintaining statistics of executions: Identify test cases with a
high false positive rate and either fix the root cause or remove from
your executions. Conversely, identify tests that find real software
- Automation scripts re-usability: Could scripts developed for feature
testing later be re-used for integration testing or performance testing.
Could other groups re-use the scripts?
Once you have the automation framework defined then pick a tool to develop
the automation framework. Open source, commercial or in-house tools are your
typical options based on whether the tool fits current and future requirements.
Other factors in tool selection include:
- Support requirements to maintain the application
- Average time required for a feature request or bug fix to be implemented
- If open source, is the community active in developing features and fixing
- If the software is home grown do you have developers in-house to support the product?
Full-time resources must be qualified and fully committed to building and maintaining
an effective automation program. When staffing up for an automation project create a
list of job requirements for the new team members. If the focus of the team is divided
between multiple complicated projects, it will reduce the chances of success of the
Automation programs contribute substantially to reducing the cost of testing and improving,
but it is critical to plan them carefully with the same rigor as you would in software
About the Author
Sreekanth Singaraju, VP of Testing Services for Alliance Global
Services, has more than 12 years of senior technology leadership experience and leads
Alliance's QA & Testing organization in developing cutting edge solutions. He will present
Integrated Test Automation for Enterprise Mobile Apps
at the Software Test Professionals Conference Fall 2011.
Alliance Global Services is a software development firm that partners with software, technology and information-intensive businesses on their mission critical work. Alliance architects and builds software applications, platforms and products that become primary drivers of innovation and revenue growth for its clients and their businesses. Alliance is recognized for driving quality and speed-to-market when business success depends on the software inside. For more information, please visit www.allianceglobalservices.com.