TEST AUTOMATION FRAMEWORK
Pattern summary
Use a test automation framework.
Category
Design
Context
This pattern is appropriate for long lasting automation.
If you just plan to write a few disposable scripts you will not need it.
Description
Using or building a test automation framework helps solve a number of technical problems in test automation. A framework is an implementation of at least part of a testware architecture.
Useful tip from James Tony: Select your test framework with care – a wrong decision (like use of “make” to run tests) can take a long time to fix further down the line.
Implementation
Test automation frameworks are included in many of the newer vendor tools. If your tools don’t provide a support framework, you may have to implement one yourself.
Actually it is often better to design your own TESTWARE ARCHITECTURE, rather than adopt the tool's way of organising things - this will tie you to that particular tool, and you may want your automated tests to be run one day using a different tool or on a different device or platform. If you design your own framework, you can keep the tool-specific things to a minimum, so when (not if) you need to change tools, or when the tool itself changes, you minimise the amount of work you need to do to get your tests up and running again.
The whole team, developers, testers, and automators, should come up with the requirements for the test automation framework, and choose by consensus. If you are comparing two frameworks (or tools) use SIDE-BY-SIDE to find the best fit for your situation.
A test automation framework should offer at least some of the following features:
- Support ABSTRACTION LEVELS
- Support use of DEFAULT DATA
- Support writing tests
- Compile usage information
- Manage running the tests, including when tests don't complete normally
- Report test results
You will have to have MANAGEMENT SUPPORT to get the resources you will need, especially developer time if you have to implement the framework in-house.
Potential problems
It is not necessarily easy to acquire or make a good test automation framework, and it does take effort and time. But it is very worthwhile when done well.
If you intend to develop a framework in-house make sure to plan for the necessary resources (developers, tools etc.) otherwise you could end up with a good framework that must be abandoned because for instance the only developer leaves the company (see issue KNOW-HOW LEAKAGE for more details).
Issues addressed by this pattern
HIGH ROI EXPECTATIONS
INADEQUATE TOOLS
MULTIPLE PLATFORMS
NON-TECHNICAL-TESTERS
STALLED AUTOMATION
Experiences
If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.