TESTABLE SOFTWARE
Pattern Summary
Locate what kind of implementations in the Software Under Test (SUT) may make test automation difficult or impossible, and find a solution as early as possible.
Category
Management
Context
Use this pattern when you start test automation from scratch
You will not need it to just write disposable scripts
Description
Find out as soon and as fast as possible what could be a problem in order to make sure that the SUT will support automated testing from the very beginning.
Both for a new project and for an older application, you should ASK FOR HELP from the developers. When the problem is understood, it is usually not difficult to find a work-around that works for both testers and developers.
If something is impossible to automate, or not economically worth the effort to automate it, make sure that these are tested manually, and try to see if it might be possible to write the software in a way that could be tested automatically in the future.
Implementation
SHARE INFORMATION with the developers to discuss the problem with them:
- Find out what kind of components they are using. Contact the tool vendor to ask for support.
- If the results keep changing, examine how you are recording them. If you check them directly in the GUI the results will be much more sensitive to eventual changes in the SUT than if you check them by extracting the expected data from the database tables
- If the time waiting for responses from the SUT is variable, build synchronization points in your scripts. For instance have the script wait until the hourglass disappears or a button is enabled.
- Sometimes small adjustments in the code of the SUT can solve the problem:
- Make sure that every object you need is uniquely named
- When some computation takes time, ask the developers to mark in some way when it is finished. For instance a disabled button will be enabled at the end
Recommendations
DO A PILOT to find out what kind of implementations may cause trouble.
Changes to the SUT or in the way it is implemented may significantly affect the automation. Developers should be made aware of this, and MANAGEMENT SUPPORT may be needed to ensure the right balance between essential changes and ensuring that the software is testable and automatable.
SELL THE BENEFITS:
- show management which returns would be possible at what costs
- show developers the advantages of getting timely feed-back through test automation
List the “Do NOTs” in a Wiki so that developers can check which programming practices or which components do not support test automation.
Try to get at least some automation running as fast as possible. As soon as the developers start getting good feedback from the automated tests, they will be much more willing to help support it.
Potential problems
If you ascertain that the tool just doesn’t fit your application, then you should select another one (RIGHT TOOLS)
Issues addressed by this pattern
HARD-TO-AUTOMATE
MANUAL INTERVENTIONS
INADEQUATE TOOLS
Experiences
If you have used this pattern and would like to contribute your experience to the wiki, please go to Experiences to submit your experience or comment.