TOO EARLY AUTOMATION

.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues

Issue summary

Test automation starts too early on an immature application or on the wrong aspect of an application, and produces only “noise”.

Category

Design

Examples

1. Since the SUT is not stable yet, you have to update the automation so often that you don't make any progress. You are always only tinkering with the same scripts.
2. Management has suspended development for the SUT. All your work was a waste of time.
3. The aspects you are trying to automate keep changing and the automation can't cope with it.

Questions

How stable is the SUT? Is there some part that is already "testable"?
How about the test cases? Do they change just as often?
Are you trying to automate aspects that are least stable, while other aspects could more usefully be automated?

For example, trying to automate tests through a GUI when the GUI design is not yet stable is asking for trouble. However, it may not be too early to automate tests through the API or at unit test level, or to put in the foundations for tests to be written later on using a high-level abstraction approach.

Resolving Patterns

Most recommended:

• WHOLE TEAM APPROACH: if development already uses an agile process then this is the pattern of choice to solve this issue.
• If the instability is due to superficial aspects, rather than fundamental design issues, you could SHARE INFORMATION about the impact of these changes on the automation, and possibly use MANAGEMENT SUPPORT in order to stabilise unnecessary changes.
• ABSTRACTION LEVELS: it may be too early to automate at the user interface level, but you may be able to design your TESTWARE ARCHITECTURE so that there are useful and long-lasting scripts you can develop now. Also, you can devise a way for tests to be written even if the user interface is not yet stable, using DOMAIN-DRIVEN TESTING and/or KEYWORD-DRIVEN TESTING.

Other useful patterns:

• AUTOMATE GOOD TESTS: this pattern helps you select what to automate.
• DO A PILOT: this pattern can be used to explore which aspects would be best to automate, and what should be automated early and what is better left until later.

.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues