LOOK FOR TROUBLE
Pattern summary
Keep an eye on possible problems in order to solve them before they become unmanageable
Category
Process
Context
Use this pattern regularly in any context. How often depends on the size of your automation effort or on the agility of your development environment (every day, every build, every iteration, every month and so on)
Description
Control regularly the health of your test automation so that you can solve eventual problems proactively
Implementation
To access how "healthy" your automation is, ask your team regularly and review the measurements you have put in place to see if you are achieving your automation goals. The value of the information lies not in the actual answers or numbers, but in seeing trends, and whether or not you are making progress towards your goals. You should use the information as a gauge that tells you when it would be sensible to go to the Diagnostic
Here a short list of possible topics to keep an eye on:
- Does everybody on the automation team know at least the basics of automation and of tool use?
- Is management up-to-date on the state of the automation effort?
- Are the automation goals (still) valid?
- Does management (still) support automation?
- Does the current tool (still) drive the SUT (System under Test) to complete satisfaction?
- Are the updates to the automation testware due to changes to the SUT (still) manageable?
- Is the number of automated test cases increasing at the expected rate?
- Are the automation stakeholders satisfied?
Potential problems
Remember: finding out where the problems are just increases frustration if you don't go on and solve them!
Issues addressed by this pattern
AUTOMATION DECAY
STALLED AUTOMATION
TOOL-DRIVEN AUTOMATION
Experiences
If you have also used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.