EASY TO DEBUG FAILURES
Pattern summary
Each test failure should make it obvious which test failed and what went wrong. It shouldn't take hours to figure this out and it shouldn't require a rerun of the test.
Category
Execution
Context
This pattern is useful for big and small automation undertakings.
Description
Develop simple clear well documented tests. Design the automated test so that it keeps track of the information you would want to know when the test fails, for example, capture screen shots on a regular basis, or log all messages that are output, or record all the input data in one place. Automate the clear up such that all the test results and associated information is available in one place and is easily accessible to the person investigating the reason for the failure.
Implementation
How to implement this pattern depends on your context. Here are some suggestions:
- develop tests with ONE CLEAR PURPOSE: if such a test fails it will be easier to understand why
- implement INDEPENDENT TEST CASES: in this way you can be sure that the failure is not caused by an earlier test
- if many tests are failing for the same (minor) bug, implement EXPECTED FAIL STATUS
- if you write SINGLE PAGE SCRIPTS you will know at a glance in which page or window the failure occurred
- and last but not least: KEEP IT SIMPLE!
Potential problems
If you do need to rerun your tests for better checking, implement ONE-CLICK RETEST
Issues addressed by this pattern
HARD-TO-AUTOMATE
INEFFICIENT FAILURE ANALYSIS
Experiences
If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.