Execution Issues
Jump to navigation
Jump to search
Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it.
The Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.
Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues
[1] Suggested by Thorsten Schönfelder
The Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.
| Issues |
Description |
|---|---|
| FALSE FAIL |
The tests fail not because of errors in the SUT, but because of errors in the test automation testware or environment issues |
| FALSE PASS |
The tests pass even if the SUT actually reacts erroneously |
| FLAKY TESTS |
The automated tests seem to pass or fail randomly |
| INADEQUATE RESOURCES |
The hardware or the network are too slow |
| INEFFICIENT EXECUTION |
Execution of automated tests is too slow or too cumbersome. |
| INEFFICIENT FAILURE ANALYSIS |
Failure Analysis is slow and difficult |
| INFLEXIBLE AUTOMATION |
Execution of automated tests is ad-hoc or too rigid. Subsets of tests cannot be selected to run - it's "all or nothing". |
| LITTER BUG [1] |
Tests leave spurious data in databases, history etc. that slows performance or hinders further tests. |
| MANUAL INTERVENTIONS |
Test automation needs lots of manual handlings to work |
Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues
[1] Suggested by Thorsten Schönfelder