Execution Issues

Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it.
The Execution Issues Mind Map Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.

Issues	Description
FALSE FAIL	The tests fail not because of errors in the SUT, but because of errors in the test automation testware or environment issues
FALSE PASS	The tests pass even if the SUT actually reacts erroneously
FLAKY TESTS	The automated tests seem to pass or fail randomly
INADEQUATE RESOURCES	The hardware or the network are too slow
INEFFICIENT EXECUTION	Execution of automated tests is too slow or too cumbersome.
INEFFICIENT FAILURE ANALYSIS	Failure Analysis is slow and difficult
INFLEXIBLE AUTOMATION	Execution of automated tests is ad-hoc or too rigid. Subsets of tests cannot be selected to run - it's "all or nothing".
LITTER BUG [1]	Tests leave spurious data in databases, history etc. that slows performance or hinders further tests.
MANUAL INTERVENTIONS	Test automation needs lots of manual handlings to work

Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues


[1]] Suggested by Thorsten Schönfelder