Difference between revisions of "Execution Issues"

From Test Automation Patterns
Jump to navigation Jump to search
Line 1: Line 1:
<div id="content_view" class="wiki" style="display: block"><span style="font-family: Arial; font-size: 16px">Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it</span>.<br /> <span style="font-family: Arial; font-size: 16px">The [[Execution%20Issues%20Mind%20Map Execution Issues Mind Map]] shows an overview of the Execution Issues, </span><span style="font-size: 16px">together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.</span><br /> <span style="font-family: Arial; font-size: 16px">.</span><br /> <br />  
+
<div id="content_view" class="wiki" style="display: block"><span style="font-family: Arial; font-size: 16px">Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it</span>.<br /> <span style="font-family: Arial; font-size: 16px">The [[Execution Issues Mind Map]] shows an overview of the Execution Issues, </span><span style="font-size: 16px">together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.</span><br /> <span style="font-family: Arial; font-size: 16px">.</span><br /> <br />  
 
{| class="wiki_table"
 
{| class="wiki_table"
 
! <span style="display: block; font-size: 16px; text-align: center">Issues</span><br />
 
! <span style="display: block; font-size: 16px; text-align: center">Issues</span><br />

Revision as of 14:00, 26 April 2018

Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it.
The Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.

Issues
Description
FALSE FAIL
The tests fail not because of errors in the SUT, but because of errors in the test automation testware or environment issues
FALSE PASS
The tests pass even if the SUT actually reacts erroneously
FLAKY TESTS
The automated tests seem to pass or fail randomly
INADEQUATE RESOURCES
The hardware or the network are too slow
INEFFICIENT EXECUTION
Execution of automated tests is too slow or too cumbersome.
INEFFICIENT FAILURE ANALYSIS
Failure Analysis is slow and difficult
INFLEXIBLE AUTOMATION
Execution of automated tests is ad-hoc or too rigid. Subsets of tests cannot be selected to run - it's "all or nothing".
LITTER BUG [1]
Tests leave spurious data in databases, history etc. that slows performance or hinders further tests.
MANUAL INTERVENTIONS
Test automation needs lots of manual handlings to work

Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues


[1]] Suggested by Thorsten Schönfelder