Difference between revisions of "Execution Issues"
Jump to navigation
Jump to search
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px">Return to issue ''NO PREVIOUS TEST AUTOMATION''</span><br /> <br /> <span style="f...") |
m (Contour lines added) |
||
| (3 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
| − | <div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px"> | + | <div id="content_view" class="wiki" style="display: block"><span style="font-family: Arial; font-size: 16px">Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it</span>.<br /> <span style="font-family: Arial; font-size: 16px">The [[Execution Issues Mind Map]] shows an overview of the Execution Issues, </span><span style="font-size: 16px">together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.</span><br /> <span style="font-family: Arial; font-size: 16px">.</span><br /> <br /> |
| + | {| class="wiki_table" border="1" style="border-collapse:collapse" | ||
| + | ! <span style="display: block; font-size: 16px; text-align: center">Issues</span><br /> | ||
| + | ! <span style="display: block; font-size: 16px; text-align: center">Description</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-family: Arial; font-size: 16px">[[FALSE FAIL]]</span>''<br /> | ||
| + | | <span style="font-family: Arial; font-size: 16px">The tests fail not because of errors in the SUT, but because of errors in the test automation testware or environment issues</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-family: Arial; font-size: 16px">[[FALSE PASS]]</span>''<br /> | ||
| + | | <span style="font-family: Arial; font-size: 16px">The tests pass even if the SUT actually reacts erroneously</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-family: Arial; font-size: 16px">[[FLAKY TESTS]]</span>''<br /> | ||
| + | | <span style="font-family: Arial; font-size: 16px">The automated tests seem to pass or fail randomly</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-size: 16px">[[INADEQUATE RESOURCES]]</span>''<br /> | ||
| + | | <span style="font-size: 16px">The hardware or the network are too slow</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-size: 16px">[[INEFFICIENT EXECUTION]]</span>''<br /> | ||
| + | | <span style="font-size: 16px">Execution of automated tests is too slow or too cumbersome.</span><br /> | ||
| + | |- | ||
| + | | <span style="font-size: 16px">''[[INEFFICIENT FAILURE ANALYSIS]]''</span><br /> | ||
| + | | <span style="font-size: 16px">Failure Analysis is slow and difficult</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-size: 16px">[[INFLEXIBLE AUTOMATION]]</span>''<br /> | ||
| + | | <span style="font-size: 16px">Execution of automated tests is ad-hoc or too rigid. Subsets of tests cannot be selected to run - it's "all or nothing".</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-size: 16px">[[LITTER BUG]]</span>'' [1]<br /> | ||
| + | | <span style="font-size: 16px">Tests leave spurious data in databases, history etc. that slows performance or hinders further tests.</span><br /> | ||
| + | |- | ||
| + | | ''<span style="font-family: Arial; font-size: 16px">[[MANUAL INTERVENTIONS]] </span>''<br /> | ||
| + | | <span style="font-family: Arial; font-size: 16px">Test automation needs lots of manual handlings to work</span><br /> | ||
| + | |} | ||
| + | <br /> <span style="font-family: Arial; font-size: 16px">[[Main Page]]</span><br /> <span style="font-family: Arial; font-size: 16px">Back to [[Test Automation Issues]]</span><br /> <span style="font-family: Arial; font-size: 16px">Back To [[Design Issues]]</span><br /> <span style="font-family: Arial; font-size: 16px">Forward to [[Process Issues]]</span><br /> <br /> <br /> [1] Suggested by Thorsten Schönfelder </div> | ||
Latest revision as of 13:03, 30 April 2018
Execution issues list the test automation problems that occur when the automated tests are run and range from unpredictable results, to interdependent tests, to automation that needs manual handling. The table below gives a short list of the issues. Clicking on an issue shows more detail and the patterns needed to solve it.
The Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.
Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues
[1] Suggested by Thorsten Schönfelder
The Execution Issues Mind Map shows an overview of the Execution Issues, together with the top level of resolving patterns, both the Most Recommended and other Useful patterns.
.
| Issues |
Description |
|---|---|
| FALSE FAIL |
The tests fail not because of errors in the SUT, but because of errors in the test automation testware or environment issues |
| FALSE PASS |
The tests pass even if the SUT actually reacts erroneously |
| FLAKY TESTS |
The automated tests seem to pass or fail randomly |
| INADEQUATE RESOURCES |
The hardware or the network are too slow |
| INEFFICIENT EXECUTION |
Execution of automated tests is too slow or too cumbersome. |
| INEFFICIENT FAILURE ANALYSIS |
Failure Analysis is slow and difficult |
| INFLEXIBLE AUTOMATION |
Execution of automated tests is ad-hoc or too rigid. Subsets of tests cannot be selected to run - it's "all or nothing". |
| LITTER BUG [1] |
Tests leave spurious data in databases, history etc. that slows performance or hinders further tests. |
| MANUAL INTERVENTIONS |
Test automation needs lots of manual handlings to work |
Main Page
Back to Test Automation Issues
Back To Design Issues
Forward to Process Issues
[1] Suggested by Thorsten Schönfelder