Difference between revisions of "INEFFICIENT EXECUTION"
Jump to navigation
Jump to search
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.........................................................................................") |
|||
| (2 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span> | <div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span> | ||
| − | =<span style="font-size: 16px">Issue summary</span>= | + | =<span style="font-size: 16px">'''Issue summary'''</span>= |
<span style="font-size: 16px">Execution of automated tests is too slow or too cumbersome.</span> | <span style="font-size: 16px">Execution of automated tests is too slow or too cumbersome.</span> | ||
| − | =<span style="font-size: 16px">Category</span>= | + | =<span style="font-size: 16px">'''Category'''</span>= |
<span style="font-size: 16px">Execution</span> | <span style="font-size: 16px">Execution</span> | ||
| − | =<span style="font-size: 16px">Examples</span>= | + | =<span style="font-size: 16px">'''Examples'''</span>= |
| − | |||
# <span style="font-family: Arial; font-size: 16px">Automated tests run slowly because synchronisation with the user interface involves waiting for a response.</span> | # <span style="font-family: Arial; font-size: 16px">Automated tests run slowly because synchronisation with the user interface involves waiting for a response.</span> | ||
# <span style="font-family: Arial; font-size: 16px">The automated tests take so long to run that they are executed only for the final release, not in a daily build.</span> | # <span style="font-family: Arial; font-size: 16px">The automated tests take so long to run that they are executed only for the final release, not in a daily build.</span> | ||
# <span style="font-family: Arial; font-size: 16px">Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.</span> | # <span style="font-family: Arial; font-size: 16px">Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.</span> | ||
# <span style="font-family: Arial; font-size: 16px">Execution is disrupted by unexpected windows (or pop-ups).</span> | # <span style="font-family: Arial; font-size: 16px">Execution is disrupted by unexpected windows (or pop-ups).</span> | ||
| − | =<span style="font-size: 16px">Questions</span>= | + | =<span style="font-size: 16px">'''Questions'''</span>= |
<span style="font-family: Arial; font-size: 16px">How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)</span><br /> <span style="font-size: 16px">How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?</span><br /> <span style="font-size: 16px">Can tests be run in parallel? Can set-up etc be separated out to separate scripts?</span><br /> <span style="font-size: 16px">Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?</span> | <span style="font-family: Arial; font-size: 16px">How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)</span><br /> <span style="font-size: 16px">How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?</span><br /> <span style="font-size: 16px">Can tests be run in parallel? Can set-up etc be separated out to separate scripts?</span><br /> <span style="font-size: 16px">Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?</span> | ||
| − | =<span style="font-size: 16px">Resolving Patterns</span>= | + | =<span style="font-size: 16px">'''Resolving Patterns'''</span>= |
<span style="font-size: 16px">Most recommended:</span><br /> | <span style="font-size: 16px">Most recommended:</span><br /> | ||
| − | |||
* <span style="font-size: 16px">[[PARALLELIZE TESTS]] - split the tests to run in parallel (on different machines), which requires [[INDEPENDENT TEST CASES]].</span> | * <span style="font-size: 16px">[[PARALLELIZE TESTS]] - split the tests to run in parallel (on different machines), which requires [[INDEPENDENT TEST CASES]].</span> | ||
* <span style="font-size: 16px">[[STEEL THREAD]]: Use this pattern to help define a minimal set of tests to be run.</span> | * <span style="font-size: 16px">[[STEEL THREAD]]: Use this pattern to help define a minimal set of tests to be run.</span> | ||
* <span style="font-size: 16px; line-height: 1.5">[[TEST SELECTOR]]: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.</span> | * <span style="font-size: 16px; line-height: 1.5">[[TEST SELECTOR]]: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.</span> | ||
* <span style="font-size: 16px; line-height: 1.5">[[EXPECT INCIDENTS]]: </span><span style="font-size: 16px">automated scripts should be able to react to unexpected incidents without disrupting execution</span> | * <span style="font-size: 16px; line-height: 1.5">[[EXPECT INCIDENTS]]: </span><span style="font-size: 16px">automated scripts should be able to react to unexpected incidents without disrupting execution</span> | ||
| − | * <span style="font-size: 16px">VARIABLE DELAYS: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes</span> | + | * <span style="font-size: 16px">[[VARIABLE DELAYS]]: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes</span> |
| − | + | <span style="font-size: 16px">'''Other useful patterns:'''</span> | |
| − | |||
* <span style="font-size: 16px">[[UNATTENDED TEST EXECUTION]]: This pattern is what you want to automate your tests for! Use it!</span> | * <span style="font-size: 16px">[[UNATTENDED TEST EXECUTION]]: This pattern is what you want to automate your tests for! Use it!</span> | ||
* <span style="font-size: 16px"><span style="font-size: 16px">[[SKIP VOID INPUTS]]:</span> in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs </span> | * <span style="font-size: 16px"><span style="font-size: 16px">[[SKIP VOID INPUTS]]:</span> in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs </span> | ||
<br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span></div> | <br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span></div> | ||
Latest revision as of 08:51, 28 June 2018
.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues
.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues
Issue summary
Execution of automated tests is too slow or too cumbersome.
Category
Execution
Examples
- Automated tests run slowly because synchronisation with the user interface involves waiting for a response.
- The automated tests take so long to run that they are executed only for the final release, not in a daily build.
- Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.
- Execution is disrupted by unexpected windows (or pop-ups).
Questions
How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)
How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?
Can tests be run in parallel? Can set-up etc be separated out to separate scripts?
Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?
Resolving Patterns
Most recommended:
- PARALLELIZE TESTS - split the tests to run in parallel (on different machines), which requires INDEPENDENT TEST CASES.
- STEEL THREAD: Use this pattern to help define a minimal set of tests to be run.
- TEST SELECTOR: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.
- EXPECT INCIDENTS: automated scripts should be able to react to unexpected incidents without disrupting execution
- VARIABLE DELAYS: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes
Other useful patterns:
- UNATTENDED TEST EXECUTION: This pattern is what you want to automate your tests for! Use it!
- SKIP VOID INPUTS: in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs
.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues