Difference between revisions of "INEFFICIENT EXECUTION"

From Test Automation Patterns
Jump to navigation Jump to search
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.........................................................................................")
 
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span>
 
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span>
=<span style="font-size: 16px">Issue summary</span>=
+
=<span style="font-size: 16px">'''Issue summary'''</span>=
 
<span style="font-size: 16px">Execution of automated tests is too slow or too cumbersome.</span>
 
<span style="font-size: 16px">Execution of automated tests is too slow or too cumbersome.</span>
=<span style="font-size: 16px">Category</span>=
+
=<span style="font-size: 16px">'''Category'''</span>=
 
<span style="font-size: 16px">Execution</span>
 
<span style="font-size: 16px">Execution</span>
=<span style="font-size: 16px">Examples</span>=  
+
=<span style="font-size: 16px">'''Examples'''</span>=  
 
 
 
# <span style="font-family: Arial; font-size: 16px">Automated tests run slowly because synchronisation with the user interface involves waiting for a response.</span>
 
# <span style="font-family: Arial; font-size: 16px">Automated tests run slowly because synchronisation with the user interface involves waiting for a response.</span>
 
# <span style="font-family: Arial; font-size: 16px">The automated tests take so long to run that they are executed only for the final release, not in a daily build.</span>
 
# <span style="font-family: Arial; font-size: 16px">The automated tests take so long to run that they are executed only for the final release, not in a daily build.</span>
 
# <span style="font-family: Arial; font-size: 16px">Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.</span>
 
# <span style="font-family: Arial; font-size: 16px">Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.</span>
 
# <span style="font-family: Arial; font-size: 16px">Execution is disrupted by unexpected windows (or pop-ups).</span>
 
# <span style="font-family: Arial; font-size: 16px">Execution is disrupted by unexpected windows (or pop-ups).</span>
=<span style="font-size: 16px">Questions</span>=
+
=<span style="font-size: 16px">'''Questions'''</span>=
 
<span style="font-family: Arial; font-size: 16px">How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)</span><br /> <span style="font-size: 16px">How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?</span><br /> <span style="font-size: 16px">Can tests be run in parallel? Can set-up etc be separated out to separate scripts?</span><br /> <span style="font-size: 16px">Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?</span>
 
<span style="font-family: Arial; font-size: 16px">How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)</span><br /> <span style="font-size: 16px">How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?</span><br /> <span style="font-size: 16px">Can tests be run in parallel? Can set-up etc be separated out to separate scripts?</span><br /> <span style="font-size: 16px">Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?</span>
=<span style="font-size: 16px">Resolving Patterns</span>=
+
=<span style="font-size: 16px">'''Resolving Patterns'''</span>=
 
<span style="font-size: 16px">Most recommended:</span><br />  
 
<span style="font-size: 16px">Most recommended:</span><br />  
 
 
* <span style="font-size: 16px">[[PARALLELIZE TESTS]] - split the tests to run in parallel (on different machines), which requires [[INDEPENDENT TEST CASES]].</span>
 
* <span style="font-size: 16px">[[PARALLELIZE TESTS]] - split the tests to run in parallel (on different machines), which requires [[INDEPENDENT TEST CASES]].</span>
 
* <span style="font-size: 16px">[[STEEL THREAD]]: Use this pattern to help define a minimal set of tests to be run.</span>
 
* <span style="font-size: 16px">[[STEEL THREAD]]: Use this pattern to help define a minimal set of tests to be run.</span>
 
* <span style="font-size: 16px; line-height: 1.5">[[TEST SELECTOR]]: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.</span>
 
* <span style="font-size: 16px; line-height: 1.5">[[TEST SELECTOR]]: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.</span>
 
* <span style="font-size: 16px; line-height: 1.5">[[EXPECT INCIDENTS]]: </span><span style="font-size: 16px">automated scripts should be able to react to unexpected incidents without disrupting execution</span>
 
* <span style="font-size: 16px; line-height: 1.5">[[EXPECT INCIDENTS]]: </span><span style="font-size: 16px">automated scripts should be able to react to unexpected incidents without disrupting execution</span>
* <span style="font-size: 16px">VARIABLE DELAYS: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes</span>
+
* <span style="font-size: 16px">[[VARIABLE DELAYS]]: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes</span>
=<span style="font-size: 16px">Other useful patterns:</span>=
+
<span style="font-size: 16px">'''Other useful patterns:'''</span>
 
 
 
* <span style="font-size: 16px">[[UNATTENDED TEST EXECUTION]]: This pattern is what you want to automate your tests for! Use it!</span>
 
* <span style="font-size: 16px">[[UNATTENDED TEST EXECUTION]]: This pattern is what you want to automate your tests for! Use it!</span>
 
* <span style="font-size: 16px"><span style="font-size: 16px">[[SKIP VOID INPUTS]]:</span> in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs </span>
 
* <span style="font-size: 16px"><span style="font-size: 16px">[[SKIP VOID INPUTS]]:</span> in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs </span>
 
<br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span></div>
 
<br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Issues]] / Back to [[ Test Automation Issues]]</span></div>

Latest revision as of 08:51, 28 June 2018

.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues

Issue summary

Execution of automated tests is too slow or too cumbersome.

Category

Execution

Examples

  1. Automated tests run slowly because synchronisation with the user interface involves waiting for a response.
  2. The automated tests take so long to run that they are executed only for the final release, not in a daily build.
  3. Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.
  4. Execution is disrupted by unexpected windows (or pop-ups).

Questions

How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)
How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?
Can tests be run in parallel? Can set-up etc be separated out to separate scripts?
Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?

Resolving Patterns

Most recommended:

  • PARALLELIZE TESTS - split the tests to run in parallel (on different machines), which requires INDEPENDENT TEST CASES.
  • STEEL THREAD: Use this pattern to help define a minimal set of tests to be run.
  • TEST SELECTOR: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.
  • EXPECT INCIDENTS: automated scripts should be able to react to unexpected incidents without disrupting execution
  • VARIABLE DELAYS: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes

Other useful patterns:

  • UNATTENDED TEST EXECUTION: This pattern is what you want to automate your tests for! Use it!
  • SKIP VOID INPUTS: in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs

.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues