INEFFICIENT EXECUTION

From Test Automation Patterns
Jump to: navigation, search
.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues

Issue summary

Execution of automated tests is too slow or too cumbersome.

Category

Execution

Examples

  1. Automated tests run slowly because synchronisation with the user interface involves waiting for a response.
  2. The automated tests take so long to run that they are executed only for the final release, not in a daily build.
  3. Test set-up and post-processing hasn't been automated, requires some manual intervention or is done in an inefficient way.
  4. Execution is disrupted by unexpected windows (or pop-ups).

Questions

How long do the tests take to run? Why is this a problem? (E.g. is there a time window that a set of tests should be run in but they take longer than that?)
How much of the test execution time is spent on activities other than running tests, e.g. set-up, clear-up?
Can tests be run in parallel? Can set-up etc be separated out to separate scripts?
Are too many tests run through the Graphical User Interface (GUI) rather than through an Application Programming Interface (API)?

Resolving Patterns

Most recommended:

  • PARALLELIZE TESTS - split the tests to run in parallel (on different machines), which requires INDEPENDENT TEST CASES.
  • STEEL THREAD: Use this pattern to help define a minimal set of tests to be run.
  • TEST SELECTOR: Implement your test cases so that you can turn on various selection criteria for whether or not a given test is included in an execution run.
  • EXPECT INCIDENTS: automated scripts should be able to react to unexpected incidents without disrupting execution
  • VARIABLE DELAYS: Use this pattern if execution times depend for instance on the state of the nework or on the different lenght of some SUT-processes

Other useful patterns:

  • UNATTENDED TEST EXECUTION: This pattern is what you want to automate your tests for! Use it!
  • SKIP VOID INPUTS: in order to make your scripts more efficient, arrange for an easy way to automatically skip void Inputs

.................................................................................................................Main Page / Back to Execution Issues / Back to Test Automation Issues