PRIORITIZE TESTS

.................................................................................................................Main Page / Back to Execution Patterns / Back to Test Automation Patterns

Pattern summary

Assign each test some kind of priority in order to be able to select easily the ones that should be run

Category

Execution

Context

Use this pattern when you may not be able to run all of the tests that you have in your automated portfolio.
Use this pattern to be selective about tests to run when something has changed.

Description

Assign each test some kind of priority, based on the criteria that are important at the time. The priority may be related to recent changes, the most critical user-facing aspects of the SUT (System Under Test), a recent change to some related system, or any other factor. The priority tests to run now may be different to the priority of the tests that you will want to run tomorrow.

You should be able to easily select subsets of tests to run, based on the priority assignment.

Implementation

• Group all the test cases that test the same functionalities of the SUT in the same test suite. In this way you can test different parts at different times or concurrently
• Reserve a parameter on every test case for the priority and configure your tool or TEST AUTOMATION FRAMEWORK so that you can select the priority of the test cases to be run. Common values are:
  • Critical
  • Important
  • Average
  • Minor
• Other categories can also be used to select which tests to run using a TEST SELECTOR, such as
  • all the tests that failed last time
  • the tests for a particular function
  • Joe's tests
  • high level smoke tests

Potential problems

You could pick the wrong priorities, but more commonly, not enough categories are chosen to be prioritised, so that you are working with sets of tests that are too large. If you choose smaller sets of tests, you can always assign the same priority to a number of them, but it is difficult to break them down if you don't think about it early enough.

Issues addressed by this pattern

INEFFICIENT EXECUTION
INFLEXIBLE AUTOMATION

Experiences

If you have used this pattern and would like to contribute your experience to the wiki, please go to Experiences to submit your experience or comment.

.................................................................................................................Main Page / Back to Execution Patterns / Back to Test Automation Patterns