.................................................................................................................Main Page / Back to Design Patterns Back to Test Automation Patterns

Pattern summary

Implement your test cases so that you can turn on various selection criteria for whether or not you include a given test in an execution run.

Context

This pattern is needed when you have a lot of automated tests, when you can no longer run all of them in the allotted time.

Description

The need for this pattern is not obvious when you first start automating, but it is an important one to take into consideration if you want to have large-scale sustainable automation.

Implementation

As part of the description or documentation of a test, include some Tags to identify this particular test in different ways. For example, as part of a Smoke Test, a regression test for a particular feature, a depth test for a function, when the test last found a bug, even the test's author. Choose your tags depending on the different ways you might like to form subsets of tests for execution.

When you DOCUMENT THE TESTWARE, this is one of the things that you will SET STANDARDS for.

When you are gathering tests for an execution run, your TEST AUTOMATION FRAMEWORK should enable you to choose the tests to run by specifying the selector tags to include. For example for tonight's overnight run, you may want to execute:

the tests that failed last time they were run (to see if they are fixed correctly)
the depth tests for the function that was most extensively changed
the highest priority tests from the full standard smoke test
Leslie's tests, as Leslie has been off sick for a few days

Note that your framework needs to be designed to enable the tests to be selected by these Test Selectors!

The sets of tests that you choose will be determined by how you PRIORITIZE TESTS.

Potential problems

If this is not considered at the beginning of an automation effort, it is more difficult to implement, but is usually still worth doing, for the flexibility it gives to test execution.

The format of the Test Selector needs to be clearly defined and the standards for their definition and use enforced.

Issues addressed by this pattern

INEFFICIENT EXECUTION
INFLEXIBLE AUTOMATION

Experiences

From Marcus Merrell:

Here's the list of test selectors that we use. Some of it is specific to our domain, but it should give a pretty good idea of how the test suite can be divided horizontally, vertically, or by priority, scope, or any other way you'd want to weight it:

 SMOKE 
  REGRESSION 
  PROD_SAFE 
  POP_UP 
  SEARCH 
  ADMIN 
  FACEBOOK 
  LONG_RUNNING 
  LOGIN 
  NEW_CMS 
  SIGN_UP 
  COMMUNITY 
  MOBILE_WEB 
  TABLET 
  MY_ACCOUNT 
  DESKTOP_ANALYTICS 
  TABLET_ANALYTICS 
  MOBILE_ANALYTICS 
  PRIVATE_API 
  DATA_DRIVEN 
  PROD_MONITOR 
  INQUISITION

If you have used also this pattern, please add your name and a brief story of how you used this pattern: your context, what you did, and how well it worked - or how it didn't work!
.................................................................................................................Main Page / Back to Design Patterns Back to Test Automation Patterns
B14B 14C 14D

TEST SELECTOR

Contents