Difference between revisions of "TEST SELECTOR"
Jump to navigation
Jump to search
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px">.........................................................................................") |
|||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] Back to [[Test Automation Patterns]]</span> | <div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] Back to [[Test Automation Patterns]]</span> | ||
| − | =<span style="font-size: 16px">Pattern summary</span>= | + | =<span style="font-size: 16px">'''Pattern summary'''</span>= |
<span style="font-size: 16px">Implement your test cases so that you can turn on various selection criteria for whether or not you include a given test in an execution run.</span> | <span style="font-size: 16px">Implement your test cases so that you can turn on various selection criteria for whether or not you include a given test in an execution run.</span> | ||
| − | =<span style="font-size: 16px">Category</span>= | + | =<span style="font-size: 16px">'''Category'''</span>= |
<span style="font-size: 16px">Design</span> | <span style="font-size: 16px">Design</span> | ||
| − | =<span style="font-size: 16px">Context</span>= | + | =<span style="font-size: 16px">'''Context'''</span>= |
<span style="font-size: 16px">This pattern is needed when you have a lot of automated tests, when you can no longer run all of them in the allotted time.</span> | <span style="font-size: 16px">This pattern is needed when you have a lot of automated tests, when you can no longer run all of them in the allotted time.</span> | ||
| − | =<span style="font-size: 16px">Description</span>= | + | =<span style="font-size: 16px">'''Description'''</span>= |
<span style="font-size: 16px">The need for this pattern is not obvious when you first start automating, but it is an important one to take into consideration if you want to have large-scale sustainable automation.</span> | <span style="font-size: 16px">The need for this pattern is not obvious when you first start automating, but it is an important one to take into consideration if you want to have large-scale sustainable automation.</span> | ||
| − | =<span style="font-size: 16px">Implementation</span>= | + | =<span style="font-size: 16px">'''Implementation'''</span>= |
<span style="font-size: 16px">As part of the description or documentation of a test, include some Tags to identify this particular test in different ways. For example, as part of a Smoke Test, a regression test for a particular feature, a depth test for a function, when the test last found a bug, even the test's author. Choose your tags depending on the different ways you might like to form subsets of tests for execution.</span><br /> <br /> <span style="font-size: 16px">When you [[DOCUMENT THE TESTWARE]], this is one of the things that you will [[SET STANDARDS]] for.</span><br /> <br /> <span style="font-size: 16px">When you are gathering tests for an execution run, your [[TEST AUTOMATION FRAMEWORK]] should enable you to choose the tests to run by specifying the selector tags to include. For example for tonight's overnight run, you may want to execute:</span><br /> | <span style="font-size: 16px">As part of the description or documentation of a test, include some Tags to identify this particular test in different ways. For example, as part of a Smoke Test, a regression test for a particular feature, a depth test for a function, when the test last found a bug, even the test's author. Choose your tags depending on the different ways you might like to form subsets of tests for execution.</span><br /> <br /> <span style="font-size: 16px">When you [[DOCUMENT THE TESTWARE]], this is one of the things that you will [[SET STANDARDS]] for.</span><br /> <br /> <span style="font-size: 16px">When you are gathering tests for an execution run, your [[TEST AUTOMATION FRAMEWORK]] should enable you to choose the tests to run by specifying the selector tags to include. For example for tonight's overnight run, you may want to execute:</span><br /> | ||
| − | |||
* <span style="font-size: 16px; line-height: 1.5">the tests that failed last time they were run (to see if they are fixed correctly)</span> | * <span style="font-size: 16px; line-height: 1.5">the tests that failed last time they were run (to see if they are fixed correctly)</span> | ||
* <span style="font-size: 16px; line-height: 1.5">the depth tests for the function that was most extensively changed</span> | * <span style="font-size: 16px; line-height: 1.5">the depth tests for the function that was most extensively changed</span> | ||
| Line 16: | Line 15: | ||
* <span style="font-size: 16px; line-height: 1.5">Leslie's tests, as Leslie has been off sick for a few days</span> | * <span style="font-size: 16px; line-height: 1.5">Leslie's tests, as Leslie has been off sick for a few days</span> | ||
<span style="font-size: 16px">Note that your framework needs to be designed to enable the tests to be selected by these Test Selectors!</span><br /> <br /> <span style="font-size: 16px">The sets of tests that you choose will be determined by how you [[ PRIORITIZE TESTS]].</span> | <span style="font-size: 16px">Note that your framework needs to be designed to enable the tests to be selected by these Test Selectors!</span><br /> <br /> <span style="font-size: 16px">The sets of tests that you choose will be determined by how you [[ PRIORITIZE TESTS]].</span> | ||
| − | =<span style="font-size: 16px">Potential problems</span>= | + | =<span style="font-size: 16px">'''Potential problems'''</span>= |
<span style="font-size: 16px">If this is not considered at the beginning of an automation effort, it is more difficult to implement, but is usually still worth doing, for the flexibility it gives to test execution.</span><br /> <br /> <span style="font-size: 16px">The format of the Test Selector needs to be clearly defined and the standards for their definition and use enforced.</span> | <span style="font-size: 16px">If this is not considered at the beginning of an automation effort, it is more difficult to implement, but is usually still worth doing, for the flexibility it gives to test execution.</span><br /> <br /> <span style="font-size: 16px">The format of the Test Selector needs to be clearly defined and the standards for their definition and use enforced.</span> | ||
| − | =<span style="font-size: 16px">Issues addressed by this pattern</span>= | + | =<span style="font-size: 16px">'''Issues addressed by this pattern'''</span>= |
''<span style="font-size: 16px">[[INEFFICIENT EXECUTION]]</span>''<br /> ''<span style="font-size: 16px">[[INFLEXIBLE AUTOMATION]]</span>'' | ''<span style="font-size: 16px">[[INEFFICIENT EXECUTION]]</span>''<br /> ''<span style="font-size: 16px">[[INFLEXIBLE AUTOMATION]]</span>'' | ||
| − | =<span style="font-size: 16px">Experiences</span>= | + | =<span style="font-size: 16px">'''Experiences'''</span>= |
<span style="font-size: 16px">From Marcus Merrell:</span><br /> <br /> <span style="font-size: 16px">Here's the list of test selectors that we use. Some of it is specific to our domain, but it should give a pretty good idea of how the test suite can be divided horizontally, vertically, or by priority, scope, or any other way you'd want to weight it:</span><br /> <br /> | <span style="font-size: 16px">From Marcus Merrell:</span><br /> <br /> <span style="font-size: 16px">Here's the list of test selectors that we use. Some of it is specific to our domain, but it should give a pretty good idea of how the test suite can be divided horizontally, vertically, or by priority, scope, or any other way you'd want to weight it:</span><br /> <br /> | ||
| − | + | <span style="font-size: 16px"> SMOKE </span><br /> <span style="font-size: 16px"> REGRESSION </span><br /> <span style="font-size: 16px"> PROD_SAFE </span><br /> <span style="font-size: 16px"> POP_UP </span><br /> <span style="font-size: 16px"> SEARCH </span><br /> <span style="font-size: 16px"> ADMIN </span><br /> <span style="font-size: 16px"> FACEBOOK </span><br /> <span style="font-size: 16px"> LONG_RUNNING </span><br /> <span style="font-size: 16px"> LOGIN </span><br /> <span style="font-size: 16px"> NEW_CMS </span><br /> <span style="font-size: 16px"> SIGN_UP </span><br /> <span style="font-size: 16px"> COMMUNITY </span><br /> <span style="font-size: 16px"> MOBILE_WEB </span><br /> <span style="font-size: 16px"> TABLET </span><br /> <span style="font-size: 16px"> MY_ACCOUNT </span><br /> <span style="font-size: 16px"> DESKTOP_ANALYTICS </span><br /> <span style="font-size: 16px"> TABLET_ANALYTICS </span><br /> <span style="font-size: 16px"> MOBILE_ANALYTICS </span><br /> <span style="font-size: 16px"> PRIVATE_API </span><br /> <span style="font-size: 16px"> DATA_DRIVEN </span><br /> <span style="font-size: 16px"> PROD_MONITOR </span><br /> <span style="font-size: 16px"> INQUISITION </span><br /> <br /> <br /> | |
| − | <span style="font-size: 16px">If you have used | + | |
| + | <span style="font-size: 16px">If you have also used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br /> | ||
| + | |||
| + | |||
| + | <span style="font-size: 16px">.................................................................................................................[[Main Page]] / Back to</span> <span style="font-size: 16px; line-height: 1.5">[[Design Patterns]] Back to [[Test Automation Patterns]]</span></div> | ||
Latest revision as of 15:59, 21 August 2018
.................................................................................................................Main Page / Back to Design Patterns Back to Test Automation Patterns
Pattern summary
Implement your test cases so that you can turn on various selection criteria for whether or not you include a given test in an execution run.
Category
Design
Context
This pattern is needed when you have a lot of automated tests, when you can no longer run all of them in the allotted time.
Description
The need for this pattern is not obvious when you first start automating, but it is an important one to take into consideration if you want to have large-scale sustainable automation.
Implementation
As part of the description or documentation of a test, include some Tags to identify this particular test in different ways. For example, as part of a Smoke Test, a regression test for a particular feature, a depth test for a function, when the test last found a bug, even the test's author. Choose your tags depending on the different ways you might like to form subsets of tests for execution.
When you DOCUMENT THE TESTWARE, this is one of the things that you will SET STANDARDS for.
When you are gathering tests for an execution run, your TEST AUTOMATION FRAMEWORK should enable you to choose the tests to run by specifying the selector tags to include. For example for tonight's overnight run, you may want to execute:
- the tests that failed last time they were run (to see if they are fixed correctly)
- the depth tests for the function that was most extensively changed
- the highest priority tests from the full standard smoke test
- Leslie's tests, as Leslie has been off sick for a few days
Note that your framework needs to be designed to enable the tests to be selected by these Test Selectors!
The sets of tests that you choose will be determined by how you PRIORITIZE TESTS.
Potential problems
If this is not considered at the beginning of an automation effort, it is more difficult to implement, but is usually still worth doing, for the flexibility it gives to test execution.
The format of the Test Selector needs to be clearly defined and the standards for their definition and use enforced.
Issues addressed by this pattern
INEFFICIENT EXECUTION
INFLEXIBLE AUTOMATION
Experiences
From Marcus Merrell:
Here's the list of test selectors that we use. Some of it is specific to our domain, but it should give a pretty good idea of how the test suite can be divided horizontally, vertically, or by priority, scope, or any other way you'd want to weight it:
SMOKE
REGRESSION
PROD_SAFE
POP_UP
SEARCH
ADMIN
FACEBOOK
LONG_RUNNING
LOGIN
NEW_CMS
SIGN_UP
COMMUNITY
MOBILE_WEB
TABLET
MY_ACCOUNT
DESKTOP_ANALYTICS
TABLET_ANALYTICS
MOBILE_ANALYTICS
PRIVATE_API
DATA_DRIVEN
PROD_MONITOR
INQUISITION
If you have also used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.