Difference between revisions of "AUTOMATE THE METRICS"
Jump to navigation
Jump to search
| (4 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] / Back to [[Test Automation Patterns]]</span> | <div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] / Back to [[Test Automation Patterns]]</span> | ||
| − | =<span style="font-size: 16px">Pattern summary</span>= | + | =<span style="font-size: 16px">'''Pattern summary'''</span>= |
<span style="font-size: 16px">Automate metrics collection.</span> | <span style="font-size: 16px">Automate metrics collection.</span> | ||
| − | =<span style="font-size: 16px">Category</span>= | + | =<span style="font-size: 16px">'''Category'''</span>= |
<span style="font-size: 16px">Design</span> | <span style="font-size: 16px">Design</span> | ||
| − | =<span style="font-size: 16px">Context</span>= | + | =<span style="font-size: 16px">'''Context'''</span>= |
<span style="font-size: 16px">This pattern allows you to collect metrics efficiently and reliably. If you just write disposable scripts you will not need it</span> | <span style="font-size: 16px">This pattern allows you to collect metrics efficiently and reliably. If you just write disposable scripts you will not need it</span> | ||
| − | =<span style="font-size: 16px">Description</span>= | + | =<span style="font-size: 16px">'''Description'''</span>= |
<span style="font-size: 16px">By automating metrics collection, your metrics will be more reliable because they will be collected consistently and will not be so easily biased as manually collected</span> | <span style="font-size: 16px">By automating metrics collection, your metrics will be more reliable because they will be collected consistently and will not be so easily biased as manually collected</span> | ||
| − | =<span style="font-size: 16px">Implementation</span>= | + | =<span style="font-size: 16px">'''Implementation'''</span>= |
<span style="font-size: 16px">If your tool doesn’t support collecting metrics, consider implementing a [[TEST AUTOMATION FRAMEWORK]].</span><br /> <span style="font-size: 16px">Some suggestions what to collect with each test run:</span><br /> | <span style="font-size: 16px">If your tool doesn’t support collecting metrics, consider implementing a [[TEST AUTOMATION FRAMEWORK]].</span><br /> <span style="font-size: 16px">Some suggestions what to collect with each test run:</span><br /> | ||
| − | |||
* <span style="font-size: 16px">Number of tests available</span> | * <span style="font-size: 16px">Number of tests available</span> | ||
* <span style="font-size: 16px">Number of tests executed</span> | * <span style="font-size: 16px">Number of tests executed</span> | ||
| Line 19: | Line 18: | ||
* <span style="font-size: 16px">SUT Release</span> | * <span style="font-size: 16px">SUT Release</span> | ||
<span style="font-size: 16px">You should also try to associate bug-fix information to your test run metrics. For instance:</span><br /> | <span style="font-size: 16px">You should also try to associate bug-fix information to your test run metrics. For instance:</span><br /> | ||
| − | |||
* <span style="font-size: 16px">Number of errors removed</span> | * <span style="font-size: 16px">Number of errors removed</span> | ||
* <span style="font-size: 16px">Number of errors not yet removed</span> | * <span style="font-size: 16px">Number of errors not yet removed</span> | ||
| Line 25: | Line 23: | ||
* <span style="font-size: 16px">Number of tests failed after retest</span> | * <span style="font-size: 16px">Number of tests failed after retest</span> | ||
* <span style="font-size: 16px">Average time to remove an error</span> | * <span style="font-size: 16px">Average time to remove an error</span> | ||
| − | + | =<span style="font-size: 16px">'''Potential problems'''</span>= | |
| − | =<span style="font-size: 16px"> | ||
<span style="font-size: 16px">If possible keep track of which bugs were found with the test automation: it will help you retain support from management and testers</span> | <span style="font-size: 16px">If possible keep track of which bugs were found with the test automation: it will help you retain support from management and testers</span> | ||
| − | =<span style="font-size: 16px">Issues addressed by this pattern</span>= | + | =<span style="font-size: 16px">'''Issues addressed by this pattern'''</span>= |
| − | + | ''<span style="font-size: 16px">[[INSUFFICIENT METRICS]]</span>'' | |
| − | =<span style="font-size: 16px">Experiences</span>= | + | =<span style="font-size: 16px">'''Experiences'''</span>= |
| − | <span style="font-size: 16px">If you have used this pattern, please | + | |
| + | <span style="font-size: 16px">If you have used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br /> | ||
| + | |||
| + | <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] / Back to [[Test Automation Patterns]]</span></div> | ||
Latest revision as of 15:51, 21 August 2018
.................................................................................................................Main Page / Back to Design Patterns / Back to Test Automation Patterns
Pattern summary
Automate metrics collection.
Category
Design
Context
This pattern allows you to collect metrics efficiently and reliably. If you just write disposable scripts you will not need it
Description
By automating metrics collection, your metrics will be more reliable because they will be collected consistently and will not be so easily biased as manually collected
Implementation
If your tool doesn’t support collecting metrics, consider implementing a TEST AUTOMATION FRAMEWORK.
Some suggestions what to collect with each test run:
- Number of tests available
- Number of tests executed
- Number of tests passed
- Number of tests failed (eventually classified by error severity)
- Execution time
- Date
- SUT Release
You should also try to associate bug-fix information to your test run metrics. For instance:
- Number of errors removed
- Number of errors not yet removed
- Number of retests
- Number of tests failed after retest
- Average time to remove an error
Potential problems
If possible keep track of which bugs were found with the test automation: it will help you retain support from management and testers
Issues addressed by this pattern
Experiences
If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.