Difference between revisions of "Solution ideas for Automation Goals Exercise"

From Test Automation Patterns
Jump to navigation Jump to search
 
(One intermediate revision by one other user not shown)
Line 6: Line 6:
 
! Good automation goal? (If not, why not?)
 
! Good automation goal? (If not, why not?)
 
|-
 
|-
| <span style="font-size: 16px">Run tests every night on all PCs</span><br />
+
| <span style="display: block; text-align: left">Run tests every night on all PCs</span><br />
| <span style="font-size: 16px">NO. or MAYBE, if we know the tests being run are worthwhile</span><br />
+
| <span style="display: block; text-align: left">NO. or MAYBE, if we know the tests being run are worthwhile</span><br />
 
|-
 
|-
 
| <span style="display: block; text-align: left">Automate all of our manual tests</span><br />
 
| <span style="display: block; text-align: left">Automate all of our manual tests</span><br />
Line 42: Line 42:
 
| <span style="display: block; text-align: left">OF WHAT? This can be a very useful objective if it is understood and not just a platitude.</span><br />
 
| <span style="display: block; text-align: left">OF WHAT? This can be a very useful objective if it is understood and not just a platitude.</span><br />
 
|}
 
|}
 +
 +
<br /> <br /> <span style="font-size: 16px">Back to [[Exercises]]</span>
 
</div>
 
</div>

Latest revision as of 16:01, 3 August 2018

Some of the goals for automation seem like a good idea, but there are often problems that are not obvious. Below are comments on the possible test automation goals in the exercise.

Possible test automation goals Good automation goal? (If not, why not?)
Run tests every night on all PCs
NO. or MAYBE, if we know the tests being run are worthwhile
Automate all of our manual tests
NO. Automate only those tests that are worth automating. Plus automate more than manual tests.
Make it easy for business users to write and run automated tests
YES. With the right testware architecture, non-technical testers can do this (with support).
Ensure repeatability of regression tests
PROBABLY. Tools will run the same tests in the same way every time. But this is not what users do!
Ensure that we meet our next release deadline (we’re under time pressure)
NO. Automation may help to run some tests that are required before release, but good automation takes time and effort to implement.
Find lots of bugs by running automated tests
NO. Automation just runs tests. It is the tests that find bugs, whether they are run manually or are automated.
Find defects in less time
NOT REALLY. Some types of defects (regression bugs) will be found more quickly by automated tests, but it may actually take longer to analyse the failures found.
Free testers from repeated (boring) test execution to spend more time doing exploratory testing
YES. This is a good objective for test execution automation.
Reduce the number of testers
NO. You will need more staff to implement the automation, not less. It can make existing staff more productive by spending more time on test design.
Reduce elapsed time for testing by x%
NOT REALLY. Elapsed time depends on many factors, and not only on whether tests are automated. The factor that most often increases elapsed time for testing is the number of bugs found - this is due to the quality of the application (and the quality of the tests), not whether its tests are automated.
Run more tests / Run tests more often
MAYBE. It is better to increase the value of tests, not the number. Good to run them more often – provided they are useful tests!
Increase coverage
OF WHAT? This can be a very useful objective if it is understood and not just a platitude.



Back to Exercises