Difference between revisions of "Solution ideas for Automation Goals Exercise"
Jump to navigation
Jump to search
| Line 6: | Line 6: | ||
! Good automation goal? (If not, why not?) | ! Good automation goal? (If not, why not?) | ||
|- | |- | ||
| − | | <span style=" | + | | <span style="display: block; text-align: left">Run tests every night on all PCs</span><br /> |
| − | | <span style=" | + | | <span style="display: block; text-align: left">NO. or MAYBE, if we know the tests being run are worthwhile</span><br /> |
|- | |- | ||
| <span style="display: block; text-align: left">Automate all of our manual tests</span><br /> | | <span style="display: block; text-align: left">Automate all of our manual tests</span><br /> | ||
Revision as of 14:03, 6 July 2018
Some of the goals for automation seem like a good idea, but there are often problems that are not obvious. Below are comments on the possible test automation goals in the exercise.
| Possible test automation goals | Good automation goal? (If not, why not?) |
|---|---|
| Run tests every night on all PCs |
NO. or MAYBE, if we know the tests being run are worthwhile |
| Automate all of our manual tests |
NO. Automate only those tests that are worth automating. Plus automate more than manual tests. |
| Make it easy for business users to write and run automated tests |
YES. With the right testware architecture, non-technical testers can do this (with support). |
| Ensure repeatability of regression tests |
PROBABLY. Tools will run the same tests in the same way every time. But this is not what users do! |
| Ensure that we meet our next release deadline (we’re under time pressure) |
NO. Automation may help to run some tests that are required before release, but good automation takes time and effort to implement. |
| Find lots of bugs by running automated tests |
NO. Automation just runs tests. It is the tests that find bugs, whether they are run manually or are automated. |
| Find defects in less time |
NOT REALLY. Some types of defects (regression bugs) will be found more quickly by automated tests, but it may actually take longer to analyse the failures found. |
| Free testers from repeated (boring) test execution to spend more time doing exploratory testing |
YES. This is a good objective for test execution automation. |
| Reduce the number of testers |
NO. You will need more staff to implement the automation, not less. It can make existing staff more productive by spending more time on test design. |
| Reduce elapsed time for testing by x% |
NOT REALLY. Elapsed time depends on many factors, and not only on whether tests are automated. The factor that most often increases elapsed time for testing is the number of bugs found - this is due to the quality of the application (and the quality of the tests), not whether its tests are automated. |
| Run more tests / Run tests more often |
MAYBE. It is better to increase the value of tests, not the number. Good to run them more often – provided they are useful tests! |
| Increase coverage |
OF WHAT? This can be a very useful objective if it is understood and not just a platitude. |