Solution ideas for Automation Goals Exercise

Some of the goals for automation seem like a good idea, but there are often problems that are not obvious. Below are comments on the possible test automation goals in the exercise.

Possible test automation goals	Good automation goal? (If not, why not?)
Run tests every night on all PCs	NO. or MAYBE, if we know the tests being run are worthwhile
Automate all of our manual tests	NO. Automate only those tests that are worth automating. Plus automate more than manual tests.
Make it easy for business users to write and run automated tests	YES. With the right testware architecture, non-technical testers can do this (with support).
Ensure repeatability of regression tests	PROBABLY. Tools will run the same tests in the same way every time. But this is not what users do!
Ensure that we meet our next release deadline (we’re under time pressure)	NO. Automation may help to run some tests that are required before release, but good automation takes time and effort to implement.
Find lots of bugs by running automated tests	NO. Automation just runs tests. It is the tests that find bugs, whether they are run manually or are automated.
Find defects in less time	NOT REALLY. Some types of defects (regression bugs) will be found more quickly by automated tests, but it may actually take longer to analyse the failures found.
Free testers from repeated (boring) test execution to spend more time doing exploratory testing	YES. This is a good objective for test execution automation.
Reduce the number of testers	NO. You will need more staff to implement the automation, not less. It can make existing staff more productive by spending more time on test design.
Reduce elapsed time for testing by x%	NOT REALLY. Elapsed time depends on many factors, and not only on whether tests are automated. The factor that most often increases elapsed time for testing is the number of bugs found - this is due to the quality of the application (and the quality of the tests), not whether its tests are automated.
Run more tests / Run tests more often	MAYBE. It is better to increase the value of tests, not the number. Good to run them more often – provided they are useful tests!
Increase coverage	OF WHAT? This can be a very useful objective if it is understood and not just a platitude.

Back to Exercises