TEST AUTOMATION BUSINESS CASE
Pattern summary
Present a business case as for any other project, but provide also factors typical for test automation
Category
Management
Context
You will need to apply this pattern if you have to convince management to support your test automation project.
It will not be necessary if you are only developing disposable scripts or if you already have good management support for automation without needing to make a business case. For example automation may be the only way to survive in agile development in a fast-moving industry.
Description
You present a business case as for any other project, but you should also provide factors typical for test automation, like the quality improvement due to broader testing or the reallocation of tester capacity to more challenging tasks once test automation has been implemented. These factors should be directly related to your own goals for automation.
Implementation
In order to succeed in test automation, you must first SET CLEAR GOALS: what do you want to achieve with the project? What not? It is crucial that you make very clear what test automation can or cannot deliver. This is important for everyone, but is particularly important when you make a business case for automation - if you set unachievable goals, you are guaranteed to fail!
The essence of a business case is to show that the benefits of the proposed course of action will be greater than its cost. The equation used for this is Return on Investment (ROI) = (benefit - cost) / cost. This gives a number (often quoted as a percentage). If the benefit is worth double the cost, then the ROI is 1 (or 100%).
In order to compare manual testing to automated testing, first determine the costs of the manual testing process that you intend to replace with automation. Factor in the number of times the tests will (or should) be run. A given set of manual tests generally costs twice as much to run them 2 times, etc.
Then, estimate the planned costs for automation, including:
- Time and resources needed to select a suitable test tool
- Tool licences and maintenance costs
- Training
- Learning times
- Time and resources to develop the TESTWARE ARCHITECTURE
- Time and resources to develop the test automation infrastructure
- Maintenance costs for a test automation infrastructure
- Time to build new automated tests
- Maintenance costs for test automation testware
- Time needed for refactoring the automation testware over time
- Time to analyse failures found by the automated tests (this may be significantly greater than in manual testing)
Now comes the more difficult part - estimating the benefits of automation, the return that could be expected on the investment. In order to use the ROI equation, the benefits need to be converted into the same units as the costs, usually money! Here are some ideas:
- Saved tester time = Time needed by the human tester to run a set of tests manually, minus the time needed for those same tests once they are automated. This is relatively easy to estimate using a "loaded salary cost" for the human testers.
- Number of automated test runs necessary for pay off - this can be calculated by the savings in human time multiplied by the number of times the tests would be run (or number of releases). When the savings catch up to the investment, then you are "in credit" and have a positive ROI.
But this is not the whole story. Other factors may not be easily converted into money to put into an ROI calculation, but are an important part of a business case.
- The automation can free the testers from mundane and repetitive activities to concentrate on devising more and better tests.
- More tests can be run more often, so more of the software can be tested in each build or release. This increased coverage can give greater confidence that the system will work well.
- Improved morale of the testers, since the tools do the tedious bits and they can use their talents more fully to do better testing.
Don’t forget to PLAN SUPPORT ACTIVITIES
Potential problems
The factors that may be easiest to put into a business case may not be all of the important factors, and may create a bias towards those factors. For example, it is easy to quantify that tester's time will be greatly reduced when tests are automated, compared to the time doing manual test execution. If this is the only factor taken into account, it may give the impression to managers that they won't need to have the testers any more! However, there are many other things that the testers need to do and there are activities that may take longer with automated tests than they did when tests were manual. Test automation should support testers, not replace them! A tool can never replace the human brain!
Issues addressed by this pattern
AD-HOC AUTOMATION
INADEQUATE SUPPORT
NO PREVIOUS TEST AUTOMATION
Experiences
Note: Freebie
If you need to calculate ROI, there is a free simple ROI calculation spreadsheet that Dot will email to you on request. Contact her using the Contact form on her web site DorothyGraham.co.uk.
If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.