Solution to Nancy Naive

Nancy is new to test automation. The company bought a test execution tool and she was given the task of automating the manual regression tests. She was shown a demo of the tool and was very impressed with the capture replay idea, so she began recording the regression tests as she performed them for the current release of the system.

However, when the next release was ready, she found that many of the recorded tests didn’t actually play back or didn’t do what they should. She spent 2 days trying to fix the tests, and ended up re-recording some of them. She feels that this is too much time to spend on maintaining the tests! She is now concerned about what will happen the next time the system changes and wants to make sure that her test maintenance costs are kept to a reasonable level.

Exercises:

1) Go to Test Automation Issues Mind Map or Test Automation Issues Mind map with clickable links. Look through the Issue names and see if one stands out as Nancy's main problem.

2) Go to the Diagnostic. Answer the questions on Nancy's behalf and see if you find the same issues and patterns.

Solution:
Using the Diagnostic, there are two possible issues on the Diagnostic page:

• NO PREVIOUS TEST AUTOMATION (Management Issue) - Nancy is new to automation so this one may apply, or LIMITED EXPERIENCE (Management Pattern)
• Another possibility is to choose Improve or revive test automation
• The most likely choice here is Maintenance expectations not met.
• The topic Maintenance costs too high is a good choice here,
• followed by Updating the automation scripts as it takes too much effort to update the scripts. This leads to the two most relevant issues:
  • BRITTLE SCRIPTS (Design Issue) that recommends for instance the following patterns:
    • MAINTAINABLE TESTWARE (Design Pattern)
    • MANAGEMENT SUPPORT (Management Pattern)
    • TESTWARE ARCHITECTURE (Design Pattern)
    • GOOD PROGRAMMING PRACTICES (Process Pattern)
  • TOOL DEPENDENCY (Design Issue). Other patterns recommended by TOOL DEPENDENCY include:
    • RIGHT TOOLS (Management Pattern)
    • TOOL INDEPENDENCE (Design Pattern)
    • OBJECT MAP (Execution Pattern)
  • Both of these issues mention ABSTRACTION LEVELS (Design Pattern), so that would be a good one to look at.

Basically, Nancy should have thought about the design of her automation before she started, rather than jumping into capture-replay. She will probably have to start again and may or may not be able to use anything that she has recorded so far.