Difference between revisions of "Solution to Freddie Frustrated"
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 16px"> When building the automation, things seemed to go well. Freddie thought about the te...") |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 26: | Line 26: | ||
* From the Diagnostic, [[Improve or revive test automation]] | * From the Diagnostic, [[Improve or revive test automation]] | ||
** leads to [[Expectations for automated test execution not met]], | ** leads to [[Expectations for automated test execution not met]], | ||
| − | *** from which the best choice is probably [[ | + | *** from which the best choice is probably [[The%20scripts%20are%20not%20reliable]]. This gives two most likely issues: |
**** [[FLAKY TESTS]] (Execution Issue) is probably the most appropriate one. There are a number of Patterns listed in this Issue, which may be relevant to Freddie’s situation: | **** [[FLAKY TESTS]] (Execution Issue) is probably the most appropriate one. There are a number of Patterns listed in this Issue, which may be relevant to Freddie’s situation: | ||
***** [[FAIL GRACEFULLY]] (Execution Pattern) | ***** [[FAIL GRACEFULLY]] (Execution Pattern) | ||
| Line 37: | Line 37: | ||
<span style="font-size: 16px"><br />It is important to address the problem of flaky tests, as this can undermine everyone’s confidence in all of the results from automated tests. | <span style="font-size: 16px"><br />It is important to address the problem of flaky tests, as this can undermine everyone’s confidence in all of the results from automated tests. | ||
| − | |||
| − | |||
Latest revision as of 11:00, 12 August 2019
When building the automation, things seemed to go well. Freddie thought about the test architecture and designing the tests for ease of maintenance, and at first, the tests ran smoothly, giving them useful information about the system they were testing.
But after a while, it seems to be taking more and more time to find out what’s going on when automated tests fail. At first, just a few tests began to fail in the overnight run, but when they were re-run in the morning, they would pass. This is happening more often now. Since the automation is seen as successful and useful, more and more people are using automated tests, but sometimes, they leave the test data in a state that isn’t what someone else’s test was expecting, so their tests fail, even though they should have passed. When the developers change the names of some objects, lots of tests fail even though the system still works fine.
People are complaining that the automation is no longer fit for purpose, because there are so many problems with failing tests, and they are losing faith in all the the automation.
1) Go to Test Automation Issues Mind Map or Test Automation Issues Mind map with clickable links. Look through the Issue names and see if one stands out as Freddie's main problem. See which patterns would solve it
2) Go to Test Automation Patterns Mind Map or Test Automation Patterns Mind Map with clickable links. Look through the Pattern names and see which stand out as possible solutions for Freddie.
3) Go to the Diagnostic. Answer the questions on Freddie's behalf and see if you find the same issues and patterns.
Solution:
- From the Diagnostic, Improve or revive test automation
- leads to Expectations for automated test execution not met,
- from which the best choice is probably The scripts are not reliable. This gives two most likely issues:
- FLAKY TESTS (Execution Issue) is probably the most appropriate one. There are a number of Patterns listed in this Issue, which may be relevant to Freddie’s situation:
- FAIL GRACEFULLY (Execution Pattern)
- VARIABLE DELAYS (Execution Pattern)
- DEDICATED RESOURCES (Management Pattern)
- FRESH SETUP (Design Pattern)
- FALSE FAIL (Execution Issue) is another likely Issue. Here the suggested Patterns, which may be relevant to Freddie’s situation:
- DEDICATED RESOURCES (Management Pattern)
- FRESH SETUP (Design Pattern)
- FLAKY TESTS (Execution Issue) is probably the most appropriate one. There are a number of Patterns listed in this Issue, which may be relevant to Freddie’s situation:
- from which the best choice is probably The scripts are not reliable. This gives two most likely issues:
- leads to Expectations for automated test execution not met,
It is important to address the problem of flaky tests, as this can undermine everyone’s confidence in all of the results from automated tests.