Difference between revisions of "COMPARE WITH PREVIOUS VERSION"
| Line 16: | Line 16: | ||
=<span style="font-size: 16px">'''Experiences'''</span>= | =<span style="font-size: 16px">'''Experiences'''</span>= | ||
| − | <span style="font-size: 16px">I learned this pattern by trial and error | + | <span style="font-size: 16px">I learned this pattern by trial and error: never run fresh test scripts with a new build, if it fails you don't know if it's the test script or the build. |
| − | I always run newer | + | I always run newer tests on old established builds, so if they fail, I know it's a test script error. (Nish Herath)</span><br /> <br /> |
<span style="font-size: 16px">If you have used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br /> | <span style="font-size: 16px">If you have used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br /> | ||
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Patterns]] / Back to [[Test Automation Patterns]]</span></div> | <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Patterns]] / Back to [[Test Automation Patterns]]</span></div> | ||
Latest revision as of 06:58, 5 October 2022
Pattern summary
Compare the current results with the results got by running the same tests in the previous version of the SUT
Category
Execution
Context
This pattern can be used when you have verified results for the previous release
Description
When it’s difficult to judge if the results are OK because of so many relationships, it may be much easier to just compare the current results with the results you got by running the same tests in the previous version of the SUT. If you know them to be OK then you don’t have to “understand” the contents, but just have to look for differences.
Implementation
First of all you must set-up the same initial conditions in the old and in the new version. Then run the tests on the old version and on the new one. Compare the results.
Potential problems
If you don’t have automated tests for the older version, run them manually
Issues addressed by this pattern
INEFFICIENT FAILURE ANALYSIS
HARD-TO-AUTOMATE RESULTS
Experiences
I learned this pattern by trial and error: never run fresh test scripts with a new build, if it fails you don't know if it's the test script or the build.
I always run newer tests on old established builds, so if they fail, I know it's a test script error. (Nish Herath)
If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.