Difference between revisions of "COMPARE WITH PREVIOUS VERSION"

From Test Automation Patterns
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 15: Line 15:
 
''<span style="font-size: 16px">[[INEFFICIENT FAILURE ANALYSIS]]</span>''<br /> ''<span style="font-size: 16px">[[HARD-TO-AUTOMATE RESULTS]]</span>''
 
''<span style="font-size: 16px">[[INEFFICIENT FAILURE ANALYSIS]]</span>''<br /> ''<span style="font-size: 16px">[[HARD-TO-AUTOMATE RESULTS]]</span>''
 
=<span style="font-size: 16px">'''Experiences'''</span>=
 
=<span style="font-size: 16px">'''Experiences'''</span>=
 +
 +
<span style="font-size: 16px">I learned this pattern by trial and error: never run fresh test scripts with a new build, if it fails you don't know if it's the test script or the build.
 +
I always run newer tests on old established builds, so if they fail, I know it's a test script error. (Nish Herath)</span><br /> <br />
  
 
<span style="font-size: 16px">If you have used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br />  
 
<span style="font-size: 16px">If you have used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br />  
  
 
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Patterns]] / Back to [[Test Automation Patterns]]</span></div>
 
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Execution Patterns]] / Back to [[Test Automation Patterns]]</span></div>

Latest revision as of 06:58, 5 October 2022

.................................................................................................................Main Page / Back to Execution Patterns / Back to Test Automation Patterns

Pattern summary

Compare the current results with the results got by running the same tests in the previous version of the SUT

Category

Execution

Context

This pattern can be used when you have verified results for the previous release

Description

When it’s difficult to judge if the results are OK because of so many relationships, it may be much easier to just compare the current results with the results you got by running the same tests in the previous version of the SUT. If you know them to be OK then you don’t have to “understand” the contents, but just have to look for differences.

Implementation

First of all you must set-up the same initial conditions in the old and in the new version. Then run the tests on the old version and on the new one. Compare the results.

Potential problems

If you don’t have automated tests for the older version, run them manually

Issues addressed by this pattern

INEFFICIENT FAILURE ANALYSIS
HARD-TO-AUTOMATE RESULTS

Experiences

I learned this pattern by trial and error: never run fresh test scripts with a new build, if it fails you don't know if it's the test script or the build. I always run newer tests on old established builds, so if they fail, I know it's a test script error. (Nish Herath)

If you have used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.

.................................................................................................................Main Page / Back to Execution Patterns / Back to Test Automation Patterns