Difference between revisions of "HARD-TO-AUTOMATE RESULTS"
Jump to navigation
Jump to search
m (Topic titles in capital letters) |
|||
| Line 10: | Line 10: | ||
# <span style="font-size: 16px">Results change from release to release</span> | # <span style="font-size: 16px">Results change from release to release</span> | ||
# <span style="font-size: 16px">Results depend on what test cases have been run before</span> | # <span style="font-size: 16px">Results depend on what test cases have been run before</span> | ||
| + | # <span style="font-size: 16px">API or AI tests deliver non-predictable results</span> | ||
=<span style="font-size: 16px">'''Questions'''</span>= | =<span style="font-size: 16px">'''Questions'''</span>= | ||
<span style="font-size: 16px">How complex are the automated test cases? Can they be split up?</span><br /> <span style="font-family: Arial; font-size: 16px">Are complex "scenario" test cases really necessary?</span> | <span style="font-size: 16px">How complex are the automated test cases? Can they be split up?</span><br /> <span style="font-family: Arial; font-size: 16px">Are complex "scenario" test cases really necessary?</span> | ||
=<span style="font-size: 16px">'''Resolving Patterns'''</span>= | =<span style="font-size: 16px">'''Resolving Patterns'''</span>= | ||
<span style="font-size: 16px">Most recommended:</span><br /> | <span style="font-size: 16px">Most recommended:</span><br /> | ||
| − | * <span style="font-size: 16px"><span style="font-size: 16px">[[ FRESH SETUP]]</span>: apply this pattern if you have issues like Example 5</span> | + | * <span style="font-size: 16px"><span style="font-size: 16px">[[FRESH SETUP]]</span>: apply this pattern if you have issues like Example 5</span> |
| − | * <span style="font-size: 16px">[[ DEDICATED RESOURCES]]: look up this pattern if you have to share resources</span> | + | * <span style="font-size: 16px">[[DEDICATED RESOURCES]]: look up this pattern if you have to share resources</span> |
| − | * <span style="font-size: 16px">[[ WHOLE TEAM APPROACH]]: if your team follows an agile development process this is the pattern to use in order to avoid this kind of problems from the beginning</span> | + | * <span style="font-size: 16px">[[THINK OUT-OF-THE-BOX]]: try to look at the problem from unusual viewpoints. This is especially important for issues like Example 6</span> |
| + | * <span style="font-size: 16px">[[WHOLE TEAM APPROACH]]: if your team follows an agile development process this is the pattern to use in order to avoid this kind of problems from the beginning</span> | ||
<br /> <span style="font-size: 16px">Other useful patterns:</span><br /> | <br /> <span style="font-size: 16px">Other useful patterns:</span><br /> | ||
* <span style="font-size: 16px">[[SHARE INFORMATION]]: use this pattern for better communication between development, testing and automation </span> | * <span style="font-size: 16px">[[SHARE INFORMATION]]: use this pattern for better communication between development, testing and automation </span> | ||
| − | * <span style="font-size: 16px">[[ DESIGN FOR REUSE]]: this pattern should be used always</span> | + | * <span style="font-size: 16px">[[DESIGN FOR REUSE]]: this pattern should be used always</span> |
| − | * <span style="font-size: 16px">[[ COMPARE WITH PREVIOUS VERSION]]: this pattern helps when nothing else does</span> | + | * <span style="font-size: 16px">[[COMPARE WITH PREVIOUS VERSION]]: this pattern helps when nothing else does</span> |
| − | * <span style="font-size: 16px">[[ TAKE SMALL STEPS]]: this pattern is always useful</span> | + | * <span style="font-size: 16px">[[TAKE SMALL STEPS]]: this pattern is always useful</span> |
<br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Issues]] / Back to [[Test Automation Issues]]</span></div> | <br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Issues]] / Back to [[Test Automation Issues]]</span></div> | ||
Revision as of 16:02, 27 June 2018
.................................................................................................................Main Page / Back to [Design Issues]] / Back to Test Automation Issues
.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues
Issue Summary
Preparing the expected results is slow and difficult
Category
Design
Examples
- Results can only be compared on the GUI
- Results are spread through different databases or media and it's difficult to put them together in a meaningful way
- Results change randomly
- Results change from release to release
- Results depend on what test cases have been run before
- API or AI tests deliver non-predictable results
Questions
How complex are the automated test cases? Can they be split up?
Are complex "scenario" test cases really necessary?
Resolving Patterns
Most recommended:
- FRESH SETUP: apply this pattern if you have issues like Example 5
- DEDICATED RESOURCES: look up this pattern if you have to share resources
- THINK OUT-OF-THE-BOX: try to look at the problem from unusual viewpoints. This is especially important for issues like Example 6
- WHOLE TEAM APPROACH: if your team follows an agile development process this is the pattern to use in order to avoid this kind of problems from the beginning
Other useful patterns:
- SHARE INFORMATION: use this pattern for better communication between development, testing and automation
- DESIGN FOR REUSE: this pattern should be used always
- COMPARE WITH PREVIOUS VERSION: this pattern helps when nothing else does
- TAKE SMALL STEPS: this pattern is always useful
.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues