Difference between revisions of "HARD-TO-AUTOMATE"
Jump to navigation
Jump to search
(Created page with "<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.........................................................................................") |
m (Topic titles in capital letters) |
||
| Line 1: | Line 1: | ||
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Issues]] / Back to [[Test Automation Issues]]</span><br /> | <div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Issues]] / Back to [[Test Automation Issues]]</span><br /> | ||
| − | =<span style="font-size: 16px">Issue summary</span>= | + | =<span style="font-size: 16px">'''Issue summary'''</span>= |
<span style="font-size: 16px">The Software Under Test (SUT) doesn’t support automated testing very well or not at all. </span><br /> <br /> | <span style="font-size: 16px">The Software Under Test (SUT) doesn’t support automated testing very well or not at all. </span><br /> <br /> | ||
| − | =<span style="font-size: 16px">Category</span>= | + | =<span style="font-size: 16px">'''Category'''</span>= |
<span style="font-size: 16px">Design</span><br /> <br /> | <span style="font-size: 16px">Design</span><br /> <br /> | ||
| − | = <span style="font-size: 16px">Examples</span>= | + | = <span style="font-size: 16px">'''Examples'''</span>= |
<span style="font-size: 16px">The problem range is:</span><br /> | <span style="font-size: 16px">The problem range is:</span><br /> | ||
| − | |||
# <span style="font-size: 16px">Non-identifiable (uniquely) GUI-Objects; caused by lack of unique name or other property to make it unique. </span> | # <span style="font-size: 16px">Non-identifiable (uniquely) GUI-Objects; caused by lack of unique name or other property to make it unique. </span> | ||
# <span style="font-size: 16px">Third Party or Customized GUI-Objects that are not recognized properly or are not 'open' (exposed methods and properties).</span> | # <span style="font-size: 16px">Third Party or Customized GUI-Objects that are not recognized properly or are not 'open' (exposed methods and properties).</span> | ||
| Line 20: | Line 19: | ||
# <span style="font-size: 16px">The SUT is very slow so it takes a lot of time to test the automation scripts</span> | # <span style="font-size: 16px">The SUT is very slow so it takes a lot of time to test the automation scripts</span> | ||
<br /> | <br /> | ||
| − | =<span style="font-size: 16px">Questions</span>= | + | =<span style="font-size: 16px">'''Questions'''</span>= |
<span style="font-family: Arial; font-size: 16px">Where does the complexity lie? In preparing the initial conditions? Test execution? Checking the results? Dependencies?</span><br /> <span style="font-family: Arial; font-size: 16px">Do you have the right resources? What do you need?</span><br /> <span style="font-family: Arial; font-size: 16px">Can you simulate hardware? or software?</span><br /> <span style="font-size: 16px">Has the current tool been selected especially for the SUT or has it been "inherited" from previous applications?</span><br /> <span style="font-size: 16px">Do the developers know about the automation problems? Do they care?</span><br /> <span style="font-size: 16px">Do the developers have guidelines/standards for building in 'testability'? If not, why not?.</span><br /> <span style="font-size: 16px">How do manual testers check the results?</span><br /> <span style="font-size: 16px">How complex are the test cases?</span><br /> <span style="font-size: 16px">How do you collect the necessary data?</span><br /> <br /> | <span style="font-family: Arial; font-size: 16px">Where does the complexity lie? In preparing the initial conditions? Test execution? Checking the results? Dependencies?</span><br /> <span style="font-family: Arial; font-size: 16px">Do you have the right resources? What do you need?</span><br /> <span style="font-family: Arial; font-size: 16px">Can you simulate hardware? or software?</span><br /> <span style="font-size: 16px">Has the current tool been selected especially for the SUT or has it been "inherited" from previous applications?</span><br /> <span style="font-size: 16px">Do the developers know about the automation problems? Do they care?</span><br /> <span style="font-size: 16px">Do the developers have guidelines/standards for building in 'testability'? If not, why not?.</span><br /> <span style="font-size: 16px">How do manual testers check the results?</span><br /> <span style="font-size: 16px">How complex are the test cases?</span><br /> <span style="font-size: 16px">How do you collect the necessary data?</span><br /> <br /> | ||
| − | =<span style="font-size: 16px">Resolving Patterns</span>= | + | =<span style="font-size: 16px">'''Resolving Patterns'''</span>= |
<span style="font-size: 16px">Most recommended:</span><br /> | <span style="font-size: 16px">Most recommended:</span><br /> | ||
| − | |||
* <span style="font-size: 16px">[[DO A PILOT]]: use this pattern to find out what the problems are and ways to tackle them</span> | * <span style="font-size: 16px">[[DO A PILOT]]: use this pattern to find out what the problems are and ways to tackle them</span> | ||
* <span style="font-size: 16px">[[RIGHT TOOLS]]: use this pattern if you are confronting issues like Examples 1 or 2.</span> | * <span style="font-size: 16px">[[RIGHT TOOLS]]: use this pattern if you are confronting issues like Examples 1 or 2.</span> | ||
| Line 32: | Line 30: | ||
* <span style="font-size: 16px">[[VARIABLE DELAYS]]: use this pattern for issues Like Example 4</span> | * <span style="font-size: 16px">[[VARIABLE DELAYS]]: use this pattern for issues Like Example 4</span> | ||
<br /> <span style="font-size: 16px">Other useful patterns:</span><br /> | <br /> <span style="font-size: 16px">Other useful patterns:</span><br /> | ||
| − | |||
* <span style="font-size: 16px">[[EASY TO DEBUG FAILURES]]. use this pattern for issues like Example 8.</span> | * <span style="font-size: 16px">[[EASY TO DEBUG FAILURES]]. use this pattern for issues like Example 8.</span> | ||
* <span style="font-size: 16px">[[GET ON THE CLOUD]]: this is the pattern of choice for issues like Example 7.</span> | * <span style="font-size: 16px">[[GET ON THE CLOUD]]: this is the pattern of choice for issues like Example 7.</span> | ||
Revision as of 18:13, 29 April 2018
.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues
Points 1 and 2 in Examples are contributed by Jim Hazen, Point 10 by Thorsten Schönfelder.
Issue summary
The Software Under Test (SUT) doesn’t support automated testing very well or not at all.
Category
Design
Examples
The problem range is:
- Non-identifiable (uniquely) GUI-Objects; caused by lack of unique name or other property to make it unique.
- Third Party or Customized GUI-Objects that are not recognized properly or are not 'open' (exposed methods and properties).
- Random processing
- Fluctuating response times.
- The expected results change after each new version of the SUT
- Different systems (hardware or software) that interact in some way.
- Tests have to run on a steadily growing number of browsers or environments
- Test results consist in entries in many database tables or files related to each other.
- For embedded systems or mobile devices results are difficult to check because of timing issues or not very visible results (e.g. intermediate results)
- Test data is hard to create due to the number of systems involved or a limited refresh cycle.
- Unexpected pop-ups
- The SUT is very slow so it takes a lot of time to test the automation scripts
Questions
Where does the complexity lie? In preparing the initial conditions? Test execution? Checking the results? Dependencies?
Do you have the right resources? What do you need?
Can you simulate hardware? or software?
Has the current tool been selected especially for the SUT or has it been "inherited" from previous applications?
Do the developers know about the automation problems? Do they care?
Do the developers have guidelines/standards for building in 'testability'? If not, why not?.
How do manual testers check the results?
How complex are the test cases?
How do you collect the necessary data?
Resolving Patterns
Most recommended:
- DO A PILOT: use this pattern to find out what the problems are and ways to tackle them
- RIGHT TOOLS: use this pattern if you are confronting issues like Examples 1 or 2.
- TAKE SMALL STEPS: use this pattern to break up the problems in more chewable chunks
- TESTABLE SOFTWARE: use this pattern for issues similar to Examples 1. 2. and 3.
- THINK OUT-OF-THE-BOX: try to look at the problem from unusual viewpoints
- VARIABLE DELAYS: use this pattern for issues Like Example 4
Other useful patterns:
- EASY TO DEBUG FAILURES. use this pattern for issues like Example 8.
- GET ON THE CLOUD: this is the pattern of choice for issues like Example 7.
- KEEP IT SIMPLE: this pattern is always helpful
- SHARE INFORMATION: use this pattern for better communication between development, testing and automation (Examples 1, 2, 4., 5., 11. and 12.)
.................................................................................................................Main Page / Back to Design Issues / Back to Test Automation Issues
Points 1 and 2 in Examples are contributed by Jim Hazen, Point 10 by Thorsten Schönfelder.