Difference between revisions of "KEYWORD-DRIVEN TESTING"

From Test Automation Patterns
Jump to navigation Jump to search
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] / Back to [[Test Automation Patterns]]</span>
 
<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Design Patterns]] / Back to [[Test Automation Patterns]]</span>
 
=<span style="font-size: 16px">'''Pattern Summary'''</span>=
 
=<span style="font-size: 16px">'''Pattern Summary'''</span>=
<span style="font-size: 16px">Tests are driven by keywords that represent actions of a test, and may include input data and expected results.</span>
+
<span style="font-size: 16px">Tests are driven by keywords (also called action words) that represent actions of a test, and may include input data and expected results.</span>
 
=<span style="font-size: 16px">'''Category'''</span>=
 
=<span style="font-size: 16px">'''Category'''</span>=
 
<span style="font-size: 16px">Design</span>
 
<span style="font-size: 16px">Design</span>
 
=<span style="font-size: 16px">'''Context'''</span>=
 
=<span style="font-size: 16px">'''Context'''</span>=
<span style="font-family: Arial; font-size: 16px">This pattern is appropriate:</span><br /> <br />
+
<span style="font-family: Arial; font-size: 16px">This pattern is appropriate:</span><br />
 
 
 
* <span style="font-size: 16px">When you want to write test cases that are practically independent from the Software under Test (SUT). If the SUT changes, the functionality behind the keyword must be adapted, but most of the time the test cases themselves are still valid.</span>
 
* <span style="font-size: 16px">When you want to write test cases that are practically independent from the Software under Test (SUT). If the SUT changes, the functionality behind the keyword must be adapted, but most of the time the test cases themselves are still valid.</span>
 
* <span style="font-size: 16px">When testers should be able to write and run automated tests even if they are not adept with the automation tools.</span>
 
* <span style="font-size: 16px">When testers should be able to write and run automated tests even if they are not adept with the automation tools.</span>
Line 12: Line 11:
 
<br /> <span style="font-family: Arial; font-size: 16px">The pattern is not appropriate for very small-scale or one-off automation efforts.</span>
 
<br /> <span style="font-family: Arial; font-size: 16px">The pattern is not appropriate for very small-scale or one-off automation efforts.</span>
 
=<span style="font-size: 16px">'''Description'''</span>=
 
=<span style="font-size: 16px">'''Description'''</span>=
<span style="font-size: 16px">Keywords are the verbs of the language that the tester uses to specify tests, typically from a business or domain perspective. A keyword specifies a sequence of actions together with any required input data and/or expected results. </span><br /> <br /> <br />
+
<span style="font-size: 16px">Keywords are the verbs of the language that the tester uses to specify tests, typically from a business or domain perspective. A keyword specifies a sequence of actions together with any required input data and/or expected results. </span><br />
 
 
 
 
 
{| class="wikitable sortable"
 
{| class="wikitable sortable"
 
|-
 
|-
Line 53: Line 50:
 
|-
 
|-
 
|}
 
|}
<br /> <br /> <span style="font-size: 16px">Keywords are most powerfully used at a high level, representing a business domain. Different domains would have different keywords. High level keywords for an insurance application, for example, might include "Create New Policy", "Process Claim" or "Renew Policy". High level keywords for a mobile phone, for example, might include "Make Call", "Update Contact" or "Send Text Message". There may be some</span><span style="font-size: 15.833333015441895px; line-height: 1.5"> keywords that are common across more than one domain, particularly at lower levels, such as "Log In" and "Print Page".</span>
+
<span style="font-size: 16px">Keywords are most powerfully used at a high level, representing a business domain. Different domains would have different keywords. High level keywords for an insurance application, for example, might include "Create New Policy", "Process Claim" or "Renew Policy". High level keywords for a mobile phone, for example, might include "Make Call", "Update Contact" or "Send Text Message". There may be some</span><span style="font-size: 15.833333015441895px; line-height: 1.5"> keywords that are common across more than one domain, particularly at lower levels, such as "Log In" and "Print Page".</span>
 +
 
 
=<span style="font-size: 16px">'''Implementation'''</span>=
 
=<span style="font-size: 16px">'''Implementation'''</span>=
<span style="font-size: 15.833333015441895px">Each keyword is processed by an associated script, which may call other reusable scripts from a library. A keyword script can be written in a common coding language, as calls to other keywords or in the scripting language of the tool. It is a good idea to keep the tool-specific scripts to a minimum as this will help to reduce script maintenance costs. The keyword script reads and processes the input data for the keyword, and/or checks the expected output. The implementation of the architecture of the keywords is often referred to as a </span><span style="font-size: 16px; line-height: 1.5">[[TEST AUTOMATION FRAMEWORK]]</span>, <span style="font-size: 16px">which determines the choice of scripting language and how composite keywords are defined.</span>
+
<span style="font-size: 15.833333015441895px">Each keyword is processed by an associated script, which may call other reusable scripts from a library. A keyword script can be written in a common coding language, as calls to other keywords, or in the scripting language of the tool. It is a good idea to keep the tool-specific scripts to a minimum as this will help to reduce script maintenance costs. The keyword script reads and processes the input data for the keyword, and/or checks the expected output. The implementation of the architecture of the keywords is often referred to as a </span><span style="font-size: 16px; line-height: 1.5">[[TEST AUTOMATION FRAMEWORK]]</span>, <span style="font-size: 16px">which determines the choice of scripting language and how composite keywords are defined.</span>
 +
 
 
=<span style="font-size: 16px">'''Potential problems'''</span>=
 
=<span style="font-size: 16px">'''Potential problems'''</span>=
 
<span style="font-size: 16px">Take care that the keywords describe a clear and cohesive action. If the keywords build the “words” for a domain specific language (as in [[DOMAIN-DRIVEN TESTING]]), your testers will easily be able to write automation test cases and will be able to start even before the SUT is available for running tests.</span><br /> <span style="font-size: 16px">Another problem can come up when you need too many parameters (input data) for a given keyword: it becomes unwieldy if you have to scroll repeatedly in order to see or input all the parameters. Break the unwieldy keyword into shorter independent functions (which can call and be called from the others).</span>
 
<span style="font-size: 16px">Take care that the keywords describe a clear and cohesive action. If the keywords build the “words” for a domain specific language (as in [[DOMAIN-DRIVEN TESTING]]), your testers will easily be able to write automation test cases and will be able to start even before the SUT is available for running tests.</span><br /> <span style="font-size: 16px">Another problem can come up when you need too many parameters (input data) for a given keyword: it becomes unwieldy if you have to scroll repeatedly in order to see or input all the parameters. Break the unwieldy keyword into shorter independent functions (which can call and be called from the others).</span>
Line 61: Line 60:
 
<span style="font-size: 16px">''[[BRITTLE SCRIPTS]]''</span><br /> ''<span style="font-size: 16px">[[LATE TEST CASE DESIGN]]</span>''<br /> ''<span style="font-size: 16px">[[MANUAL MIMICRY]]</span>''<br /> ''<span style="font-size: 16px">[[NON-TECHNICAL-TESTERS]]</span>''<br /> ''<span style="font-size: 16px">[[TOO EARLY AUTOMATION]]</span>''
 
<span style="font-size: 16px">''[[BRITTLE SCRIPTS]]''</span><br /> ''<span style="font-size: 16px">[[LATE TEST CASE DESIGN]]</span>''<br /> ''<span style="font-size: 16px">[[MANUAL MIMICRY]]</span>''<br /> ''<span style="font-size: 16px">[[NON-TECHNICAL-TESTERS]]</span>''<br /> ''<span style="font-size: 16px">[[TOO EARLY AUTOMATION]]</span>''
 
=<span style="font-size: 16px">'''Experiences'''</span>=
 
=<span style="font-size: 16px">'''Experiences'''</span>=
<span style="font-size: 16px">If you have used this pattern, please add your name and a brief story of how you used this pattern: your context, what you did, and how well it worked - or how it didn't work!</span><br />
+
 
=<span style="font-size: 16px">Example</span>=
+
<span style="font-size: 16px">Example from Seretta:</span><br />  
<span style="font-size: 16px">Seretta:</span><br />  
 
 
{| class="wiki_table"
 
{| class="wiki_table"
 
| <span style="font-size: 16px">We use a variation of this pattern that we called Command-Driven Testing. Here is how it works:</span><br /> <span style="font-size: 16px">First of all, and that’s why we gave it its name, our keywords are not words in a domain-specific language, but just plain commands like “SELECT,Button” or “INPUT”. These commands drive the test tool and they have to be written in the proprietary script language of the tool. The functional scripts are written in a kind of Meta language that calls the commands. An interpreter script in the tool script language parses the Meta scripts and executes the appropriate commands. In this way, you can extract the test and application know-how from the proprietary tool scripts. This again means that when we had to migrate to a different tool, from QARun to TestComplete, we only needed to rewrite the interpreter script for the basic commands in TestComplete, and to remap our GUI-Objects in the [[OBJECT MAP]], but that otherwise the tests didn’t have to be touched. Another advantage was that we were thus able use the same “commands” (the interpreter scripts) for all our different applications (they all run in the same environment). </span><br /> <span style="font-size: 16px">Having studied the advantages of the pattern [[DATA-DRIVEN TESTING]] we devised a way to get the same benefits also for Command-Driven Testing: we split our command scripts into a DRIVER part and a DATA part. In the DRIVER-file the variable data is replaced with placeholders and in the DATA-file the placeholders are substituted with the actual data. In this way we can have any number of DATA files to one DRIVER file</span><span style="font-family: &#39;Arial&#39;,&#39;sans-serif&#39;; font-size: 14.6667px">.</span><br /> <span style="font-size: 16px">To make the scripting even more flexible the interpreter script ignores statements that contain placeholders that are not to be found in the corresponding DATA file ([[SKIP VOID INPUTS]])</span><br /> <br /> <span style="font-size: 16px">Here some examples of our Meta language:</span><br /> <br /> <span style="font-size: 16px">GOTO,FTestSuite</span><br />
 
| <span style="font-size: 16px">We use a variation of this pattern that we called Command-Driven Testing. Here is how it works:</span><br /> <span style="font-size: 16px">First of all, and that’s why we gave it its name, our keywords are not words in a domain-specific language, but just plain commands like “SELECT,Button” or “INPUT”. These commands drive the test tool and they have to be written in the proprietary script language of the tool. The functional scripts are written in a kind of Meta language that calls the commands. An interpreter script in the tool script language parses the Meta scripts and executes the appropriate commands. In this way, you can extract the test and application know-how from the proprietary tool scripts. This again means that when we had to migrate to a different tool, from QARun to TestComplete, we only needed to rewrite the interpreter script for the basic commands in TestComplete, and to remap our GUI-Objects in the [[OBJECT MAP]], but that otherwise the tests didn’t have to be touched. Another advantage was that we were thus able use the same “commands” (the interpreter scripts) for all our different applications (they all run in the same environment). </span><br /> <span style="font-size: 16px">Having studied the advantages of the pattern [[DATA-DRIVEN TESTING]] we devised a way to get the same benefits also for Command-Driven Testing: we split our command scripts into a DRIVER part and a DATA part. In the DRIVER-file the variable data is replaced with placeholders and in the DATA-file the placeholders are substituted with the actual data. In this way we can have any number of DATA files to one DRIVER file</span><span style="font-family: &#39;Arial&#39;,&#39;sans-serif&#39;; font-size: 14.6667px">.</span><br /> <span style="font-size: 16px">To make the scripting even more flexible the interpreter script ignores statements that contain placeholders that are not to be found in the corresponding DATA file ([[SKIP VOID INPUTS]])</span><br /> <br /> <span style="font-size: 16px">Here some examples of our Meta language:</span><br /> <br /> <span style="font-size: 16px">GOTO,FTestSuite</span><br />
<span style="font-size: 16px"> INPUT,ComboBox,cboPriority,<Priority></span><br /> <span style="font-size: 16px"> INPUT,ComboBox,cboTestType,<TestType></span><br /> <span style="font-size: 16px"> SELECT,Button,<ButtonDRIVER></span><br /> <span style="font-size: 16px"> GOTO,<SelectDirDRIVER></span><br /> <span style="font-size: 16px"> INPUT,edtDirectory,<DRIVERDirName></span><br /> <span style="font-size: 16px"> SELECT,Button,<ConfirmSelectionDRIVER></span><br /> <br />  
+
<span style="font-size: 16px"> INPUT,ComboBox,cboPriority,<Priority></span><br /> <span style="font-size: 16px">INPUT,ComboBox,cboTestType,<TestType></span><br /><span style="font-size: 16px"> SELECT,Button,<ButtonDRIVER></span><br /><span style="font-size: 16px">GOTO,<SelectDirDRIVER></span><span style="font-size: 16px">INPUT,edtDirectory,<DRIVERDirName></span><br /><span style="font-size: 16px"> SELECT,Button,<ConfirmSelectionDRIVER></span><br /><br />  
<span style="font-size: 16px">The capitalized words are our keywords. In the example above we have GOTO, INPUT and SELECT. Certain keywords, such as SELECT have to be further specified, because you can select for instance a button or a menu or a line in a list. </span><br /> <span style="font-size: 16px">The words in the square brackets represent the placeholders that can be given in the external DATA-file</span><br />
+
<span style="font-size: 16px">The capitalized words are our keywords. In the example above we have GOTO, INPUT and SELECT. Certain keywords, such as SELECT have to be further specified, because you can select for instance a button or a menu or a line in a list. </span><br /> <span style="font-size: 16px">The words in the square brackets represent the placeholders that can be given in the external DATA-file</span><br /><br />
 
|}
 
|}
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[ Design Patterns]] / Back to [[Test Automation Patterns]]</span><br /> <span style="font-size: 14px">B3</span></div>
+
 
 +
<span style="font-size: 16px">If you have also used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br />
 +
 
 +
 
 +
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[ Design Patterns]] / Back to [[Test Automation Patterns]]</span>

Latest revision as of 09:18, 4 October 2018

.................................................................................................................Main Page / Back to Design Patterns / Back to Test Automation Patterns

Pattern Summary

Tests are driven by keywords (also called action words) that represent actions of a test, and may include input data and expected results.

Category

Design

Context

This pattern is appropriate:

  • When you want to write test cases that are practically independent from the Software under Test (SUT). If the SUT changes, the functionality behind the keyword must be adapted, but most of the time the test cases themselves are still valid.
  • When testers should be able to write and run automated tests even if they are not adept with the automation tools.
  • If you want testers to start writing test cases for automation before the (SUT) is available to test.


The pattern is not appropriate for very small-scale or one-off automation efforts.

Description

Keywords are the verbs of the language that the tester uses to specify tests, typically from a business or domain perspective. A keyword specifies a sequence of actions together with any required input data and/or expected results.

Keyword Input Data Expected Results
1 2 ... n 1 ...
Action 1 Data 1 Data 1.2 ... ...
Action 2 Data 2 Data 2.2 ... Data 2.n Result 1 ...
... ... ... ... ... ... ...

Keywords are most powerfully used at a high level, representing a business domain. Different domains would have different keywords. High level keywords for an insurance application, for example, might include "Create New Policy", "Process Claim" or "Renew Policy". High level keywords for a mobile phone, for example, might include "Make Call", "Update Contact" or "Send Text Message". There may be some keywords that are common across more than one domain, particularly at lower levels, such as "Log In" and "Print Page".

Implementation

Each keyword is processed by an associated script, which may call other reusable scripts from a library. A keyword script can be written in a common coding language, as calls to other keywords, or in the scripting language of the tool. It is a good idea to keep the tool-specific scripts to a minimum as this will help to reduce script maintenance costs. The keyword script reads and processes the input data for the keyword, and/or checks the expected output. The implementation of the architecture of the keywords is often referred to as a TEST AUTOMATION FRAMEWORK, which determines the choice of scripting language and how composite keywords are defined.

Potential problems

Take care that the keywords describe a clear and cohesive action. If the keywords build the “words” for a domain specific language (as in DOMAIN-DRIVEN TESTING), your testers will easily be able to write automation test cases and will be able to start even before the SUT is available for running tests.
Another problem can come up when you need too many parameters (input data) for a given keyword: it becomes unwieldy if you have to scroll repeatedly in order to see or input all the parameters. Break the unwieldy keyword into shorter independent functions (which can call and be called from the others).

Issues addressed by this pattern

BRITTLE SCRIPTS
LATE TEST CASE DESIGN
MANUAL MIMICRY
NON-TECHNICAL-TESTERS
TOO EARLY AUTOMATION

Experiences

Example from Seretta:

We use a variation of this pattern that we called Command-Driven Testing. Here is how it works:
First of all, and that’s why we gave it its name, our keywords are not words in a domain-specific language, but just plain commands like “SELECT,Button” or “INPUT”. These commands drive the test tool and they have to be written in the proprietary script language of the tool. The functional scripts are written in a kind of Meta language that calls the commands. An interpreter script in the tool script language parses the Meta scripts and executes the appropriate commands. In this way, you can extract the test and application know-how from the proprietary tool scripts. This again means that when we had to migrate to a different tool, from QARun to TestComplete, we only needed to rewrite the interpreter script for the basic commands in TestComplete, and to remap our GUI-Objects in the OBJECT MAP, but that otherwise the tests didn’t have to be touched. Another advantage was that we were thus able use the same “commands” (the interpreter scripts) for all our different applications (they all run in the same environment).
Having studied the advantages of the pattern DATA-DRIVEN TESTING we devised a way to get the same benefits also for Command-Driven Testing: we split our command scripts into a DRIVER part and a DATA part. In the DRIVER-file the variable data is replaced with placeholders and in the DATA-file the placeholders are substituted with the actual data. In this way we can have any number of DATA files to one DRIVER file.
To make the scripting even more flexible the interpreter script ignores statements that contain placeholders that are not to be found in the corresponding DATA file (SKIP VOID INPUTS)

Here some examples of our Meta language:

GOTO,FTestSuite

INPUT,ComboBox,cboPriority,<Priority>
INPUT,ComboBox,cboTestType,<TestType>
SELECT,Button,<ButtonDRIVER>
GOTO,<SelectDirDRIVER>INPUT,edtDirectory,<DRIVERDirName>
SELECT,Button,<ConfirmSelectionDRIVER>

The capitalized words are our keywords. In the example above we have GOTO, INPUT and SELECT. Certain keywords, such as SELECT have to be further specified, because you can select for instance a button or a menu or a line in a list.
The words in the square brackets represent the placeholders that can be given in the external DATA-file

If you have also used this pattern and would like to contribute your experience to the wiki, please go to Feedback to submit your experience or comment.


.................................................................................................................Main Page / Back to Design Patterns / Back to Test Automation Patterns