View source for SET STANDARDS

<div id="content_view" class="wiki" style="display: block"><span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Process Patterns]] / Back to [[Test Automation Patterns]]</span><br /> 
=<span style="font-size: 16px">Pattern summary</span>=
<span style="font-size: 16px">Set and follow standards for the automation artefacts.</span>
=<span style="font-size: 16px">Category</span>=
<span style="font-size: 16px">Process</span>
=<span style="font-size: 16px">Context</span>=
<span style="font-size: 16px">This pattern is appropriate for long-lasting automation. It is essential for larger organisations and for large-scale automation efforts. This pattern is not needed for single-use scripts.</span>
=<span style="font-size: 16px">Description</span>=
<span style="font-size: 16px">Set and follow standards: otherwise when many people work on the same project it can easily happen that everyone uses their own methods and processes. Team members do not “speak the same language” and hence cannot efficiently </span><span style="font-size: 16px; line-height: 24px">share </span><span style="font-size: 16px">their work; you get </span>''<span style="font-size: 16px">[[OBSCURE TESTS]]</span>''<span style="font-size: 16px"> or </span>''<span style="font-size: 16px">[[SCRIPT CREEP]]</span>''<span style="font-size: 16px">.</span><br /> <span style="font-size: 16px">As an extra bonus, standards make it easier for new team members to integrate into the team.</span>
=<span style="font-size: 16px">Implementation</span>=
<span style="font-size: 16px">Some suggestions for what you should set standards for:</span><br /> <br /> <span style="font-size: 16px">Naming conventions:</span><br /> 

* <span style="font-size: 16px">Suites: the names should convey what kind of test cases are contained in each test suite.</span>
* <span style="font-size: 16px">Scripts: if the names are not consistent, an existing suitable script may not be found so a duplicate may be written.</span>
* <span style="font-size: 16px">Keywords: it’s important that the name immediately conveys the functionality implemented by the keyword.</span>
* <span style="font-size: 16px">Data files: it should be possible to recognize from the name what the file is for and its status.</span>
* <span style="font-size: 16px">If you implement [[OBJECT MAP]], the right names facilitate understanding of the scripts and enable you to change tools without having to rewrite all of your automation scripts (only the tool-specific ones).</span>
* <span style="font-size: 16px">Test data: if possible use the same names or IDs in all data files, as this will facilitate reuse.</span>
<br /> <span style="font-size: medium">Organisation of testware:</span><br /> 

* <span style="font-size: 16px">Test Definition: Define a standard format or template to document all automated tests, for example as a standard block of comment, where the information is presented in a consistent way across all tests (e.g. same titles and levels of indentation). This should contain the following:</span>
** <span style="font-size: 16px">Test case ID or name</span>
** <span style="font-size: 16px">What this test does</span>
** <span style="font-size: 16px">Materials used (scripts, data files etc)</span>
** <span style="font-size: 16px">Set-up instructions</span>
** <span style="font-size: 16px">How it is called (input variables if any)</span>
** <span style="font-size: 16px">Execution instructions</span>
** <span style="font-size: 16px">What it returns (including output variables)</span>
** <span style="font-size: 16px">Tear-down instructions</span>
** <span style="font-size: 16px">Length (how long it takes to run)</span>
** <span style="font-size: 16px">Related tests</span>
** <span style="font-size: 16px">[[TEST SELECTOR]] tags</span>
** <span style="font-size: 16px">Any other useful information such as EMTE (Equivalent Manual Test Effort - how long this test would have taken manually)</span>
* <span style="font-size: 16px">File and folder organisation and naming conventions</span>
<br /> <span style="font-size: medium">Other standards:</span><br /> 

* <span style="font-size: 16px">Documentation conventions for scripts and batch files</span>
* <span style="font-size: 16px">Coding conventions for scripts </span>
* <span style="font-size: 16px">Data anonymization rules</span>
* <span style="font-size: 16px">Develop a [[TEMPLATE TEST]]</span>
<br /> <span style="font-size: 16px">Other advice:</span><br /> 

* Document the standards!
* <span style="font-size: 16px">Standards should be reviewed periodically in order to adjust or enhance them.</span>
* <span style="font-size: 16px">Put your standards in a Wiki so that everybody can access them at any time.</span>
* <span style="font-size: 16px">If something has to be changeable, use a translation table so that the scripts can stay stable.</span>
* <span style="font-size: 16px">Allow exceptions when needed.</span>
* <span style="font-size: 16px">Setting standards for test data, for example using the test ID as the customer's name, can help to [[VISUALIZE EXECUTION]] as a way of monitoring test progress.</span>
=<span style="font-size: 16px">Potential problems</span>=
<span style="font-size: 16px">When devising standards, it is useful to get input from a selection of people who will be using the automation, to make sure that the conventions adopted will serve all of its users well. An additional benefit of getting others involved is that they will be more supportive of something that they helped to devise. Not many people like having standards imposed arbitrarily when they can't see the reason for them.</span><br /> <br /> <span style="font-size: 16px">Once you have settled on your standards, however, then you need to make sure that everyone does use them in a consistent way, and this will take effort.</span>
=<span style="font-size: 16px">Issues addressed by this pattern</span>=
''<span style="font-size: 16px">[[CAN'T FIND WHAT I WANT]]</span>''<br /> ''<span style="font-size: 16px">[[INADEQUATE DOCUMENTATION]]</span>''<br /> ''<span style="font-size: 16px">[[LOCALISED REGIMES]]</span>''<br /> ''<span style="font-size: 16px">[[OBSCURE TESTS]]</span>''<br /> ''<span style="font-size: 16px">[[SCRIPT CREEP]]</span>''<br /> ''<span style="font-size: 16px">[[TOOL DRIVEN AUTOMATION]]</span>''
=<span style="font-size: 16px">Experiences</span>=
<span style="font-size: 16px">'Jochim Van Dorpe writes':</span><br /> <br /> <span style="font-size: 16px">One day I was put on a project that had been going on for more than two years. The first tests were automated in one of the first releases. However, for the past months they were only keeping the tests 'green' without adding new ones, although development was still ongoing.</span><br /> <br /> <span style="font-size: 16px"> The first reason: staffing has dramaticaly changed over the last two months:</span><br /> 

* <span style="font-size: 16px"> The old PL was a consultant who had been replaced by a new one</span>
* <span style="font-size: 16px">The developers were consultants who went away</span>
* <span style="font-size: 16px">Most of the analysts were sacked.</span>
<br /> <span style="font-size: 16px"> The second reason: the automated tests were chaotic!</span><br /> 

* <span style="font-size: 16px">Test suites were made per test designer/automator, not per grouping functionality of some sort.</span>
* <span style="font-size: 16px">Naming was meaningless: test01(), test02(), ...</span>
* <span style="font-size: 16px">No comment or documentation was added</span>
<br /> <span style="font-size: 16px">Those two reasons combined made that nobody understood what the tests did or what they tested, but the client ... he saw a grass-green chart every time he asked for test results.</span><br /> <br /> <span style="font-size: 16px"> I've made up some standards (together with the architect and an analyst) and explained them to all the automators, they contained the following simple rules:</span><br /> 

* <span style="font-size: 16px"> the lowest level test suites (who are translated to one java test class) contain the tests of 1 Use case, 1 data flow document, 1 end-2-end-test, ...</span>
* <span style="font-size: 16px"> naming conventions:</span>
** <span style="font-size: 16px">first characters: UC/DF/E2E defines if we are testing a use case, dataflow, ...</span>
** <span style="font-size: 16px">followed by the number of the document</span>
** <span style="font-size: 16px">followed by TC</span>
** <span style="font-size: 16px">followed by the number of the case in the lowest level test suite</span>
** <span style="font-size: 16px">followed by the identifier in the testtool (which is generated automatically)</span>
<br /> <br /> <span style="font-size: 16px"> For example UC20TC04_abc124() translates to the fourth test case of use case 20, which is test abc124 in the tool</span><br /> <br /> <span style="font-size: 16px">A similar set of standards was made for test analysis and design so we could exactly map the high level test cases with those that were implemented by the automators.</span><br /> <span style="font-size: 16px"> They included:</span><br /> 

* <span style="font-size: 16px">a nested structure of the test suites:</span>
** <span style="font-size: 16px">type on the highest level</span>
** <span style="font-size: 16px">use case, data flow, functionality or other grouping and identifying element on the lowest level </span>
* <span style="font-size: 16px">naming conventions of the test cases</span>
* <span style="font-size: 16px">numbering of the test cases</span>
* <span style="font-size: 16px">a way to present the prerequisites</span>
* <span style="font-size: 16px">a way to present the actions to be taken</span>
* <span style="font-size: 16px">a way to present the expected results</span>
* <span style="font-size: 16px">a test importance execution level (going from 1 to 4) </span>
* <span style="font-size: 16px">keywords to add for searching (like: automated, MSS, ...)</span>
* <span style="font-size: 16px">some extra info like the issue, change request, ... or any other reason why the test was added</span>
<br /> <span style="font-size: 16px"> We also added a set of standards for commenting in and around the tests so even an a non-technical person could easily understand what a certain line of test-code does or what it asserts.</span><br /> <br /> <span style="font-size: 16px"> These simple rules added the following advantages:</span><br /> 

* <span style="font-size: 16px">readability: everybody knows what a certain test does, and how it does what it does</span>
* <span style="font-size: 16px">maintainability: Not only the original automator can adapt the automated tests</span>
* <span style="font-size: 16px">Now we know what the tests test</span>
* <span style="font-size: 16px"> We have a clear view of which high level test cases are automated and which aren't </span>
<br /> <span style="font-size: 16px"> We also added similar guidelines for:</span><br /> 

* <span style="font-size: 16px">the size and content (like id-naming conventions)of datasets</span>
* <span style="font-size: 16px">the amount of 'things' a script or test case chould contain.</span>
<br /> <br /> <br /> <br /> <span style="font-size: 16px">If you have used this pattern, please add your name and a brief story of how you used this pattern: your context, what you did, and how well it worked - or how it didn't work!</span><br /> <span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Process Patterns]] / Back to [[Test Automation Patterns]]</span><br /> <br />  B12 </div>