View source for AUTOMATE WHAT'S NEEDED

<div id="content_view" class="wiki" style="display: block">
<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Process Patterns]] / Back to [[Test Automation Patterns]]</span> 
=<span style="font-size: 16px">'''Pattern summary'''</span>=
<span style="font-size: 16px">Automate what the developers or the testers need, even if it isn’t tests!</span> 
=<span style="font-size: 16px">'''Category'''</span>=
<span style="font-size: 16px">Process</span> 
=<span style="font-size: 16px">'''Context'''</span>=
<span style="font-size: 16px">This pattern is appropriate when your automated tests will be around for a long time, but also when you just want to write one-off or disposable scripts.</span><br /> <span style="font-size: 16px">Automating what isn't needed is never a good idea!</span> 
=<span style="font-size: 16px">'''Description'''</span>=
<span style="font-size: 16px">Automate the tests that will give the most value. "Smoke tests" that are run every time there is any change, for example, would be good candidates for automation, as they would be run frequently and give confidence that the latest change has not destroyed everything.</span><br /> <span style="font-size: 16px">When you think about test automation, the first tests that usually come to mind are regression tests that execute themselves at night or on week-ends. Actually with automation one can support testers in many other ways, because even when testing manually (even with exploratory tests) there are lots of repetitive tasks that could be easily automated. You may need only a little script in Python or SQL to make a tester’s life that much easier.</span><br /> <br /> <span style="font-size: 16px">'''James Tony''': Keep a balance between trying to test all the code paths (which is generally good) and trying to test every possible combination of inputs (which is generally bad, because it means the same lines of code get executed thousands of times, and the test suite takes so long that it doesn’t get used) – the aim should be to maximize the (customer-relevant) “bugs for your buck”, i.e. the maximum number of customer-relevant issues highlighted for the smallest expenditure of time and money</span> 
=<span style="font-size: 16px">'''Implementation'''</span>=
<span style="font-size: 16px">Some suggestions:</span><br /> 
* <span style="font-size: 16px">[[AUTOMATE GOOD TESTS]]: Automate only the tests that bring the most Return on Investment (ROI)</span>
* <span style="font-size: 16px">[[KNOW WHEN TO STOP]]: Not all test cases can or should be automated</span>
* <span style="font-size: 16px">[[SHARE INFORMATION]]: what do testers need? What could you deliver them? Start supporting them there.</span>
* <span style="font-size: 16px">Try to get at least some developers "into the boat".</span>
<br /> <span style="font-size: 16px">Good candidates for automation in addition to tests are:</span><br /> 
* <span style="font-size: 16px">Complex set-ups: they can be easily automated and are great time savers for testers.</span>
* <span style="font-size: 16px">DB-Data: Database-data can be automatically extracted or loaded for use in creating initial conditions or checking results. Such support is valued by developers and testers alike.</span>
=<span style="font-size: 16px">'''Potential problems'''</span>=
<span style="font-size: 16px">When people get "stuck in" to automation, they can get carried away with what can be done, and may want to automate tests that aren't really important enough to automate.</span> 
=<span style="font-size: 16px">'''Issues addressed by this pattern'''</span>=
<span style="font-size: 16px">''[[INADEQUATE COMMUNICATION]]''</span><br /> <span style="font-size: 16px">''[[NO PREVIOUS TEST AUTOMATION]]''</span><br /> ''<span style="font-size: 16px">[[UNFOCUSED AUTOMATION]]</span>''<br /> <br /> 
=<span style="font-size: 16px">'''Experiences'''</span>=
<span style="font-size: 16px">'''Example 1'''</span>
<br /> <br /> <span style="font-size: 16px">''Jochim Van Dorpe'' writes:</span><br /> <br /> <span style="font-size: 16px">I like the idea/theory of automated tests vs. automated testing. That's why I look as much as possible for things that we could automate in our test process, things I used to do manually, but are repetitive, boring and not interesting. So we automated some stuff, some right from the beginning of the project, others I still come up with.</span><br /> <br /> <span style="font-size: 16px">Some solutions are interesting for me as a test-analyst, others are for my Project Lead (PL) or even for the client. Some are offered by the tools we use, some are custom-made by our team.</span><br /> <br /> <span style="font-size: 16px">Some examples are:</span><br /> 
* <span style="font-size: 16px">Before any tests begin, we automatically drop the test Database (DB)</span>
* <span style="font-size: 16px">We populate the DB automatically with 'the things we want': typecodes, dataset, ...</span>
* <span style="font-size: 16px">Of course we have the scripts that run the tests ...</span>
* <span style="font-size: 16px">Tests (unitils, selenium, soapUi) are automatically run after a new build</span>
* <span style="font-size: 16px">Test results are automatically transferred from the continuous integration server that executes them (jenkins) to the tool that contains the test suites with high level test cases (testlink)</span>
* <span style="font-size: 16px">HTML- test reports are made automatically on a scheduled basis, or on demand</span>
* <span style="font-size: 16px">A readable document of all the test cases can be composed on demand (testlink)</span>
* <span style="font-size: 16px">A TSH (test scenario/case hierarchy) can be composed automatically</span>
<br /> <span style="font-size: 16px">The custom-made things are re-usable by other projects in our company.</span><br /> <br /> <br /> 
<span style="font-size: 16px">'''Example 2'''</span>
<br /> <span style="font-size: 16px">''Gaby Spengler'' describes how she helps testers find out if a test case has already been implemented:</span><br /> 
{| class="wiki_table"
| <span style="font-size: 16px">Sometimes testers don’t know if a test case for a specific requirement has already been implemented. I have written a macro in Excel that allows testers to search the test case directory for a specified keyword and that creates a list of all the test cases that contain the keyword.</span><br /> <br /> <span style="font-size: 16px">Here is how the macro is coded:</span><br />
| <br />
|-
| <span style="font-size: 16px">'Public full_name As String</span><br /> <span style="font-size: 16px">Public TFDir As String</span><br /> <span style="font-size: 16px">Public Searchtext As String</span><br /> <span style="font-size: 16px">'Public Makro As String</span><br /> <span style="font-size: 16px">Sub TF_Search()</span><br /> <br /> <span style="font-size: 16px">'The directory to be searched and the search argument are taken from the register „Testcase search“.</span><br /> <span style="font-size: 16px"> TFDir = ""</span><br /> <span style="font-size: 16px"> Worksheets("Testcase search").Activate</span><br /> <span style="font-size: 16px"> TFDir = Cells(2, 1)</span><br /> <span style="font-size: 16px"> Searchtext = Cells(5, 1)</span><br /> <br /> 
<span style="font-size: 16px">'The following procedure browses thru the test case directory and looks for test cases that contain the Searchtext and lists them in the register “Testcase search”</span><br /> <span style="font-size: 16px"> TFFiles_Search TFDir</span><br /> <br /> <span style="font-size: 16px">End Sub</span><br /> <br /> <span style="font-size: 16px">Sub TFFiles_Search(TFDir)</span><br /> <span style="font-size: 16px">'</span><br /> <span style="font-size: 16px">'This procedure browses thru the test case directory and looks for test cases that contain the Searchtext and lists them in the register “Testcase search”</span><br /> <span style="font-size: 16px">'</span><br /> <span style="font-size: 16px">Dim LineSource As Long</span><br /> <span style="font-size: 16px">Dim LineDest As Long</span><br /> <br /> <span style="font-size: 16px">Dim This_Workbook As String</span><br /> <span style="font-size: 16px">Dim Searched_Workbook As String</span><br /> <br /> <span style="font-size: 16px">Dim WSHShell, fso, oArgs</span><br /> <span style="font-size: 16px">Dim oFolders, oSubFolder, oFiles, Folder</span><br /> <span style="font-size: 16px">Dim i, Text, FileX, TCFile()</span><br /> <br /> <br /> <span style="font-size: 16px">'</span><br /> <span style="font-size: 16px"> This_Workbook = "Makro Test case search V1.0.xls"</span><br /> <span style="font-size: 16px"> Searched_Workbook = "Testcases INK.xls"</span><br /> <span style="font-size: 16px"> Set WSHShell = CreateObject("WScript.Shell")</span><br /> <span style="font-size: 16px"> Set fso = CreateObject("Scripting.FileSystemObject")</span><br /> <span style="font-size: 16px">' Set oArgs = Arguments</span><br /> <br /> <span style="font-size: 16px"> i = 0</span><br /> <span style="font-size: 16px"> Set oFolders = fso.GetFolder(TFOrdner)</span><br /> <span style="font-size: 16px"> Set oFiles = oFolders.Files</span><br /> <br /> <span style="font-size: 16px"> For Each FileX In oFiles</span><br /> <span style="font-size: 16px"> ' Test case file names are stored in the array TCFile</span><br /> <span style="font-size: 16px"> Text = Mid(FileX.Name, 1, 9)</span><br /> <span style="font-size: 16px"> If Text = "Testcases" Then</span><br /> <span style="font-size: 16px"> ReDim Preserve TCFile(i)</span><br /> <span style="font-size: 16px"> TCFile(i) = TFDir + "\" + FileX.Name</span><br /> <span style="font-size: 16px"> i = i + 1</span><br /> <span style="font-size: 16px"> End If</span><br /> <span style="font-size: 16px"> Next</span><br /> <br /> <span style="font-size: 16px"> ' Test cases that contain the sought text are inserted starting with line 10 </span><br /> <span style="font-size: 16px"> LineDest = 10</span><br /> <br /> <span style="font-size: 16px"> For i = 0 To UBound(TCFile)</span><br /> <span style="font-size: 16px"> 'Open the test case file</span><br /> <span style="font-size: 16px"> Workbooks.Open TCFile(i)</span><br /> <br /> <br /> <span style="font-size: 16px"> Searched_Workbook = TCFile(i)</span><br /> <br /> <span style="font-size: 16px"> Length = Len(TCFile(i))</span><br /> <span style="font-size: 16px"> pos1 = 1</span><br /> <span style="font-size: 16px"> Do Until Length = 0</span><br /> <span style="font-size: 16px"> pos1 = InStr(1, Searched_Workbook, "\")</span><br /> <span style="font-size: 16px"> If pos1 = 0 Then</span><br /> <span style="font-size: 16px"> Exit Do</span><br /> <span style="font-size: 16px"> End If</span><br /> <span style="font-size: 16px"> Length = Length - pos1</span><br /> <span style="font-size: 16px"> Searched_Workbook = Mid(Searched_Workbook, (pos1 + 1), Length)</span><br /> <br /> <span style="font-size: 16px"> Loop</span><br /> <br /> <span style="font-size: 16px"> With Worksheets("Testcases").Range("A2:J1000")</span><br /> <span style="font-size: 16px"> Set c = .Find(Searchtext)</span><br /> <span style="font-size: 16px"> ' If the Searchtext is found then the test case is copied to the macro file</span><br /> <span style="font-size: 16px"> If Not c Is Nothing Then</span><br /> <span style="font-size: 16px"> firstAddress = c.Address</span><br /> <br /> <br /> <span style="font-size: 16px"> ' The search goes on until the first hit has been found again</span><br /> <span style="font-size: 16px"> Do</span><br /> <span style="font-size: 16px"> 'The line in which the Searchtext was found, is stored</span><br /> <span style="font-size: 16px"> LineSource = c.Row</span><br /> <span style="font-size: 16px"> Worksheets("Testcases").Activate</span><br /> <span style="font-size: 16px"> 
Rows(LineSource).Select</span><br /> <span style="font-size: 16px"> Selection.Copy</span><br /> <span style="font-size: 16px"> Workbooks(This_Workbook).Worksheets("Testcase_Search").Activate</span><br /> <span style="font-size: 16px"> Rows(LineDest).Select</span><br /> <span style="font-size: 16px"> 'Selection.Insert Shift:=xlDown</span><br /> <span style="font-size: 16px"> 'Selection.Insert</span><br /> <span style="font-size: 16px"> Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks _</span><br /> <span style="font-size: 16px"> :=False, Transpose:=False</span><br /> <span style="font-size: 16px"> ' Rows("10:10").Select</span><br /> <span style="font-size: 16px"> With Selection</span><br /> <span style="font-size: 16px"> .HorizontalAlignment = xlGeneral</span><br /> <span style="font-size: 16px"> .VerticalAlignment = xlTop</span><br /> <span style="font-size: 16px"> .WrapText = True</span><br /> <span style="font-size: 16px"> .Orientation = 0</span><br /> <span style="font-size: 16px"> .AddIndent = False</span><br /> <span style="font-size: 16px"> .IndentLevel = 0</span><br /> <span style="font-size: 16px"> .ShrinkToFit = False</span><br /> <span style="font-size: 16px"> .ReadingOrder = xlContext</span><br /> <span style="font-size: 16px"> .MergeCells = False</span><br /> <span style="font-size: 16px"> End With</span><br /> <br /> <span style="font-size: 16px"> LineDest = LineDest + 1</span><br /> <br /> <span style="font-size: 16px"> 
'Workbooks(TCFile(i)).Worksheets("Testcases").Activate</span><br /> <span style="font-size: 16px"> Workbooks(Searched_Workbook).Worksheets("Testcases").Activate</span><br /> <span style="font-size: 16px"> Set c = .FindNext(c)</span><br /> <span style="font-size: 16px"> Loop While Not c Is Nothing And c.Address <> firstAddress</span><br /> <span style="font-size: 16px"> End If</span><br /> <span style="font-size: 16px"> End With</span><br /> <span style="font-size: 16px"> Workbooks(Searched_Workbook).Worksheets("Testcases").Cells(1, 1).Copy</span><br /> <span style="font-size: 16px"> 'Close the testcase file</span><br /> <span style="font-size: 16px"> Workbooks(Searched_Workbook).Close SaveChanges:=False</span><br /> <span style="font-size: 16px"> '</span><br /> <span style="font-size: 16px"> Next</span><br /> <br /> <span style="font-size: 16px"> ' Set Autofilter</span><br /> <span style="font-size: 16px"> Workbooks(This_Workbook).Worksheets("Testcase search").Activate</span><br /> <span style="font-size: 16px"> Rows("9:9").Select</span><br /> <span style="font-size: 16px"> Selection.AutoFilter</span><br /> <br /> <span style="font-size: 16px"> End Sub</span><br />
| <br />
|}
<span style="font-size: 16px">'''Example 3'''</span>
<br /><span style="font-size: 16px">''Sergiusz Golec'' writes:</span><br /> <br /> <span style="font-size: 16px">One day a set of sample clients/partners observed a not desired end state for their transactions. "Hard to find & reproduce". Critical details & triggers - missing. Except the known day. Hard to reproduce & analyze.</span><br /> 
<span style="font-size: 16px">Drop from webapp debug logs contained 9 files around 90 MB. Jms message response content with an issue not to be covered with known log analyzers. Every file around 30'000 lines long, 10'486'203 characters long. Interesting details - spreaded among different hours (timestamps), contexts and transactions. As if extremely thin layer of a butter on a 90 MB slice of bread.</span><br /> 
<span style="font-size: 16px">Automatically it was half a day task. To write and check the needed outcome.</span><br /> 
<span style="font-size: 16px">Script read the input file(s), line by line, and with a function (fWorthOrNotWriting) based upon keywords (like interesting XML tags, Partner ID(s) (shown in a different formats), environment, transaction  type, state or known errors) started writing the desired output (into another file). Script collected the entire XML messages into a separate file. 808 lines from around 9*30'000 lines ( = 270 K lines).</span><br />
<span style="font-size: 16px">808 lines long content manually was trimmed and boiled eventually to the single perfect example: 36-lines-long XML. Alienated & identified issue. Easy to work with and fix by developers.</span><br />
<span style="font-size: 16px">Manually - it would make my soul cry for a week. To look for that specific issue.</span><br /><br /> 
<br /> <span style="font-size: 16px">If you have also used this pattern and would like to contribute your experience to the wiki, please go to [[Feedback]] to submit your experience or comment.</span><br /> <br /> 

<span style="font-size: 14px">.................................................................................................................[[Main Page]] / Back to [[Process Patterns]] / Back to [[Test Automation Patterns]]</span></div>