Tag Archives: selenium

WebDriver

Posted on by

Faster than a speeding bullet! Easier to maintain than something that’s really easy to maintain! Reliable! That’s what we want from our tests, but how do we get there? This presentation covers key strategies and patterns for writing test suites using WebDriver, a developer focused tool for web application testing similar in spirit to Selenium RC. We’ll cover why it was written, the problems it addresses and how to integrate it into your projects and testing process.

This talk presented by Simon Stewart, creator of WebDriver, still serves as a useful introduction to the tool, even though the talk itself is a couple of years old and WebDriver has moved on since then.

I’ve been experimenting with WebDriver as an alternative to Selenium/Selenium RC, although it is worth bearing in mind that both projects are merging. I’m enjoying getting to grips with WebDriver and am finding that the tight integration between WebDriver and each of the browsers it automates is much faster than Selenium, since it uses the mechanism most appropriate for controlling that browser. For example in Firefox, WebDriver is implemented as an extension, whereas in IE, WebDriver makes use of IE’s Automation controls. In addition to Firefox and IE, WebDriver also supports the following:

  • Safari 3 on OSX
  • iPhone
  • Selenium Emulation

It’s the Selenium Emulation I’d like to touch upon here, what this basically means is that the Java version of WebDriver provides an implementation of the existing Selenium API, and therefore can an be dropped in as a substitute for the Selenium Driver. Here’s an example of how you’d do this:

  1.  
  2. // You may use any WebDriver implementation. Firefox is used here as an example
  3. // A "base url", used by selenium to resolve relative URLs
  4. "http://www.google.com";
  5.  
  6. // Create the Selenium implementation
  7. // Perform actions with selenium
  8. selenium.open("http://www.google.com");
  9. selenium.type("name=q", "cheese");
  10. selenium.click("name=btnG");
  11.  
  12. // And get the underlying WebDriver implementation back. This will refer to the
  13. // same WebDriver instance as the "driver" variable above.

This allows for WebDriver and Selenium to live side-by-side, and provides developers with a simple mechanism for a managed migration from the existing Selenium API to WebDriver. I’m still experimenting with it but I have to admit I really like it simplicity.

Automated regression testing and CI with Selenium PHP

Posted on by

I’ve been doing some work this iteration on getting Selenium RC integrated into our build process so we can run a suite of automated functional regression tests against our application on each build. The application I’m working on is written in PHP, normally when you use Selenium IDE to record a test script it saves it as a HTML file.

For example a simple test script that goes to Google and verifies that the text “Search:” is present on the screen and the title of the page is “iGoogle” looks like this:

  1.  
  2. <html>
  3. <head>
  4. <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  5. <title>New Test</title>
  6. </head>
  7. <body>
  8. <table cellpadding="1" cellspacing="1" border="1">
  9. <thead>
  10. <tr><td rowspan="1" colspan="3">New Test</td></tr>
  11. </thead><tbody>
  12. <tr>
  13.     <td>open</td>
  14.     <td>/ig?hl=en</td>
  15.     <td></td>
  16. </tr>
  17. <tr>
  18.     <td>verifyTextPresent</td>
  19.     <td>Search:</td>
  20.     <td></td>
  21. </tr>
  22. <tr>
  23.     <td>assertTitle</td>
  24.     <td>iGoogle</td>
  25.     <td></td>
  26. </tr>
  27.  
  28. </tbody></table>
  29. </body>
  30. </html>
  31.  

You can choose to export the script in several other languages, including PHP, in which case the test script it produces looks like this:

  1. span class=”st0″>’Testing/Selenium.php’‘PHPUnit/Framework/TestCase.php’"*firefox", "http://localhost:4444/""/ig?hl=en""Search:""iGoogle"

The Export produces a valid PHPUnit test case that uses the Selenium PHP Client Driver(Selenium.php). Whilst the script is valid and will run you do need add a little more to it before the test will correctly report errors. As it stands all errors captured during the test are added to an array called verificationErrors, by catching the assertion Exceptions that are thrown when an assert fails, in other words if you ran this test as it is, and it did fail you wouldn’t know! To correct this we need to do two things. Firstly, each assert needs to have a message added to it which will printed out in the test report if the assert fails. Secondly we need to modify the tearDown method so that once a test has run, it checks the verificationErrors array, and if any failures have occurred, fails the test. After making these changes the PHP test script looks like this:

  1. span class=”st0″>’Testing/Selenium.php’‘PHPUnit/Framework/TestCase.php’"*firefox",
  2.                          "http://localhost:4444/""VERIFICATION ERRORS:""\n""/ig?hl=en""Search:"),
  3.                "The string Search: was not found""iGoogle",
  4.                    $this->selenium->getTitle(),
  5.                    "The page title did not match iGoogle."

Obviously, I have also given the PHP Class and test function slightly more meaningful names. Now you have a PHP Unit Test case that will use the Selenium PHP Client Driver with Selenium Remote Control to launch a browser, go to the specified URL, and test a couple of assertions. If any of those assertions fail, the tearDown method fails the test … pretty cool, right?

Well now it get’s better. Because the Selenium Client Driver has a published api which is pretty easy to follow, there’s no reason why you can’t just write test cases without using Selenium IDE … for those who want to you could even incorporate this into a TDD process. But for all this to hang together we need to be able to run a build, on a Continuous Integration server which checks out the code, runs unit tests and selenium regression tests against that code line, and only if all tests succeed passes the build.

We are currently using ANT, and CruiseControl to handle our CI/Automated build process. When running the automated suite of tests we need to ensure that the Selenium Remote Control server is also running which creates some complications. The Selenium Remote Control server takes several arguments which can also include the location of a test suite of html based selenium tests – which is really nice because the server will start, execute those tests and then end. Unfortunately you can’t invoke the server and pass it the location of a PHP based test suite. This means you need to find a way to start up the server, then run your tests, and once they are complete, shut the selenium server down.

He are the ANT targets that I have written to achieve this, if anyone can think of better ways of doing this I’d welcome any feedback or suggestions, to run this example you’d simply enter the command “ant selenium” :

  1.  
  2. <target name="selenium" depends="clean, init" description="Run the Selenium tests">
  3.   <parallel>
  4.     <antcall target="StartRCServer" />
  5.     <antcall target="RunSeleniumTests" />
  6.   </parallel>
  7. </target>
  8.                
  9. <target name="StartRCServer" description="Start the Selenium RC server">
  10.   <java jar="dependencies/SeleniumRC/lib/selenium-server.jar"
  11.         fork="true" failonerror="true">
  12.     <jvmarg value="-Dhttp.proxyHost=host.domain.com"/>
  13.     <jvmarg value="-Dhttp.proxyPort=80"/>
  14.   </java>
  15. </target>
  16.    
  17. <target name="RunSeleniumTests" description="RunAllSeleniumTests">    
  18.   <sleep milliseconds="2000" />
  19.   <echo message="======================================" />
  20.   <echo message="Running Selenium Regression Test Suite" />
  21.   <echo message="======================================" />
  22.   <exec executable="php"
  23.        failonerror="false"
  24.        dir="test/seleniumtests/regressiontests/"
  25.        resultproperty="regError">
  26.        <arg line="../../../dependencies/PHPUnit/TextUI/Command.php –log-xml ../../../doc/SeleniumTestReports/RegressionTests/TestReport.xml AllRegressionTests" />
  27.   </exec>
  28.   <get taskname="selenium-shutdown"
  29.     src="http://localhost:4444/selenium-server/driver/?cmd=shutDown"
  30.     dest="result.txt" ignoreerrors="true" />
  31.                    
  32.   <condition property="regressionTest.err">
  33.     <or>
  34.        <equals arg1="1" arg2="${regError}" />
  35.        <equals arg1="2" arg2="${regError}" />
  36.     </or>
  37.   </condition>
  38.  
  39.   <fail if="regressionTest.err" message="ERROR: Selenium Regression Tests Failed" />               
  40. </target>
  41.  

A couple of notes, the reason I have to use a conditional check at the end of the selenium target is because, if the exec task that runs the PHP tests was set to failonerror=true then the build would never reach the next line which shuts the Selenium RC server down. To ensure that always happens I have to set the exec to failonerror=false, but this means I have to check what the result was from the exec. Which if successful will return 0, if test failures exist will return 1, and if there were any errors (preventing a test to be exectuted ) will return 2. Hence the conditional check sets regressionTest.err if either of these latter two conditions are true.

Also in order to start up the server, which could take up to a second, but can’t be sure precisely how long. I have to use the Ant Parallel task, which calls the task to start the server and the task to run the tests at the same time. The task to run the tests has a 2 second sleep in it, which should be more than enough time to allow the server to start. This all kind of feels a little clunky, but at the moment it does work very well.

In a nutshell, thats how you integrate PHP based Automated Selenium Regression tests into a continuous build.