Selenium Grid 2 - Up and Running

WebDriver and Grid 2 WebDriver is the latest greatest API for testing browsers, and is incorporated into Selenium 2 in a variety of languages, including Python. I'm building it into my ptest framework as well, in the WebDriverLib class in the weblib.py module.

The plan is to move all the legacy Selenium 1 tests to Grid 2

Since all of my tests are in Selenium 1 and WebDriver is a completely different API (woo hoo) I have a translation layer to map the existing tests to the new API.

cd $TEST_HOME/src
python
   then, in the python shell ...

 from weblib import WebDriverLib settings = {"browser": "*iehta"}
lib = WebDriverLib(settings)
email = settings.get('email', 'auto_test@your.co.com')
password = settings.get('password', 'autotest1')
dev_url = settings.get("dev_url", "https://your.co.com")
lib.open(dev_url) # translation layer for 'get'
lib.wait_and_type("//input[@id='Email']", email) # translation layer for find element and send_keys
lib.wait_and_type("//input[@id='Password']", password)
lib.wait_and_click("//a[@class='ui-button submit']") # translation layer for find element and click it
lib.wait_for_page_to_load(30000)

 Grid 2
BASICS:
 to launch the hub locally:
    java -jar selenium-server-standalone-2.15.0.jar -role hub

to launch a node (formerly 'remote control') locally:
    java -jar selenium-server-standalone-2.15.0.jar -role node -port 5555 -hubHost localhost -hubPort 4444 -hub http://localhost:4444/grid/register

 running on my qa-jenkins machine at port 4445:
hub:
     nohup java -jar selenium-server-standalone-2.15.0.jar -role hub -port 4445 node: java -jar selenium-server-standalone-2.15.0.jar -role node -port 5555 -hubHost qa-jenkins -hubPort 4445 -hub http://qa-jenkins:4445/grid/register node

multiple firefox nodes on the remote control machine (note the different versions and executable paths specified)
     java -jar selenium-server-standalone-2.15.0.jar -role node -port 2333 -hubHost qa-jenkins -hubPort 4445 -hub http:// qa-jenkins :4445/grid/register -browser browserName=firefox,version=3.6,firefox_binary=c:\progra~2\mozill~2\firefox.exe,maxInstances=1,platform=WINDOWS
and
    java -jar selenium-server-standalone-2.15.0.jar -role node -port 2888 -hubHost  qa-jenkins  -hubPort 4445 -hub http:// qa-jenkins :4445/grid/register -browser browserName=firefox,version=8,firefox_binary=c:\progra~2\mozill~1\firefox.exe,maxInstances=1,platform=WINDOWS

for ptest config:
 -b *firefox3 will invoke the 3.6 node, and *firefox8 will invoke the 8 node ptest
example:
 python ptest.py -c webtest_config.json -t test_webdriver -w true -b *firefox8

 Internet Explorer node:
     java -jar selenium-server-standalone-2.15.0.jar -role node -hub http://qa-jenkins:4445/grid/register -browser browserName=iexplore,version=9,platform=WINDOWS -port 8000

 Google Chrome node: (note the path is to the chromedriver executable, not to an installed Chrome browser. Also, the path to the chromedriver executable must be set in the PATH environment variable)
 java -jar selenium-server-standalone-2.15.0.jar -role node -port 6000 -hubHost qa-jenkins -hubPort 4445 -hub http://qa-jenkins:4445/grid/register -browser browserName=chrome,chrome_binary=C:\chromedriver\chromedriver,maxInstances=1,platform=WINDOWS

 see http://code.google.com/p/selenium/wiki/ChromeDriver for more information

For SSL Issues with Google Chrome, initialize the WebDriver client with a chrome.switches array as below (python bindings example) ->

                    from selenium import webdriver

                    dc = webdriver.DesiredCapabilities.CHROME
                    dc["chrome.switches"] = ["--ignore-certificate-errors"]
                    driver = webdriver.Remote(str(remote_url), self.dc)

Selenium and SSL

If you have to test https sites using Selenium on a variety of browsers, you are asking for a world of pain!  Practically every version of every browser on every platform requires either a different jar file or different arguments for both the selenium server and the remote client.

First of all, the all-important version disclaimer!
   selenium-server-standalone-2.15.0.jar
   selenium-grid-1.0.8
   Firefox 3.6  on Windows 7 and Ubuntu 11.04
   Firefox 8 on Windows7
   Chrome 15.0.874.121 on Windows 7
   Internet Explorer 9 on Windows 7

For Local Selenium Server
-------------------------------------
 java -jar selenium-server-standalone-2.15.0.jar


For Selenium RC Clients (your test app)
-----------------------------------------------------
  call selenium.start with the commandLineFlag -disable-web-security

  Python example:
    from selenium import selenium
    sel = selenium(test_host, int(test_port), self.browser, url)
    sel.start('commandLineFlags=-disable-web-security')

For Selenium Servers and Selenium Grid Remote Clients
---------------------------------------------------------------------------

For Firefox, I needed to use a profile and pass it into the startup script.
  1. start firefox from the command-line with "firefox -profileManager" and create a new profile
  2. Manually go into the site using Firefox, accept the various certificate challenges, and then quit the browser
  2. Start the selenium client using your new firefox profile. For example, as a Selenium Grid remote client:
           ant -Dport=5777 -Denvironment="*chrome" -Dhost=MY_IP
 -DhubURL=http://SELENIUM_GRID_SERVER_IP:4444 -DseleniumArgs="-firefoxProfileTemplate C:\Users\ME\FIREFOX_PROFILE_DIRECTORY_COPY\firefox_profile"  launch-remote-control

you may need to add some lines to the prefs.js file in the firefox profile. for example, if you see 403's and/or 404's being returned because of the browser looking for /favico.ico, you should add these two lines
    user_pref("browser.chrome.favicons", false);
    user_pref("browser.chrome.site_icons", false);
NOTE that this USUALLY WORKS but, in my case, it simply STOPPED WORKING on Windows 7. The -firefoxProfileTemplate argument passed in to ant as seleniumArgs is not passed along by the remote control to the firefox startup command. Here the specified profile is simply ignored:
    [java] 09:16:47.113 INFO - Preparing Firefox profile...
     [java] 09:16:49.072 INFO - Launching Firefox...
     [java] 09:16:49.074 DEBUG - Execute:Java13CommandLauncher: Executing 'C:\Program Files
(x86)\Mozilla Firefox\firefox.exe' with arguments:
     [java] '-profile'
     [java] 'C:\Users\ME\AppData\Local\Temp\customProfileDirc829f057ff9e497ea065add1ca892726'


The solution was to replace the selenium-server-standalone jar file in selenium-grid-XX/vendor with a newer one from Selenium org (2.15) - However, this broke IE which needed selenium server standaloine 2.12.0 !!!)


For Internet Explorer
    Disable popup blockers - Select Tools/Popup Blocker/Turn off pop-up blocker
    Disable IE protected mode - Untick Tools/Internet Options/Security/Enable protected mode - do this for all four zones

For Google Chrome

    Open Chrome Options

    go to 'Under the Hood'

    click on the 'Manage Certificates' button at HTTPS/SSL

    IMPORT your https server's PFX certificate and save it under Trusted Root Certification Authorities (input the password when prompted)

Selenium xpath arrays

Accessing xpath array elements in Selenium can be tricky! Sometimes you may have a number of elements on a page that can only be referenced with the same locator (for example, you can't rely on unique id's, only on some field they all have in common)

example:

a website has 4 buttons with 'addToCart' in onClick being the only reliable bit to check:

  ("//button[contains(@onclick,'addToCart')])

  you might think you could click on them by subscripting like this:

  sel.click("//button[contains(@onclick,'addToCart')][1]")

  sel.click("//button[contains(@onclick,'addToCart')][2]")

but no. the first one succeeds and fakes you into thinking you can access the array this way. but you can't. if you take out the [1] you get the same result. 

So what you have to do is isolate the array first, using the 'xpath=' notation, and then subscript it:

  sel.click("xpath=(//button[contains(@onclick,'addToCart')])[3]")

notice the use of 'xpath=' and the extra parens around the locator, followed by the subscript!

Reading Outlook Email With Python

There are several ways to read Outlook Email with Python, and I scouted around a number of blogs and websites to piece together a method that worked for me. This may not work for you, depending on how your company sets up its Outlook, but here it is, as a Python class

here is some usage ...

  outlook = OutlookLib()
  messages = outlook.get_messages('you@yourcompany.com')
  for msg in messages:
      print msg.Subject
      print msg.Body

and here is the class ...

import win32com.client

class OutlookLib:
        
    def __init__(self, settings={}):
        self.settings = settings
        
    def get_messages(self, user, folder="Inbox", match_field="all", match="all"):      
        outlook = win32com.client.Dispatch("Outlook.Application")
        myfolder = outlook.GetNamespace("MAPI").Folders[user] 
        inbox = myfolder.Folders[folder] # Inbox
        if match_field == "all" and match =="all":
            return inbox.Items
        else:
            messages = []
            for msg in inbox.Items:
                try:
                    if match_field == "Sender":
                        if msg.SenderName.find(match) >= 0:
                            messages.append(msg)
                    elif match_field == "Subject":
                        if msg.Subject.find(match) >= 0:
                            messages.append(msg)
                    elif match_field == "Body":
                        if msg.Body.find(match) >= 0:
                            messages.append(msg)
                    #print msg.To
                    #msg.Attachments
                        # a = item.Attachments.Item(i)
                        # a.FileName
                except:
                    pass
            return messages
        
    def get_body(self, msg):
        return msg.Body
    
    def get_subject(self, msg):
        return msg.Subject
    
    def get_sender(self, msg):
        return msg.SenderName
    
    def get_recipient(self, msg):
        return msg.To
    
    def get_attachments(self, msg):
        return msg.Attachments

Selenium RC and Confirmation Dialogs

This is worth noting. Sometimes in Selenium RC you will click on a button and it will bring up a confirmation dialog (Are You Sure You Want To Do This?)

Here's how to handle it in python-selenium


sel.choose_ok_on_next_confirmation() # will 'ok' the next one that comes up
sel.click('link='Delete') # your delete button
sel.get_confirmation() # absorbs the confirmation dialog

Simplify with Django

As a legacy from a previous job, I had continued using Ruby on Rails as a web front-end to display test regression results and other data. There is a lot about Ruby on Rails that I never really understood (such as 'routes'), and the sheer number of folders and files created by a new Rails project always intimidated me. After all, I am not doing much with it, so why should I have all this extra stuff I never even seem to need? Yet, I did grow fond of Ruby itself, so I didn't mind the Rails wilderness too much, as long as I could get my little stuff to work.


In a subsequent job, I needed to learn Python, and I found the transition from Ruby to be pretty easy. There are a few Ruby-isms I missed but for the most part I was fine with Python. I still used a variant on the old Rails project, but decided to explore Django, since it's Python as well, and it's always better to simplify, if you can. What I didn't expect, though, was just how much simpler Django was going to be than Rails.


My projects, as I said, are very simple. Regression test results (and other data, such as apache benchmarks and server performance monitoring data) are stored in simply MySQL databases. I want access to these results through a browser. Most of the results are presented in HTML tables. Others are displayed in Google Charts.


in Rails I needed separate files for each database table's corresponding controllers, models and and views. I also touched the databse.yml file, the routes.rb and migrate files. I don't really have anything in the app/helpers or lib or log or public or script or test or tmp or vendor directories, yet there they are! In Rails, the views directories for each table contain index, new, show, edit and delete html files. In sum, there are more than 40 files to edit and maintain in my bare-bones Rails project.


In the Django version, there are the main 3 files (manage,py, settings.py and urls.py), and then one file for all models (models.py) and one file for all views (views.py). I have html template files for each table, so that's another 5 - for a total of 10 files. There are no unused folders, no other clutter.


And that's not all. In Rails, I had to write MySQL scripts to extract the data and Ruby code to pass the results up through the controller to the view, stuff like this:

def self.find_history
    date = (Date.today-30).to_s
    stmt = "select *
            from (select server, date
                   from ads f1
                   group by server
                   order by date desc) f1
            left join ads f2 on f1.server=f2.server and f2.date > '#{date}'"
            
     result = find_by_sql(stmt)
     @server = Array.new
     @mean1 = Array.new
     @mean2 = Array.new
     @mean3 = Array.new
     @date = Array.new
     
     result.each do | a |
        @server << a.server
        @mean1 << a.mean1
        @mean2 << a.mean2
        @mean3 << a.mean3
        p = a.date.to_s.split(" ")
        @date << p[0]
      end
      
      return result
  end

In Django, no. I just define the db in the models file:

class Benchmarks(models.Model):
    server = models.CharField(max_length=128)
    users = models.CharField(max_length=32)
    mean1 = models.CharField(max_length=32)
    mean2 = models.CharField(max_length=32)
    mean3 = models.CharField(max_length=32)
    date = models.DateTimeField()

And in the views file I use a built-in method to get the data from MySQL, then use a 'context' to pass it along to a 'template' file which will be rendered by the browser

def benchmark_index(request):
    all_entries = Regression.objects.all()
    t = loader.get_template('benchmark/index.html')
    c = Context({
        'all_entries': all_entries,
    })
    return HttpResponse(t.render(c))

The template is similar in look-and-feel to embedded Ruby. You put Python code in between {% and %} markers, and the rest is html:


table here:
tr
th>Server
th>Users
th>Mean Response Time 1
th>Mean Response Time 2
th>Mean Response Time 3
/tr

{% if all_entries %}
{% for a in all_entries %}
tr>
td>{{ a.server }}
td>{{ a.users }}
td>{{ a.mean1 }}
td>{{ a.mean2 }}
td>{{ a.mean3 }}
td>{{ a.date }}
{% endfor %}
{% endif %}

/tr>




Notice that 'all_entries' - the variable used in the template, was explicitly defined in the views file and passed into the context. Also, the urls used by Django are explicitly defined, formed by regular expressions, and stored all together in a file called urls.py:

urlpatterns = patterns('',
    # Examples:
    (r'^benchmarks/$', 'regression.views.benchmark_index'),


)

It makes it clear that the url is ROOT/benchmarks, and when you go there, you invoke the method 'benchmark_index' in the views.py file in the 'regression' application. It's all quite explicit and easy to track.


This is most of the project in a nutshell. A simple db with values easily retrieved and displayed in an HTML table in a browser.


bonus coverage: The google charts aspect was also straightforward. 1) Install the Python module: sudo easy_install -U GChartWrapper 2) embed some google chart code in your template file 3) create and pass the data to the chart from the views.py file


Building on the example above, add to the benchmark_index and template file:

def benchmark_index(request):
    all_entries = Benchmarks.objects.all()
    data = []
    max_val = 0
    for a in all_entries:
        if a.mean3 > max_val:
            max_val = a.mean3
        data.append(a.mean3)
    #print max_val
    mid_val = float(max_val) / 2
    t = loader.get_template('benchmark/index.html')
    c = Context({
        'all_entries': all_entries,
        'data': data,
        'max_val': max_val,
        'mid_val': mid_val,
    })
    return HttpResponse(t.render(c))




// in the template, below the sample code above

{% load charts %}
{% chart Line data %}
{% title 'Max Mean Response Times' 0000FF 36 %}
{% color 3072F3 %}
{% line 3 2 0 %}
{% size 600 200 %}
{% axes type xy %}
{% scale 0 max_val %}
{% marker 's' 'blue' 0 -1,5 %}%}
{% legend 'Mean Response Times' %}
{% axes range 1 0,max_val %}
{% axes label 0 %}
{% axes label 1 0 mid_val max_val %}
{% img alt=DataScaling height=200 id=img title=DataScaling %}
{% endchart %}



The result:

Nothing fancy, but there you have it. Why complicate your life? Simplify with Django and Python.

Customizing Post-Build Email Notifications in Jenkins

Jenkins, the successor to Hudson, is a general purpose jobs-management console in Java. Lately I've been using this tool a lot to create and monitor nightly automation tests. After a job is run, I like to get an email notification, and in this notification, I want only a summary of the results, a custom-parsing of my console output. It happens that in my console output I've printed each test's results with a line that looks like this: RESULT ==> PASSED: nameOfTest or RESULT ==> FAILED: nameOfTest. All I want in the email is the list of these lines.

The default post-build email job in Jenkins is not very configurable. Fortunately there is an extensible email plugin which provides more functionality. To use it, though, you need to become at least a little familiar with something called 'jelly' - Java/XML hybrid language that apparently belongs to the Maven project.

The plugin comes with a couple of 'jelly' scripts which demonstrate a lot of stuff, including how to paste the console output into your post-build email notification. I only wanted to paste part of the output, though, and had to modify the original script. This was tricky only until I understood that within this jelly language, you call normal Java functions.

Eventually, I did have  success parsing the console output using the jelly script. I made a copy of the html.jelly included in the email-ext, found in /var/lib/jenkins/plugins/email-ext/WEB-INF/classes/hudson/plugins/emailext/templates

at the bottom of html.jelly is this section: <br />
<j:getstatic classname="hudson.model.Result" field="FAILURE" var="resultFailure">
<j:if test="${build.result==resultFailure}">


<j:foreach items="${build.getLog(100)}" var="line"></j:foreach>
<table cellpadding="0" cellspacing="0"><tbody>
<tr><td class="bg1">CONSOLE OUTPUT</td></tr>
<tr><td class="console">${line}</td></tr>
</tbody></table>
</j:if></j:getstatic>


I wanted to parse out any line containing the word 'RESULT' regardless of whether the build passed or failed, so I removed the build.result lines and added an if statement inside the forEach loop. Reading the last 100 lines of the console output (retrieved by build.getLog), I tested each line for 'RESULT' using the Java String indexOf function, and when that resulted in 'true', I printed out the line
<br />
<j:foreach items="${build.getLog(100)}" var="line">
<j:if test="${line.indexOf('RESULT')&gt;=0}">
</j:if></j:foreach>


<table cellpadding="0" cellspacing="0"><tbody>
<tr><td class="console">${line}</td></tr>
</tbody></table>
</pre>
<br />
<br />
in the Jenkins job configuration (Editable Email Notification, in the post-build options), I selected HTML output and referenced this new, modified file (I called it custom_html2.jelly)
in DEFAULT CONTENT, like this: <br />
${JELLY_SCRIPT,template="custom_html2"} <br />
<br />
On post-build, the plugin uses the custom_html2.jelly to prepare and include the parsed output into the email notification