Two Python Open Source Projects for Test

I've recently added two open source projects to github, useful for software testing in python.

One is a general purpose test automation framework, suitable for running any and all tests. I've use it for web testing, api testing, serial port/firmware testing, network traffic scanning, tv remote control testing, game testing - all sorts of stuff

You can find it on github as pytaf26

The other is an abstraction layer for Selenium using Python bindings. I've found - to my consternation - that some things work better in Selenium RC than they do in WebDriver, and vice versa. In fact, in my current work I've had to switch between the two quite frequently. To ease the pain, I wrote an abstraction layer - two classes that use the same method signatures - so that  I only have to write the tests once, and pass along a command-line switch to determine which version of Selenium to use. The details of each version lie under the hood of the easy-to-remember api methods.


You can find it on github as py-selenium-layers

Pytaf

My new open source project - PYTAF (Python Test Automation Framework) - is in the process of being announced on Freecode.com. The project can be accessed directly through GitHub. It's a distillation of the various test harnesses I've developed over the past several years, beginning in Java, polished in Ruby, until finally fully formed in Python. This version is actually in Python 3, which I'm not using yet in everyday work. It's not much different, but developing this project was partly a mechanism for learning about it.


I'm happy to put it out there in the world, as happy as putting out any of my ebooks or earlier open-source projects. If one person finds it useful, that will be 'abundance'.

announcement also here on opencode

Porting PTest from Python 2.6 to Python 3.0

work-in-progress ... experiment porting my python automation test framework from Python 2.6 to Python 3.0 (on Windows 7, it should be noted)

python.org/ftp/python/3.2.2

---------------------
import changes
---------------------
  import httplib becomes import http.client as httplib
  import urllib2 becomes import urllib as urllib2
  import urllib also becomes import urllib as urllib

  import MySQLdb - no such module. download pymsql instead and, after installing, import pymysql

  lxml
   from http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml, lxml-2.3.4.win-amd64-py3.2.‌exe
    failed to install - couldn't find python3.2 in the freaking registry
  so ... not using lxml  (BeautifulSoup is way better anyway)

  import thread becomes import _thread

BeautifulSoup 4 - https://groups.google.com/forum/#!msg/beautifulsoup/VpNNflJ1rPI/sum07jmEwvgJ
and after installing
from BeautifulSoup import BeautifulSoup becomes   from bs4 import BeautifulSoup

import Queue becomes import queue

--------------------------
execution changes
--------------------------


#set the right TEST_HOME
set TEST_HOME=C:\Users\tlichtenberg\workspace\tlichtenberg_win\QA\ptest3

# use the right python
c:\python32\python ptest3.py -c api_config.json -t test_api

---------------------
code changes
---------------------

print statements are now functions
     print "this is my statement"
becomes
     print("this is my statement")

 MySQLdb.Connection(... = pymysql.connect(...

thread.get_ident() becomes _thread.get_ident()

http response.read() returns a bytes object, not a string. so it needs to be
response.read().decode() to turn it into a string

for k,v in dict.iteritems():
   print k,v
becomes
for (k,v) in dict.items():
    print(k,v)


--------------
unicode
________


all text is unicode
  u"\xe9" no longer supported
  backslashes in strings are now just backslashes, not escapes

? not sure about this yet ...
instead of u'\x80abc'.encode('utf-8')
                  b'\x80abc'.decode("utf-8", "replace")

Basic Django on Apache2

My Django project is called 'automation' and lives in the directory /home/tlichtenberg/workspace/ptest/django/src/automation

It has CSS files in the regression/static/styles directory under the home project. I needed to add some stuff to Apache to allow it to find the files it needs to run the Django project

--------------------------------

My Apache2 httpd.conf:


WSGIScriptAlias / /home/tlichtenberg/workspace/ptest/django/src/automation/django.wsgi

AliasMatch ^/([^/]*\.css) /home/tlichtenberg/workspace/ptest/django/src/automation/regression/static/styles/$1

Alias /static/ /home/tlichtenberg/workspace/ptest/django/src/automation/regression/static/

<Directory /home/tlichtenberg/workspace/ptest/django/src/automation/regression/static>
Order deny,allow
Allow from all
</Directory>

<Directory /home/tlichtenberg/workspace/ptest/django/src/automation>
Order deny,allow
Allow from all
</Directory>

------------------------------

The django,wsgi file is critical for setting the path and connecting Django to Apache. It looks like this:


import os
import sys

path = '/home/tlichtenberg/workspace/ptest/django/src/automation/'
if path not in sys.path:
   sys.path.append(path)

os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

------------------------------------------

The 'settings' defined by django,wsgi points to the Django project's 'settings.py' file, which defines a lot of the paths that Django needs. Also important is the 'urls.py' file, like settings.py, it is a standard for all Django projects and defines all the URLs associated with the web applications being served by Django. For example:

settings.py:

  STATIC_ROOT = '/home/tlichtenberg/workspace/ptest/django/src/automation/regression/static/'
  STATIC_URL = '/static/'
  ROOT_URLCONF = 'urls'
  INSTALLED_APPS = (
    'my_project',
    'my_other_project',
 )

urls.py
   (r'^my_project/$', 'my_project.views.my_project_main'),
defines that any url beginning with 'my_project' (e.g. http://localhost/my_project) will direct the code to the my_project_main method in the project's views.py file.

---------------------------------------------------


 

Pyjamas on Django for Web Automation

I've been working on developing a web application for use in automating some product testing. The commands being implemented are communicated via HTTP, so there isn't strictly a need for a browser interface just to test the api's, but certain functionality requires doing stuff on the device, then going to a different website and verifying some side effects (e.g. purchases recorded, emails sent, etc ...) Using a web application for these cases lets me do some automation using Selenium.

When I first thought about the design, I assumed I would do a Java Servlet or Java Server Pages, but since I've been mostly working in Python recently, I thought I'd see if there was anything I could do that way. Sure enough, I came across a project called Pyjamas, a Python port of Google Web Toolkit. I was interested right away, because not only was it Python, but it could run in Django, and I was already running a Django server on Apache2 on Ubuntu. This web app would theoretically plug right in. And it did, after some minor bumps in the road.

Pyjamas is a sort of miracle wonder drug, where you create your UI in Python, and then "build" it into JavaScript/HTML/Ajax using its 'pyjsbuild' tool. So you can make real-world web apps without having to look under the hood, so to speak. It provides a simple GUI kit, similar to Java AWT 1.0 - primitive but useful. If I was a better UI designer, the above would look a lot nicer, but since I'm primarily a backend coder, this will just have to do :}

Codewise, you import a lot of Pyjamas modules:
   from pyjamas.ui.TextBox import TextBox

and then go about building your UI, creating panels and sticking widgets into them. I won't go into great detail here. There are a lot of excellent examples in the Pyjamas download and elsewhere, but basically, something like this to get going ...

  class RemoteController:
     
    def onModuleLoad(self):
        self.remote = DataService()    
        dockPanel = DockPanel()
        ...
       # create all your widgets in here, add them to you main panel,
       # and then add the main panel to the RootPanel object
       self.myTextBox = HTML()
       self. myTextBox.setID('my_text_box')
       dockPanel.add( myTextBox )
       RootPanel().add(dockPanel)

The DataService class is interesting - this is what lets you connect into functions running on Django, and mine looks like this:
 from pyjamas.JSONService import JSONProxy
 class DataService(JSONProxy):
    def __init__(self):
        JSONProxy.__init__(self, "/services/", ["method_one", "method_two"])


In my Django views.py, I just have to define those methods:
  from django.pimentech.network import * # use the pimentech module described here to define the JSON service and decorator

  service = JSONRPCService()

 @jsonremote(service)
 def method_one(request):
    print >> sys.stderr,  'method_one called"
    request.session['ip'] = text # set the ip as specific to this session
    ecp.set_url(text)
    return "put this into my text box, please"

now in your Django urls.py you define the /services/ path used by the DataService class above:
  (r'^services/$', 'YOUR_PROJECT.views.service'),

when your method returns from Django, you handle it in your Pyjamas class like this:

 def onRemoteResponse(self, response, request_info):
        try:    
            if request_info.method == 'method_one':
                self.myMyTextBox.setText(response)

Other GUI stuff should be familiar if you've done any similar programming - with clickListeners, mouseListeners, keyListeners and so on.

finally, have your main and have it call onModuleLoad:

if __name__ == "__main__":
    app = RemoteController()
    app.onModuleLoad()

I have myTextBox an id just like Ajax would. I did this for Selenium. My Selenium test could now find the widget By.Id or by xpath using the id:
    text = selenium.get_text("//*[@id='my_text_box']")

  For Selenium you usually want to be able to get text out of text fields but the PyjamasTextBox and TextArea widgets don't actually put the text into the HTML they render. Instead, I used an HTML() widget. It's like a Label and DOES embed the text in the web page so you can get to it from Selenium.

Some other stuff:

CSS - It's nice to set the background colors of your various widgets, and you can do that using inline CSS  in the Python code like this:
   from pyjamas import DOM
   DOM.setStyleAttribute(myWidget.getElement(), "background-color", "#8fbc8f")

 If you have multiple concurrent users, you want your web app to be session based. This is done, in Django, by adding a few things to your settings.py. I also had some issues with CSRF so I needed to disable that (you may not want to, depending on your security concerns. I had none in my test environment)

   MIDDLEWARE_CLASSES = (
    'regression.middleware.DisableCSRF',            
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.middleware.csrf.CsrfResponseMiddleware',
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',
)

 CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.filebased.FileBasedCache',
        'LOCATION': 'c:/django_cache',
    }
}

CACHE_MIDDLEWARE_ALIAS = "default"
CACHE_MIDDLEWARE_SECONDS = 600
CACHE_MIDDLEWARE_KEY_PREFIX = ""