My raison d’être in the workplace is ensuring the continuity of our systems and functionality, come what may: crazy requirements, infrastructure failures, and general oddness. If I do my job right, everyone (from the CEO down) gets to sleep well at night, go on vacations with their family, and spend their weekends on chores.

One hopes, anyways.

Test automation is an critical tool in the quality engineering toolbox. I like using Python for my end-to-end and complex integration tests — though of course many companies have great success with Java, Ruby, or even Javascript.

Below are several Python packages (grouped by category) I’ve found useful, along with any tips and tricks I’ve picked up over the years.

(This isn’t by any means an exhaustive list. I’ll be periodically editing and expanding this section as I find new tools to play with!)

Test Frameworks

pytest — This fully-featured testing tool offers an excellent foundation for building any kind of testing framework. It’s well-maintained and integrates with PyCharm and Jenkins (in particular wrt test statistics and reporting).

One of the big benefits of pytest is it’s rich ecosystem. Some packages I’d recommend are:

  • pytest-cov — Test automation should be treated like any other application for coding standards, code coverage, etc. pytest-cov lets you keep an eye on your modular packages!
  • pytest-capturelog — Offers a quick and easy way to set up logging for your framework.
  • pytest-html — Lets you create human-readable summaries of your test automation. It hooks up well to the Jenkins HTML Reports plugin — but remember to use a self-contained report!
  • pytest-selenium — A pretty new package that lets you quickly and scale-ably “hook up” your Selenium automation to Selenium Grid, Saucelabs, Browserstack, and many other Selenium SAAS companies. It also adds very helpful information to the HTML report (such as links to Saucelabs videos or screenshots on exceptions).

robotframework — This framework is based on the BDD methodology (though it certainly be used for other approaches). It’s very flexible, allowing coders to mix functionality from libraries with custom code. Reports are easy to read and the Jenkins Robot Framework plugin makes monitoring an ease.

Robot Framework is based on a Domain Specific Language. This can be a pro (easy to read, self-documenting test cases) or a con (difficult to scale, keyword collisions).

Testing Your System

selenium — ‘nuff said. It’s new releases are issue prone, it’s browser support is hit-or mix, and it’s one of the single most important tools in the test automation toolbox!

  • I’d strongly implementing modularizing your Selenium code (i.e. by separating out locators from Page Objects).

requests — Bread-and-butter for API testing. Well-written API automation also sets you up well for custom performance testing framework.

requests-oauthlib — This is critical if working with an API with OAUTH2.

paramiko — Got a job to test? Got dozens of jobs in a complex, inter-dependent schedule to test? You can use paramiko to automate job execution on remote servers, as well as sending/retrieving data.

  • If you can’t SSH to a server using username/password (common in more secure industries like finance), you can use local private keys as well (with appropriate user provisioning).
  • If you’re company’s policy requires passphrases for your SSH key, you’ll need to explicitly add support!

Data Sources

psycopg2 — This package lets you automate a whole boatload of database-related QA tasks like data validation and data setup.

  • Building the psycopg2 package locally can be tricky. On Ubuntu, you’ll have to install the libpq-dev and python-dev packages. On OSX, you’ll…OK just don’t. Use a pre-compiled package from Anaconda (more on that later!).
  • If you’re performing data validation, dictionary-like cursors are highly recommended! result_set[“ZipCode”] is so much easier to read and maintain than result_set[8].

pymongo — Absolutely awesome for working with Mongo. I’ve found a quick python script is far more effective than MongoDB’s own built-in command line client.

simple-salesforce — simple-salesforce provides an easy-to-use client to Salesforce’s own REST API. It’s easy to use, effective, and most importantly kept up-to-date.

beatbox, it’s main competition, is built on top of Salesforce’s SOAP API. It’s also not well supported, with the last release in 2014.

Utilities

jsonschema — I prefer using JSON files for provisioning, data, etc. JSON schema lets you maintain consistency when multiple people are modifying, editing, and doing other wonky things with your config files.

  • This site is the best online schema generator by miles.

pylint — With caveats about pointless consistency, it’s a great tool for pointing out silly things (i.e. unused imports) in your code. Results can be integrated with the Jenkins Violations plugin.

Anaconda — Aside from it’s (significant) usefulness to data scientists, anaconda offers an easy-to-use encapsulation of virtualenvs as well as access to pre-compiled binaries (like psycopg2).

  • If working on Jenkins or any other CI, I’d suggest looking into miniconda!