Previously, on Avatar…
In prior introductory posts (part 1, part 2), we covered why E-gAT is a toolkit and not a framework, dropped some philosophy around leveraging the power of an organization’s existing workforce, touted the Page Object pattern for writing automated tests, and mentioned that E-gAT is freely available on GitHub.
In this post, we’d like to cover some of the technical aspects of E-gAT.
Test Runner: Multi-threaded and Resource-smart
Two technical aspects that we wanted to address in a better test runner solution were parallel execution and resource smartness. Testing large, complex web applications is time consuming. And that’s if you only ever intend to support one browser, which is pretty rare in our experience. For each browser your application needs to support, the time it takes to validate and test the applications grows, too (usually linearly). Compound this problem with organization-wide contention over access to Dev/Test server environments and the time it takes to test/validate is a real issue in organizations – large or small.
In order to reduce the overall time for running tests, we set out to create a toolkit that allowed the test lead / architect to leverage parallel execution, while also being smart enough to allow the configuration of serial/sequential tests. Multithreading comes in two different modes: Auto Threading and User Defined Threading. Auto Threading is the default mode and lets the test runner figure out how to run everything most efficiently. When you need a bit more control, User Defined Threading allows the user to specify which tests will run in which threads in the configuration file.
And we needed to provide support for what we call “Resource Groups.” Resource groups are our way of supporting requirements and conditions where multiple tests should not access resources (such as an Admin login) at the same time. Our toolkit needed to be “resource smart.”
E-gAT test runner also provides a fast-failure mechanism called “Execution Groups”. Using simple and powerful annotations, decorate test classes or even individual test methods in different classes/modules with the same execution group. If one of the test methods in the execution group fails, the remaining test methods will be skipped. This can save a lot of time and processing power by preventing the run from needless further testing.
Failure Screenshots and DOM Capture
The test runner has special support for tests using the Selenium API. In INFO mode, any tests that fail in the TestSet will have a screenshot of the browser taken and the DOM captured. In DEBUG mode, the test runner will take a screenshot and save the DOM after every test. By default, the test runner runs with the log level set to ERROR, which disables Selenium debugging.
Run Via Command-line or Hook Into IDE
E-gAT tests can be developed in any standard text editor and executed via the command-line. Optionally, some may prefer to ease the test-writing process by hooking E-gAT into an IDE. This allows you to give your tests a quick trial execution while stepping through line-by-line in a debugger and also provides intellisense.
Highly Customizable, Concise JSON Config File
The test runner supports a very clean JSON configuration file that has the same flags as the command-line client but gives more flexibility over test execution. Various flags, custom threading, and configuration parameters can be defined. The nesting, inheritance hierarchy allows for as much or as little granularity as desired. Custom Configuration and Environment variables are parsed by Python’s JSON library and added as “Configuration” and “Environments” dictionaries to every TestSet the test runner runs.
Pluggable, Configurable Results Logger
We wanted to be able to display test results in a way that would be easy to analyze and report on. So we built in a pluggable, configurable results logger that outputs in HTML, and allows us to “skin” it for each client with simple CSS. But anyone can write their own logger to create whatever kind of log, output, or report fits the organization they’re serving.
Multiple Browsers and BrowserStack
Of course any modern automated testing toolkit needs to support multiple browsers. And we’re able to do that very easily by leveraging Selenium’s core abilities. By leveraging Selenium’s WebDriver, E-gAT supports every browser that Selenium supports (which includes, but it not limited to: Firefox, Chrome, IE, Opera, and PhantomJS).
We are also very excited about E-gAT’s integration with BrowserStack. BrowserStack allows E-gAT to run tests in pretty much any OS and Browser combination out there. Lets say you are targeting IE9, IE10 and the latest version of Firefox. With a simple configuration E-gAT will call out to BrowserStack and get a WebDriver for the specific OS and Browser combination you request. This allows you the flexibility of running automated tests on browsers that are not installed on the local machine as well as testing out different OS combinations.
A Sample, Reference Implementation
To cap it all off, we wanted to build out a sample project that would show off E-gAT’s technical abilities, as well as manifest some our philosophy that drove its creation. So be sure to check out the E-gAT Example Project on GitHub as well. All of the features mentioned above (and more!) are covered in either this example project, the example project’s accompanying wiki, or the E-gAT wiki page.
