Creating Powerful Remote Test Fixtures Using RPyC & the Squish GUI Tester

RPyC is a powerful Python module for distributed computing. It's especially useful in conjunction with Squish: by integrating RPyC into Python-based Squish GUI tests, creating complex test fixtures on remote system becomes a breeze!

Setting The Stage

Repeatable tests are a key feature of automated test projects. By being able to repeat a test at will, it's easy to demonstrate defects to other team members and to validate proposed bug fixes. To be able to repeat test runs requires that the test setup itself remains unchanged - except of course the application under test (AUT). This means that the only 'moving' part is the AUT itself, except of course the AUT build.

Test fixtures are used to achieve a high degree of control over the environment in which a test is executed. Test fixtures initialize the application to a well-defined state by configuring the environment in which the AUT operates. A fixture typically covers aspects such as environment variables, the file system and the set of processes which are running.

However, different test cases might want to execute the application in different scenarios. One test case might exercise the application in the way it starts the first time the user launches it. Another test case might be defined based on the premise that there is a set of recently opened documents, and so on. Thus, it's critical that test fixtures are flexible - they need to be configurable such that they can assume whatever initial state a test case demands.

Doing this for a single local computer is challenging enough, but simulating a specific state for a remote system is even more complex. The Squish GUI Tester supports running applications on many different platforms and operating systems, and typically the application under test is running on a different computer than the Squish IDE. This is possible thanks to the client-server architecture of Squish.

It's this distributed architecture of Squish which makes e.g. running automated GUI tests for applications running on embedded devices a breeze: the Squish IDE as well as any test script data (such as expected screenshot data) resides on your local workstation while the application under test is executed on the actual device. The network transparency allows working with remotely executed applications as if they were running locally. However, test fixtures remain a challenge. How can we implement flexible test fixtures which apply arbitrary changes to a remote system such as an emedded device?

Enter RPyC

RPyC (pronounced as are-pie-see), is a library for transparent remote procedure calls: executing arbitrary Python code on a remote machine as if it were running on locally machine. This means that the full power of Python scripts is available to configure any aspect of the remote system. RPyC, then - as the website puts it - "Excels in testing environments ". The website on good RPyC use cases) explains:

The first and foremost use case of RPyC is in testing environments, where the concept of the library was conceived (initially as pyinvoke).

Classic-mode RPyC is the ideal tool for centralized testing across multiple machines and platforms: control your heterogeneous testing environment (simulators, devices and other test equipment) and test procedure from the comfort of your workstation. Since RPyC integrates so well with python, it is very easy to have your test logic run on machine A, while the side-effects happen on machine B.

There are other solutions which provide this functionality, but RPyC offers a few key benefits:

• Cross-platform: just like Squish, the local and the remote system can be running different CPU architectures or operating systems.
• Low overhead: no complex setup required (name servers etc.)
• Self-contained: no installation via 'pip' required, the RPyC module can be shipped alongside the test suite (as a shared module.)
• Very few dependencies: the remote side needs a Python interpreter and, for a zero-deploy installation, an SSH server.

Server-Side Installation Of RPyC

RPyC supports two kinds of servers exposing functionality of the system. The 'classic' servers permit doing everything you can think of whereas the new-style, 'service-based' servers permit finer-grained control. For our case, a classic server will do just fine and requires no programming.

Unless you pursue a zero-deploy installation (which is beyond the scope of this article), you will need to launch a very simple RPyC server on the remote side. This is where your application under test is running. Launching the server is a simple three-step process, assuming that you have a Python interpreter installed:

1. Create a Python virtual environment, acting as a sandbox environment, to avoid modifying the system under test beyond a single directory.
2. Installing the rpyc module onto the virtual environment.
3. Launching the rpyc_classic.py script to run the server -- this script is part of the rpyc module.

Here's a sample session:

$ python3 -m venv .

$ bin/pip install rpyc

Collecting rpyc

  Downloading rpyc-5.0.1-py3-none-any.whl (68 kB)

     |████████████████████████████████| 68 kB 1.8 MB/s

Collecting plumbum

  Downloading plumbum-1.7.0-py2.py3-none-any.whl (116 kB)

     |████████████████████████████████| 116 kB 4.2 MB/s

Installing collected packages: plumbum, rpyc

Successfully installed plumbum-1.7.0 rpyc-5.0.1

$ bin/rpyc_classic.py

INFO:SLAVE/18812:server started on [127.0.0.1]:18812

Adding RPyC To A Squish Test Suite

It's possible to add the RPyC module to the Python interpreter used by a Squish installation (we discussed this approach in an earlier article), but since RPyC has so few dependencies, an even easier approach is just to ship the module along with the test suite. This avoids any customizations to the Squish installation - the test suite is self-contained. Here's how to do it:

1. First, download an archive with the source code of the most recent RPyC release.
2. Next, decompress the archive in a temporary location.
3. Copy the rpyc directory in the archive into the shared/scripts directory of your Squish test suite, leaving you with the directory shared/scripts/rpyc.

To make sure that it's all working, a quick sanity check is useful. Make sure that the RPyC server is running and then create a new Squish test case with the following contents:

import rpyc

def main():
    conn = rpyc.classic.connect("localhost") # use default TCP port (18812)
    print("Hello, World!", file=conn.modules.sys.stdout)

What this does is to connect to a RPyC classic server running on the local machine (localhost) and then print the text 'Hello, World!' to the standard output on the remote computer, a handle to which is provided via conn.modules.sys.stdout. Running this script should cause output to be executed by the rpyc_classic.py script, similar to:

INFO:SLAVE/18812:accepted ('127.0.0.1', 55788) with fd 4

INFO:SLAVE/18812:welcome ('127.0.0.1', 55788)

Hello, World!

INFO:SLAVE/18812:goodbye ('127.0.0.1', 55788)

RPyC in Use: Setting Up A Workspace

At this point, you're ready to take advantage of RPyC. Here are a few examples to get you started:

Creating Directory Structures

Creating directory structures is a common task as part of setting up a text fixture, e.g. for setting up a sample workspace. The Python function os.makedirs is great for this, and of course it's readily available via RPyC:

import rpyc

conn = rpyc.classic.connect(...)

conn.modules.os.makedirs("/tmp/Projects/Sample Project1/Images")

Note how instead of os.makedirs, we're using conn.modules.os.makedirs. The modules field of the connection object returned by rpyc.classic.connect provides access to any modules. There's no need to import them up front, they are automatically loaded as needed.

Writing Files

You can not only call plain functions such as os.makedirs, RPyC also allows using objects such as the ones returned by open. This is very useful for accessing (i.e. creating or modifying) files read by the application under test.

Here's an example creating a simple configuration file on the remote side and then writing three lines of text into it:

import rpyc

conn = rpyc.classic.connect(...)

with conn.builtins.open('/tmp/Projects/config.ini', 'w') as f: f.write('TEMPLATE=plain\n') f.write('CREATOR=Frerich Raabe\n') f.write('VERSION=2\n')

To access built-in functions such as open, the builtins field offered by connection objects is useful: here, we used it via conn.builtins.open to call the open function on the remote side. The returned file object can be used as usual - all methods will automatically get executed on the remote side.

Running Programs

Of course, you can also run arbitrary programs on the remote system. The subprocess module is the standard way of doing this, and it, too, is readily available via RPyC. Here's a quick example:

import rpyc

conn = rpyc.classic.connect(...)

proc = conn.modules.subprocess.Popen("ls", stdout = -1, stderr = -1)

stdout, stderr = proc.communicate()

test.log(stdout.split())

What this does is to connect to a remote computer and then run ls on the remote side. Finally, the output of the ls command is logged to the Squish test report.

Where To Go From Here

Of course, we're just scratching the surface here. It's not hard to imagine that by having the full power of Python at hand, executing arbitrary code on the remote side, our test fixtures can be arbitrarily complex and extremely sophisticated.

In fact, RPyC is not just useful for creating test fixtures. It's also very useful during test execution. For example, you can upload (or download) files for the application under test to process. Or you could send operating system signals to the application to signal e.g. error conditions.

The RPyC documentation explains the features of RPyC in much greater detail. In particular, the HowTo's provide a number of thought-provoking examples of what to do with the module.

How did you use RPyC in your test project? Let us know in the comments below!