Sending E-Mail From Squish Tests

Automated testing using the Squish GUI Tester is a great thing: all the mundane work of exercising and verifying the GUI is done automatically. Humans can concentrate on more creative and abstract tasks, such as test design or root cause analysis based on test reports, using e.g. Squish Test Center. There's often a dedicated team for the latter task, which means that report analysis is typically done by the same people, test engineers. Faults are found, regressions are noticed -- tickets get created.

As it happens, not all faults are equal: GUI test automation setups are complex systems, and some causes of test failures can be outside of the usual GUI verifications. One example of this is problems with the underlying hardware on which the tests are executed. In such cases, it can be very useful to involve other people in the discussion. People, who are not directly part of the test engineering team.

Consider for example a known weakness in the test setup which causes a network connection to a remote device to go down every now and then. It might not happen often, but when it does it breaks the entire test run. Also, it might not be immediately clear whom to involve for fixing the issue: hardware issues like this might be the domain of a DevOps engineer or a System Administrator. In such cases, it can be extremely helpful to encode the responsible person right in the test script such that if the problem occurs, the right colleague gets notified right away -- preferably with as much context as possible, to help them with resolving the issue quickly.

E-mails can be a great way to go about notifying colleagues in other teams about abnormal test results. Including verbose error descriptions in the mail body and attaching artifacts such as screenshots and log files greatly simplifies diagnosing the cause of the issue. Plus -- everyone already has an email client. No separate messenger platform needed!

Prerequisites

The general principlies apply to all scripting languages supported by Squish, but for the sake of the demonstration we'll assume that you're using a Python-based test suite.

Also, you need access to an SMTP server which accepts your mail and then relays it to the recipient. Chances are that you already do have access to one: it's required for sending any email! If in doubt, check back with your System Administrator.

Sending E-mail Using Python

Sending e-mail messages using Python scripts is a two-step process, each of which is based on a dedicated module which is part of the standard library:

1. First, a message is composed using the email module. This module modes e-mail messages as discrete objects (using the class EmailMessage), including any components of the message such as a subject, some recipients, a body -- and potentially some attachments.
2. Next, the composed message is sent to a SMTP server via the smtplib module. A simple send_message function takes care of all the grunt work and makes sure that your message gets relayed.

Let's put these pieces together, defining a simple function which, given a subject, a recipient and a body, sends out an e-mail:

import email, smtplib
def sendMail(subject, to, body):
    msg = email.message.EmailMessage()
    msg.set_content(body)
    msg['Subject'] = subject
    msg['From'] = 'Your Friendly Neighborhood GUI Tester <noreply@example.org>'
    msg['To'] = to

    server = smtplib.SMTP('mail.example.org')
    server.send_message(msg)

Of course, you most certainly want to adjust the host name of the SMTP server to use.

Sending an e-mail in response to critical errors is then a matter of a single function call. Here's an example which sends an e-mail out in case a startApplication call fails unexpectedly - we'll use textwrap.dedent to avoid an unintended whitespace in the e-mail body:

try:
    startApplication("my_application")
except RuntimeError as err:
    sendMail( subject="Error while executing " + testSettings.testCaseName, to="Frerich Raabe <raabe@example.org>", body=textwrap.dedent(f"""\
Hi!
Unfortunately, starting the application failed once again. :-/
The exception text is: {err}
Can you please have a look? Thanks!
- Your friendly GUI Tester
""") 

Nice!

Screenshots For Extra Context

We're off to a great start, but our e-mails can be made much more useful by attaching additional information.

Screenshots are often very helpful when understanding an issue. The sendMail function could be extended such that a screenshot of the entire desktop is automatically attached to the e-mail. The grabDesktopScreenshot function comes in handy here. Here's a revised definition of sendMail:

import email, smtplib, tempfile
def sendMail(subject, to, body):
    msg = email.message.EmailMessage()
    msg.set_content(body)
    msg['Subject'] = subject
    msg['From'] = 'Your Friendly Neighborhood GUI Tester <noreply@example.org>'
    msg['To'] = to

    tmpFileName = tempfile.mktemp(suffix='.png')
    try:
        img = grabDesktopScreenshot()
        img.save(tmpFileName)
        with open(tmpFileName, 'rb') as f:
            msg.add_attachment(f.read(), maintype='image', subtype='png')
    finally:
        os.remove(tmpFileName)

    server = smtplib.SMTP('mail.example.org')
    server.send_message(msg)

Notice the new paragraph in the middle of the function definition: we use tempfile.mktemp to calculate a temporary file name -- this isn't usually considered a best practice, but it's required here since the save method exposed on images returned by grabDesktopScreenshot expects a file name. The function saves a screenshot to a file, attaches the file contents to the e-mail, and then removes the temporary file again.

Even Better E-mail

Screenshots are a great start, but there are many other things which you might want to consider for inclusion in e-mails. Here are some ideas:

• The output of the application under test, as returned by the ApplicationContext.readStdout function
• Any log files which the application under test might have written out
• A stack trace, documenting the exact location in the test script code at which the error occurred -- see test.stackTrace.

Remember, the recipient of the e-mail is possibly not intimately familiar with the testing setup. Anything which can provide additional context to get a better understanding of what's going on could be useful!

Do you have common pieces of information you like to include in e-mail reports? Feel free to let us know in the comment section below!