Model-Based Testing in Practice with the Squish IDE

In a follow-up to our earlier introduction to model-based testing (MBT) this article delves deeper into the principles and best practices of MBT and demonstrates its practical usage in the upcoming 8.0 release of Squish GUI Tester.

Model-based testing in a nutshell

Model-based testing is a systemic approach to designing and executing software tests that utilizes models as abstractions of a system under test (SUT). Unlike traditional testing, which often involves scrutinizing system’s intricate details, model-based testing takes a more general approach, where an abstract, partial model aims to represent the desired behaviour of SUT and guides the testing process.

A model of SUT can be a state chart with boxes and arrows, representing states and state transitions (other diagrams that may be used to render the abstraction of the system include: activity diagram, BPNM, UML diagram etc.). The principal idea behind the model-based approach to testing is to use such system abstraction to create or generate test cases.

Choosing the right level of abstraction

Models used both for software development and testing involve various levels of abstraction. Choosing the right level for our model is essential, as it allows us to select only meaningful test cases that cover exactly what we need and helps us to focus the testing effort.

 At the highest level of abstraction. case diagrams or behaviour models provide a broad overview of system behaviour. When capturing more specific aspect of the system is desired, a good choice of model can be a state diagram depicting states and state transitions, or sequence diagram representing interactions. 
At the lowest level of abstraction, there are highly specific models where individual states or interactions are broken down into more details, as well as complex multi-level models where an individual components can be a composite of two or more atomic sub-components.

Model-based testing editors in the Squish IDE

The upcoming release of Squish GUI Tester 8.0 makes its first strides into providing a comprehensive set of tools for model-based testing in the Squish IDE and delivers an easy-to-use graphical MBT model editor and MBT test case (path) editor.

MBT models and MBT test cases are fully integrated into existing test projects: they can co-exist in test suites alongside other resources and scripted or behaviour-driven (BDD) test cases, while MBT step implementations can make use of shared test scripts. Step skeleton generation, code completion and script API for all supported scripting languages are provided as well.

In our practical demonstration of model-based testing, SUT model is a simplified activity diagram, defining what actions/steps (boxes) can be carried out on the system in what order. The connections between the steps (arrows) outline which steps have to precede and which may follow any given step. Any path through the diagram is considered a possible test scenario.

First model-based test case

Our application under test (AUT) would normally run on a screen of a coffee machine and let the user choose a caffeinated beverage, adjust the amount of sugar and milk and finish the brewing process. Welcome screen of the AUT alongside a model representing its behaviour may look as follows:

We begin our demo with two simple test cases, represented by one path through the activity diagram each. In the first test case we simply brew an espresso with no extras.

In the second test case, the coffee machine brews a latte, while we adjust the amount of milk and sugar.

On executing the test case, Squish GUI Tester takes control of the AUT and replays given steps in the order specified by the path through the activity diagram. Each step is associated with so-called implementation: a function written in a scripting language (Python in our case) that is executed when the test case reaches this particular step. Such a function may automate user interactions with the AUT or verify properties of application objects. An example implementation of several steps of our example model is listed in the following snippet:

@mbt.step("Click on Macchiato")
def step():
    mouseClick(waitForObject(names.macchiato_button))
    test.vp("VP_macchiato")

# -------- Setting milk and sugar --------

@mbt.step("Set milk")
def step():
    mousePress(waitForObject(names.milk_slider_handle))
    slider = waitForObject(names.coffee_sugarSlider_Slider)
    x = builtins.int(slider.width * random.random())
    y = builtins.int(slider.height * 0.5)
           mouseMove(slider, x, y)
    mouseRelease()
    snooze(0.5)

Adding a step to MBT test case

One functionality is however missing from our model and the two simple test cases we covered above: going back from “Brew me a cup” screen and selecting a different beverage. To be able to test this scenario, we use MBT model editor in the Squish IDE to extend our model as follows:

1. In Add Step mode, we introduce a new step representing ‘Go back’ operation

2. In Connect Steps mode, we connect it to the remaining steps in the diagram
   

3. With the help of Generate missing steps implementations function, we generate an empty template implementation behind the new step

4. Finally, we replace the empty implementation with the real function performing a tap or click on ‘Go back’ button:

   @mbt.step("Go back")
   def step():
       mouseClick(waitForObject(names.back_button))

    

Let’s now create a new model-based test case that will exercise the functionality of going back to change the selection and brewing a different beverage. The workflow is represented by a path through the activity diagram (the model of our SUT) that contains the step we just added to the model:

We execute our MBT test case in the same way we would execute any other test case (scripted or behaviour-driven) in the Squish IDE. The list of steps in the right sidebar of MBT test case editor, as well as test results view, will indicate by colour and/or icon whether the given step has passed or failed.

Conclusion

Model-based testing offers a strategic approach to automated testing. It facilitates early defect detection and fosters better communication within the development teams. To test engineers who seek systematic and organised testing methods, Squish GUI Tester 8.0 offers a new set of powerful tools to incorporate model-based testing into their software development life-cycle.

Next steps

Interested in exploring model-based testing? Here are some ideas what to do next:

• Watch our accompanying video with a live MBT demo in the Squish IDE
• Request a free trial of Squish GUI Tester 8.0 to try out MBT tools. Existing customers and evaluators can download the newest Squish release from the  Qt Customer Portal after logging in with their Qt account
• Follow through Squish MBT tutorial to learn about MBT and related tools shipped with Squish in more detail
• Get inspired by example MBT test suites included in Squish 8.0 binary packages

Note: MBT support in Squish GUI Tester is a technical preview (see Squish knowledge base for the list of known limitations). In order to improve and provide the best quality tools in the future, we encourage all MBT-curious users and testers to submit their feedback.