Synchronized batch execution of Squish

Squish test execution involves multiple processes which may even run on
different machines. Ensuring that all parts are up and running before actual
test execution can be challenging sometimes. The following shows a little trick
how automated setups can wait for a local or remote squishserver to become
available to avoid execution failures if the squishserver host is slow to
respond.

Automated setups often follow the same sequence of steps:

• start squishserver as a background process (either local or remotely)
• execute tests using squishrunner
• stop squishserver

Depending on the timing however, squishrunner execution may fail if
squishserver startup takes a bit longer and does not accept squishrunner
connections in time. There is of course an easy workaround by always waiting a
fixed amount of time before starting squishrunner but even that may either be
too long (which increases overall test execution time) or too short (which may
still cause sporadic failures).

Waiting for squishserver

An alternative to fixed delays is to find out if squishserver is actually reachable by letting squishrunner request something from squishserver and check the result. One simple request can be querying for some configuration setting. On commandline this can be achieved by calling squishrunner --config ... or squishrunner --info ... followed by a check of its exit code.

The following Unix shell example shows, how squishrunner reacts if squishserver is not reachable:
$ squishrunner --info responseTimeout
squishrunner: Error while retrieving information: responseTimeout
Could not connect to Squish server at host 127.0.0.1,
port 4322: Connection refused (possible reasons:
'squishserver' process not running or listening on a
different port, or firewall issue)
$ echo $
255

Combining this with a loop that tries to fetch this information can then be used to wait for as long as needed, essentially synchronizing on squishserver availability:

for f in $(seq 1 10)
do
    squishrunner --info responseTimeout 2>/dev/null >/dev/null
    if [ $? -eq 0 ]
    then
        exit 0
    fi
    sleep 0.2 # wait a bit before retry
done
exit 1 # waited too long

The same approach can also be used to synchronize on squishserver shutdown, only the exit code to wait for differs.
squishrunner will return 255 on Unix and -1 on Windows in case squishserver is not reachable (anymore).

Putting everything together

Now we can combine the above approach with actual squishserver start and stop as well as executing one or more testsuites.
For Unix systems (Linux, macOS) something like the following can be used as a starting point:

#!/bin/sh

wait_for_exit_code() { EXPECTED_EXIT_CODE=$1
    shift
    for f in $(seq 1 10)
    do
        $@ 2>/dev/null >/dev/null
        if [ $? -eq $EXPECTED_EXIT_CODE ]
        then
            return 0
        fi
        sleep 0.3 # wait a bit before retry
    done
    return 1
}

# Installation prefix of Squish SQUISH=.

# Start squishserver in background
$SQUISH/bin/squishserver --daemon >/dev/null

# Wait for server to become available
wait_for_exit_code 0 $SQUISH/bin/squishrunner --info responseTimeout
if [ $? -eq 1 ]
then
    exit 1
fi

# Run your tests here
# $SQUISH/bin/squishrunner --testsuite "/path/to/suite_test" ...

# Send stop request to squishserver
./bin/squishserver --stop >/dev/null

# Wait for server to become unavailable
wait_for_exit_code 255 $SQUISH/bin/squishrunner --info responseTimeout
if [ $? -eq 1 ]
then
    exit 1
fi

# Finish

The same approach can also be taken on Windows using the command line batch
interpreter:

@REM OFF
SETLOCAL

REM Installation prefix of Squish
SET SQUISH=.

REM Start squishserver in background
START /B %SQUISH%\bin\squishserver >NUL 2>NUL

REM Wait for server to become available
CALL :wait_for_exit_code 0 "%SQUISH%\bin\squishrunner.exe" --info responseTimeout
IF ERRORLEVEL 1 EXIT /B 1

REM Run your tests here

REM Send stop request to server
"%SQUISH%\bin\squishserver" --stop >NUL

REM Wait for server to become unavailable
CALL :wait_for_exit_code -1 "%SQUISH%\bin\squishrunner.exe" --info responseTimeout
IF ERRORLEVEL 1 EXIT /B 1

REM Finish
EXIT /B 0

:wait_for_exit_code
FOR /L %%A IN (1,1,10) DO (
   %2 %3 %4 2>NUL >NUL
   IF ERRORLEVEL %1 EXIT /B 0
   REM ping 127.0.0.1 -n 1 >NUL
)
EXIT /B 1

Finishing touches

There are several ways to improve upon this approach. For even slower systems or for remote servers that were just rebooted as part of a test you may want to wait even longer. This can be achieved by increasing the number of iterations in the for-loop inside the script, currently both scripts retry up to 10 times. Another way to increase the overall waiting time is to increase the delay between connection attempts. In case of the Unix script this can be done by increasing the value passed to the sleep command. For the Windows batch script uncommenting the ping command should add a similar delay, although only in a rather coarse granularity of seconds.