.webp)
IC Manage is a company that provides next generation design management solutions for hardware and software design, enabling companies to efficiently and reliably manage single and multi-site development efforts. They have been using Squish since January 2006.
We spoke to Gary Gendel their Chief Software Architect, and asked him about the products they test with Squish, and their experience of using Squish.
What IC Manage Test with Squish
Squish is currently used to test three of IC Manage’s major products:
- icmpm—a tool that allows managers to control the data for a design in managed workspaces.
- icmg—a GUI client that replaces the Perforce GUI and which is enhanced to handle IC Manage workspaces.
- icmanage—a library manager program that is integrated into the Cadence design environment; this is shown in the screenshot.
These applications are tested and deployed on Linux, Unix, and Windows, with other platforms in the pipeline. In addition, IC Manage are already planning to use Squish to test a range of other programs including defect trackers and various data synchronization products. But why did they choose Squish and why are they using it with more and more products?
Why IC Manage are using Squish
IC Manage first learned about Squish through informal networking and through Internet searching. Gary told us that at the time:
There really wasn’t any competition for a Qt testing tool that could meet our requirements.
In view of this, their initial choice was easy to make. It’s now been more than three years since that decision was made, yet Gary tells us:
We haven’t had a reason to look elsewhere, since Squish has exceeded our expectations.
We asked Gary if Squish provided all the functionality they needed, and he pointed out that although there had been some missing features early on:
Every important enhancement we’ve requested over the years has been implemented in a timely fashion.
Gary also pointed out that one feature that has long been provided by Squish is the ability to attach to a running process that can then be tested—and that this functionality was essential to IC Manage, since it is the only means by which some of their products can be properly tested.
Like most technically-oriented commercial products, Squish’s technical support is something that many customers rely on. We asked Gary what his experience had been, and he told us:
We have been more than satisfied with the technical support we’ve received from froglogic. The support team has always responded in a timely fashion to provide workarounds or fixes to any issues we’ve had.
Gary went further and added:
In addition, froglogic’s support team have often provided insightful help to our “What’s the best way to do …?” questions.
Challenges Faced and Met
While IC Manage’s use of Squish has been very successful it has not been without difficulties and problems along the way. One of the biggest challenges faced by IC Manage’s testers was to validate multiple sets of MySQL and Perforce data contents that were changed by a single program. Another challenge was to produce tests that could work where there were multiple applications to control and validate during the execution of a single test. And like most other testers, IC Manage’s testers have the problem of ensuring that the test coverage keeps up to date with rapidly evolving products. IC Manage devised ingenious solutions to solve all of these problems.
Initially, IC Manage began by creating data-driven tests where the data from manual sessions was stored in MySQL tables. The data was then used in Squish scripts to programmatically control the application being tested, using the application’s GUI to recreate application objects, and then saving the results back into a MySQL table. This proved unsatisfactory in practice, first, because the production of the tests did not keep pace with the changes in the applications being tested, and second, because the error reporting was too verbose to be useful.
IC Manage overcame these problems by developing a custom validation library that is used in conjunction with Squish’s recording and playback facilities to generate high quality tests that can then be incorporated into the automated testing processes. In addition IC Manage developed “smart” comparison routines that understand MySQL and Perforce data sets to produce accurate comparisons that are independent of platform, architecture, and other environment details. IC Manage also developed a custom error reporting library—used for both GUI and non-GUI testing—that produces much more precise error reports than before, and which are combined with Squish’s error reports to produce useful and concise test reports.
In fact, IC Manage have integrated Squish very successfully into the workflow of their development and testing process. As Gary tells us:
We can launch specific build and test jobs on multiple platforms driven by appropriate changes in the source repository using our “Sync Engine”, a product we sell commercially.
Over thirty test suites are in use, but these are all scheduled to be replaced by a new generation of tests that utilize the libraries and infrastructure IC Manage have built around Squish. The new test suites will also unify both GUI and non-GUI testing (since these are currently done separately), and fully implement automated test and report generation. IC Manage have already experienced dramatic productivity improvements thanks to their new infrastructure, with significant test suites taking from a day to a week to develop, compared with a week to a month previously.
Conclusion
IC Manage have fully integrated Squish into their development and testing process. In conjunction with some in-house tools and scripts that they developed for the purpose, they are able to generate tests based on validation criteria and to run those tests and produce precise custom reports detailing the results.
Clearly, IC Manage take testing seriously, and invest considerable time and effort into making testing a part of their development workflow. We asked Gary how he justifies such a commitment; he responded:
Improved product quality is the biggest benefit. Once test suites are completed, they are added to our automated regression suite. Also, test engineers are able to work on testing new product features concurrently with product development.
froglogic’s team would like to thank Gary for taking the time to share IC Manage’s experience with Squish, and we looking forward to a continued successful relationship.
