.webp)
Founded in 1994 and headquartered in Quebec City, Canada, with subsidiaries worldwide, InnovMetric Software Inc. is the leading provider of universal 3D metrology software solutions. The world’s largest industrial manufacturing organizations trust InnovMetric’s PolyWorks® software solutions to maximize the benefits of 3D measurement technologies for their engineering and manufacturing applications.
One key element of PolyWorks is its powerful macro-programming language that allows customers to develop and deploy customized workflows and automated processes. This language is also heavily used internally to automate the testing process while developing new functionalities or refactoring software components.
InnovMetric needed to understand how much code was covered by tests
InnovMetric has deeply invested in functional testing and continuous integration since 2003. The company writes about 1,000 tests and adds them to a framework every year. There are currently over 15,000 test scripts, and to run them all takes over 14 hours.
Running each of the tests daily helps InnovMetric notice if there are any regressions, but until recently, it was impossible to know precisely how much code was covered by tests. They had a general idea, but without the right tools, it was impossible to say with certainty.
InnovMetric decided that adding a code coverage tool would help them measure things more precisely. After seriously evaluating different tools, InnovMetric ended up deciding on the Coco Code Coverage Analysis tool (known as Coco from the froglogic acquisition) for their daily workflow.
“Using Coco, we can observe that an amount of work we do brings the coverage value up by a certain amount, and that is what is most important to us. It helps us see how good our tests are.” -InnovMetric
Coco measures code coverage and saves developers’ time
Coco immediately showed InnovMetric which parts of the software were not yet covered by tests. However, there was another problem to solve: When a developer changed something in the code base and wanted to run the tests, they had to wait until the next morning before those changes could be pushed into version control. This took too long and interrupted developers’ workflow to run all of the tests before each possible change.
InnovMetric ended up using Coco to find out which tests were using a specific part of code so that developers would run only the relevant tests, and get faster results.
Every two days, an instrumented version of the tests are run (close to 24 hours running time, hence the launch every two days). This gives them a full report with condition coverage, which can be used by developers to find out whether the code they’re modifying is covered by tests, and by which ones.
“Coco brings together the missing link between code and tests. Coco seemed like a regular code coverage tool at first, but after we started using it, we found some very advanced features.” -InnovMetric
InnovMetric was able to write macros for Visual Studio to get the right HTML coverage report according to the current file in the editor. The next step is to get a full list of test executions that cover the modified code (at the function level) and possibly run those tests.
Coco can also help optimize the execution order of tests so that the maximum coverage is performed by the earliest tests executed.
“Software without good support is not good software.”
Support is very important for InnovMetric, so this was one thing that made choosing Coco feel like the right choice. While getting up to speed, the Coco customer support team helped InnovMetric work out some of the initial kinks. As part of the process, the support staff found and quickly fixed some bugs in Coco, making it possible for InnovMetric to instrument their entire code base without modifications.
To learn more about Qt’s complete suite of quality assurance products, visit https://www.qt.io/product/testing-tools.
