Qbs 1.9 released

Qbs (pronounced “Cubes”) is The Qt Company’s latest build tool, which aims to be the replacement for qmake in the Qt build system. In addition to its use within Qt, for our customers Qbs will also provide faster build times coupled with a rich feature set in order to reduce build engineering complexity and decrease time to market.

Today I'm excited to introduce you to the latest release of Qbs - 1.9. This is an important release with foundational new features.

What's New?

In addition to the usual bug fixes and performance enhancements, this release includes two of the most foundational changes to the future of Qbs.

Language Improvements

The first of these is something we call Dependency Parameterization. Depends items can now be parameterized to set special module parameters for one particular product dependency. The new item type, Parameter, is used to declare such parameters in a module, while the new item type Parameters (note the 's') is used to allow products to set default values for such parameters in their Export item.

So, what can dependency parameterization do for you? First, let's step back and examine the meaning and usage of a Depends item.

Depends items are used to create a dependency between product A and product B, such that the target artifacts of the dependency can be used as inputs to the rules in the context of the depending product. In practice, Depends items have so far been used mostly to link an executable or library to another library. But a dependency is a very abstract concept, and can represent any type of relationship between two products. The fundamental problem is that whatever the meaning of the dependency, how that dependency took effect was only controllable from one side of the dependency relationship - the dependency's side.

Qbs 1.9 allows this relationship to be controlled from the depending product's side as well. For example, one of the products in your project may be a plugin which is implemented as a shared library. Your main application should depend on that plugin, because you want to ensure the plugin is built whenever the main application is, but you don't want to directly link to that plugin because you want to load it entirely at runtime.

In Qbs 1.8, we would create a Depends items pointing to our shared library, like so:

Depends { name: "library" }

But there's a problem - now our main application will actually link against the plugin shared library, which we don't want. This is why we need to control the dependency relationship from the depending product's side as well. In Qbs 1.9, we can now add a dependency parameter to the Depends item, like so:

Depends { name: "library"; <strong>cpp.link: false</strong> }

This tells Qbs that we want to depend on the "library" product, but don't want to actually link to it. This gets exactly the result we want - ensuring that "library" is built whenever our main application is, but without creating a direct linking relationship between the two.

So how does it work? First, the cpp module in Qbs declares the availability of a boolean dependency parameter called "link", using the Parameter item. Whenever a Depends item sets this property, its value will be applied to any input artifacts coming from the target artifacts of that product, when the current product's rules are processed. The linker Rule in the cpp module then checks the cpp.link parameter of any dependent shared libraries and only links to them if it is true.

You might ask, why go through the trouble of extending the language in this way if we could just express within "library" itself that it should not be linked? Perhaps via using a different file tag for the output? The reason is that you might have multiple products A and B within your project that want to depend on another product C, but A and B both want to depend on C in a different way. For example, if C is a macOS dynamic framework, B might want to Depend on C by linking to it, because B is another library which depends on some of C's functionality, whereas A is an application bundle that wants to depend on C by embedding it within itself, but not by linking to it, since the A application doesn't directly call any functions in C.

The dependency parameter that we've added so far are cpp.link, cpp.linkWholeArchive, and cpp.symbolLinkMode. Check out the cpp module documentation to find out how you can use them in your projects.

Essentially, Depends items now provide a much more flexible way for build engineers to express the relationship between products, leading to more correct and efficient builds, without complicated hacks and workarounds.

Multiplexing

The other important new feature we have is called Product Multiplexing. This feature replaces and is functionally a superset of the old profile multiplexing functionality we had previously implemented in order to support multi-architecture Android APKs.

The new implementation has been generalized to cover more use cases, and is a lot more powerful. Its primary function is to allow the creation of multi-architecture Android APKs and the creation of multi-architecture "fat" binaries and multi-variant frameworks (where a single framework can contain both a release and debug build of a library) on Apple platforms. We can also easily extend the multiplexing to more areas solely using QML, without having to modify the core build engine.

The problem with the old profile multiplexing is that it enforced a rigid project structure that was inherently dependent on the naming convention of the build profiles that were used. Such a structure wasn't in keeping with our philosophy to make profiles optional, nor did it play particularly nice with how Qt Creator handles Qbs-based projects.

Enter product multiplexing: instead of multiplexing a product over multiple profiles, we now allow a product to be multiplexed over a number of more general properties. Currently, these are qbs.architectures (which maps to qbs.architecture), qbs.buildVariants (which maps to qbs.buildVariant) and qbs.profiles (which maps to qbs.profile, hence keeping a certain degree of backwards compatibility with the old multiplexing as well).

Product multiplexing works by examining a special property on the Product item called multiplexByQbsProperties, which can be set to the list of properties your product should multiplex over. For example, let's say multiplexByQbsProperties contains two strings "architectures" and "buildVariants". Qbs will then evaluate the values of qbs.architectures and qbs.buildVariants, which in turn may contain the values, ["x86", "x86_64"] and ["debug", "release"]. Qbs will then build the product N times according to the number of values in the matrix. In our example, 4 configurations of the product will be built: (x86, debug), (x86, release), (x86_64, debug), and (x86_64, release). If the Product's aggregate property is true, the product will also be built a fifth time, where the values of the multiplexed properties are left undefined. The aggregate product will have an automatic dependency on the original four instances of the product, allowing it to collect and operate on their outputs artifacts.

The aggregate product is used in situations where the target artifacts of the individually multiplexed instances must be combined into one final aggregate artifact that makes up the overall product. For example, bundle products on macOS, iOS, tvOS, and watchOS use the aggregate product to create the bundle artifacts like Info.plist, PkgInfo, etc., which aren't dependent on a particular architecture or build variant, and also use the lipo tool to join together the built native code for different architectures (like x86 and x86_64) into the final, multi-architecture "fat binary" that goes inside the app bundle or framework.

Due to its general design and easily extensible implementation in the build engine, we hope that in the future we can discover even more use cases for product multiplexing. One case I've considered so far but have not yet investigated is building multi-architecture app bundles for the Universal Windows Platform.

Deployment and Packaging

One of the interesting new features with this release is the ability to build macOS disk images with custom backgrounds and icon layouts.

Most applications on macOS are installed via a .dmg file, which is usually customized by a fancy image background and custom icon layout. The problem is, it's quite challenging to construct such DMG files correctly. The custom background and icon layout relies on several undocumented proprietary file formats, some of which date back to the Mac OS Classic days, and which are even nested within one another.

Most of the existing tools to create DMG files do so by using AppleScript to manipulate the Finder graphically to generate the right icon layout, which is both unstable and incompatible with headless build servers since the necessary OS services to launch graphical applications may not be running at all. Other solutions will try to create the primary .DS_Store file manually, and commit to their source repository. This binary blob is difficult to inspect and edit, and might not be backwards compatible with older versions of the OS, depending on how it was generated.

The solution used by Qbs suffers none of these problems. It generates the necessary files programmatically, in an entirely reproducible manner, and there are no external dependencies that need to be separately installed and zero binary blobs committed to your source repository. It Just Works. Transform this:

AppleApplicationDiskImage {

    targetName: "cocoa-application-" + version

    version: "1.0"
    files: [

        "CocoaApplication/dmg.iconset",

        "CocoaApplication/en_US.lproj/LICENSE",

        // comment out the following line to use a solid-color background

        // (see dmg.backgroundColor below)

        "CocoaApplication/background*"

    ]
    dmg.backgroundColor: "#41cd52"

    dmg.badgeVolumeIcon: true

    dmg.iconPositions: [

        {"x": 200, "y": 200, "path": "Cocoa Application.app"},

        {"x": 400, "y": 200, "path": "Applications"}

    ]

    dmg.windowX: 420

    dmg.windowY: 250

    dmg.windowWidth: 600

    dmg.windowHeight: 422 // this *includes* the macOS title bar height of 22

    dmg.iconSize: 64

}

...into this:

macOS disk image built with Qbs

Qbs even supports multi-language license agreement prompts that appear when the DMG is opened,  with full Unicode and rich-text formatting support. And no, you don't have to deal with Carbon resource files from Mac OS Classic to do it!

In order to implement this functionality, we made use of several high quality Open Source Python modules (dmgbuild, ds_store, and mac_alias). I'd like to thank the developer, Alastair Houghton, for his hard work on and maintenance of these modules. The Qt Company has also contributed back a number of fixes and enhancements to these modules as part of the development of this feature.

Other New Features

• Setting module property values from the command line can now be done per product
• When rebuilding a project, the environment, project file, and property values are now taken from the existing build graph, which makes builds more self-contained and reliable
• Rules now have a property requiresInputs. If it is false, the rule will be run even if no artifacts are present that match its input tags
• Precompiled headers are now used by default if present in the product (cpp.use*PrecompiledHeader defaults to true now)
• Added support for Unified Headers in the Android NDK
• The property cpp.cxxLanguageVersion now gets mapped to MSVC's /std option, if applicable
• Added a new How-To section in the documentation with step-by-step guides on how to achieve common workflows
• Context-sensitive help is now enabled for Qbs items in Qt Creator 4.4, which includes Qbs 1.9

Also, the loadFile() and loadExtension() function have now been formally deprecated and replaced by require(), as originally stated in the Qbs 1.8 release blog.

Try It!

The Open Source version is available on the download page, and you can find commercially licensed packages on the Qt Account Portal. Please post issues in our bug tracker. You can also find us on IRC in #qbs on chat.freenode.net, and on the mailing list. The documentation and wiki are also great places to get started.

Qbs is also available on a number of packaging systems (Chocolatey, MacPorts, Homebrew) and updated on each release by the Qbs development team. It can also be installed through the native package management system on a number of Linux distributions including but not limited to Debian, Ubuntu, and Arch Linux. An official Fedora package is in progress and should be available in a future release of Fedora.

Qbs 1.9 is also included in Qt Creator 4.4, which was also released today.