The Qt Quick Graphics Stack in Qt 5.8

This is a joint post with Andy. In this series of posts we are going to take a look at some of the upcoming features of Qt 5.8, focusing on Qt Quick.

OpenGL... and nothing else?

When Qt Quick 2 was made available with the release of Qt 5.0, it came with the limitation that support for OpenGL (ES) 2.0 or higher was required.  The assumption was that moving forward OpenGL would continue its trajectory to be the hardware acceleration API of choice for both desktop, mobile and embedded development. Fast forward a couple years down the road to today, and the graphics acceleration story has gotten more complicated.  One assumption we made was that the price of embedded hardware with OpenGL GPUs would continue to drop and they would be ubiquitous.  This is true, but at the same time there are still embedded devices available without OpenGL-capable GPUs where customers continue to wish to deploy Qt Quick applications.  To remedy this we released the Qt Quick 2D Renderer as separate plugin for Qt Quick in Qt 5.4.

At the same time it turned out that Qt Quick applications deployed on a wide range of machines including older systems often have issues with OpenGL due to missing or unavailable drivers, on Windows in particular. Around Qt 5.4 the situation got improved with the ability to dynamically choose between OpenGL proper, ANGLE, or a software OpenGL rasterizer. However, this does not solve all the problems and full-blown software rasterizers are clearly not an option for low-end hardware, in particular in the embedded space. All this left us with the question of why not focus more on the platforms' native, potentially better supported APIs (for example, Direct3D), and why not improve and integrate the 2D Renderer closer with the rest of the Qt Quick instead of keeping it a separate module with a somewhat arcane installation process.

Come other APIs

Meanwhile, the number of available graphics hardware APIs has increased since the release of Qt Quick 2. Now rather than the easy to understand Direct3D vs OpenGL choice, there is a new generation of lower level graphics APIs available: Vulkan, Metal, and Direct3D 12. So for Qt 5.8 we decided to explore how we can make Qt Quick more future proof, as introduced in this previous post.

Modularization

The main goal for the ScenegraphNG project was to modularize the Qt Quick Scene graph API and remove the OpenGL dependencies in the renderer.  By removing the strong bindings to OpenGL and enhancing the scenegraph adaptation layer it is now possible to implement additional rendering backends either built-in to Qt Quick itself or deployed as dynamically loaded plugins. OpenGL will still be the default backend with full compatibility for all existing Qt Quick code. The changes are not just about plugins and moving code around, however. Some internal aspects of the scenegraph, for instance the material system, exhibited a very strong OpenGL coupling which could not be worked around in a 100% compatible manner when it comes to the public APIs. Therefore some public scenegraph utility APIs got deprecated and a few new ones got introduced. At the time of writing work is still underway to modularize and port some additional components, like the sprite and particle systems, to the new architecture.

To prove that the changes form a solid foundation for future backends, Qt 5.8 introduces an experimental Qt Quick backend for Direct3D 12 on Windows 10 (both traditional Win32 and UWP applications). In the future it will naturally be possible to create a Vulkan backend as well, if it is deemed beneficial. Note that all this has nothing to do with the approaches for integrating custom rendering into QWidget-based or plain QWindow applications. There adding Vulkan or D3D12 instead of OpenGL is possible already today with the existing Qt releases, see for instance here and here.

Qt Quick 2D Renderer, integrated

The Qt Quick 2D Renderer was the first non-OpenGL renderer, but when released, it lived outside of the qtdeclarative code base (which contains the QtQml and QtQuick modules) and carried a commercial-only license. In Qt 5.7 the Qt Quick 2D Renderer was made available under GPLv3, but still as a separate plugin with the OpenGL requirement inherited from Qt Quick itself. In practice this got solved by building Qt against dummy libGLESv2 library, but this was neither nice nor desirable long-term. With Qt 5.8 the Qt Quick 2D renderer is merged into qtdeclarative as the built-in software rendering backend for the Qt Quick SceneGraph. The code has also been relicensed to have the same licenses as QtDeclarative. This also means that stand-alone 2D renderer plugin is no longer under development and the qtdeclarative-render2d repository will become obsolete in the future.

Supercharging the 2D Renderer: Partial updates

The 2D Renderer, which is now referred to mostly as the software backend (or renderer or adaptation), is getting one huge new feature that was not present in the previous standalone versions: partial updates. Previously it would render the entire scene every frame from front to back, which meant that a small animation in a complicated UI could be very expensive CPU-wise, especially when moving towards higher screen resolutions. Now with 5.8 the software backend is capable of only rendering what has changed between two frames, so for example if you have a blinking cursor in a text box, only the cursor and area under the cursor will be rendered and copied to the window surface, not unlike how the traditional QWidgets would operate. A huge performance improvement for any platform using the software backend.

QQuickWidget with the 2D Renderer

Another big feature that the new software backend introduces with Qt 5.8 is support for QQuickWidget. The Qt Quick 2D Renderer was not available for use in combination with QQuickWidget, which made it impossible for apps like Qt Creator to fall back to using the software renderer. Now because of the software renderer's closer integration with QtDeclarative it was possible to enable support for the software renderer with QQuickWidget. This means that applications using simple Qt Quick scenes without effects and heavy animation can use the software backend in combination with QQuickWidget and thus avoid potential issues when deploying onto older systems (think the OpenGL driver hassle on Windows, the trouble with remoting and X forwarding, etc.). It is important to note that not all types of scenes will perform as well with software as they do with OpenGL (think scrolling larger areas for instance) so the decision has to be made after investigating both options.

No OpenGL at all? No problem.

One big limitation of the Qt Quick 2D Renderer plugin was that in order to build QtDeclarative, you still had to have OpenGL headers and libraries available. So on devices that did not have OpenGL available you had to use provided "dummy" libraries and headers to trick Qt into building QtDeclarative and then enforce your developers not to call any code that could call into OpenGL. This always felt like a hack, but with the hard requirement in QtDeclarative there was no better options available. Until now. In Qt 5.8 this is not an issue because QtDeclarative can now be built without OpenGL. In this case the software renderer becomes the default backend instead of OpenGL. So whenever Qt is configured with -no-opengl or the development environment (sysroot) lacks OpenGL headers and libraries, the QtQuick module is no longer skipped. In 5.8 it will build just fine and default to the software backend.

Switching between backends

Now that there are multiple backends that can render Qt Quick we also needed to provide a way to switch between which API is used. The approach Qt 5.8 takes mirrors how QPA platform plugins or the OpenGL implementation on Windows are handled: the Qt Quick backend can be changed on a per-process basis during application startup. Once the first QQuickWindow, QQuickView, or QQuickWidget is constructed it will not be possible to change it anymore.

To specify the backend to use, either set the environment variable QT_QUICK_BACKEND (also known as QMLSCENE_DEVICE that is inherited from previous versions) or use the C++ API of the static functions QQuickWindow provides. When no request is made, a sensible default will be used. This is currently the OpenGL backend, except in Qt builds that have OpenGL support completely disabled.

As an example, let's force the software backend in our application:

int main(int argc, char **argv)
{
    // Force the software backend always.
    QQuickWindow::setSceneGraphBackend(QSGRendererInterface::Software);
    QGuiApplication app(argc, argv);
    QQuickView view;
    ...
}

Or launch our application with the D3D12 backend instead of the default OpenGL (or software):

C:\MyCoolApp>set QT_QUICK_BACKEND=d3d12

C:\MyCoolApp>debug\MyCoolApp.exe

To verify what is happening during startup, set the environment variable QSG_INFO to 1 or enable the logging category qt.scenegraph.general. This will lead to printing a number of helpful log messages to the debug or console output, depending on the type of the application. To monitor the debug output, either run the application from Qt Creator or use a tool like DebugView.

With an updated version of the Qt 5 Cinematic Experience demo the result is something like this:

Qt 5 Cinematic Experience demo application running on Direct3D 12

Everything in the scene is there, including the ShaderEffect items that provide a HLSL version of their shaders. Unsupported features, like particles, are gracefully ignored when running with such a backend.

Now what happens if the same application gets launched with QT_QUICK_BACKEND=software?

Qt5 Cinematic Experience demo application running on the Software backend

Not bad. We lost the shader effects as well, but other than that the application is fully functional. And all this without relying on a software OpenGL rasterizer or other extra dependencies. No small feat for a framework that started out as a strictly OpenGL-based scene graph.

That's it for part one. All this is only half of the story - stay tuned for part two where are going to take a look at the new Direct3D 12 backend and what the multi-backend Qt Quick story means for applications using advanced concepts like custom Quick items.