Graphics in Qt 6.0: QRhi, Qt Quick, Qt Quick 3D

Last year we had a three part blog series about Qt's new approach to working with 3D graphics APIs and shading languages: part 1, part 2, part 3. For Qt Quick, an early, opt-in preview of the new rendering architecture was shipped in Qt 5.14, with some improvements in Qt 5.15. With the release of Qt 6.0 upcoming, let's see what has happened since Qt 5.15. It will not be possible to cover every detail of the graphics stack improvements for Qt Quick here, let alone dive into the vast amount of Qt Quick 3D features, many of which are new or improved in Qt 6.0. Rather, the aim is just to give an overview of what can be expected from the graphics stack perspective when Qt 6.0 ships later this year.

Note that the documentation links refer to the Qt 6 snapshot documentation. This allows seeing the latest C++ and QML API pages, including all changed and new functions, but the content is also not final. These links may also break later on.

QRhi improvements

QRhi, the Qt Rendering Hardware Interface, is Qt's internal graphics abstraction when 3D APIs, such as OpenGL, Vulkan, Metal, and Direct 3D, are involved. Compared to 5.15, the main improvements in 6.0 are a lot of polishing fixes here and there, and, most importantly, a large set of performance optimizations. While benefitting Qt Quick as well, these become especially important with Qt Quick 3D when complex scenes with many renderable objects are involved.

With some simplifications, the main layers of the Qt 6.0 graphics stack can be visualized like this:

Shader management

The Qt Shader Tools module is now a selectable module in the installer. For applications this can be relevant because this is the module that provides the qsb command-line tool (not to be confused with qbs) and its associated CMake build system integration. In addition, the module is a mandatory dependency for Qt Quick 3D at the moment.

Qt 6 no longer uses OpenGL-compatible GLSL source snippets directly. Rather, shaders are all written in Vulkan-style GLSL, then reflected and translated to other shading languages, and finally packaged up into a serializable QShader object that can be consumed by QRhi. The shader preparation pipeline in Qt 6 is the following:

In QML applications using Qt Quick, whenever working with ShaderEffect, or subclassing QSGMaterialShader, the application will need to provide a baked shader pack in form of a .qsb file. These are generated by the qsb tool. This does not however mean that developers have to start dealing with a new tool directly: with the CMake integration one can easily list the vertex, fragment, and compute shaders in CMakeLists.txt via the qt6_add_shaders() CMake function. Invoking qsb and packing the resulting .qsb files into the Qt resource system is then taken care of by the build system.

See the shadertools documentation for an overview of how graphics and compute shaders are handled in Qt 6 and the details of the qsb tool and its CMake integration.

Direct OpenGL is no more for Qt Quick

In Qt 5.14 and 5.15, Qt Quick shipped with an optional QRhi-based rendering path that could be enabled by setting the environment variable QSG_RHI. This allowed painless experimenting with the new stack, while keeping the traditional, battle tested direct OpenGL code path the default.

In Qt 6.0 all such switches are gone. There is no way get rendering go directly to OpenGL with Qt Quick scenes. Rather, the new default is the QRhi-based rendering path of the Qt Quick scene graph. Other than the defaults changing, the ways to configure what QRhi backend, and so which graphics API to use are mostly unchanged compared to Qt 5.15. See the documentation for details. One difference is better API naming: in C++ code to request, and so effectively tie the application to, a given QRhi backend (and by extension graphics API) is now done through the QQuickWindow::setGraphicsApi() function, whereas in 5.15 this task used to be pushed onto an overload of setSceneGraphBackend(), leading to fairly inaccurate naming.

There are a number of implications, although many applications will not notice any of these. If an application uses neither shader code (ShaderEffect, QSGMaterial) nor does it perform its own rendering with OpenGL directly, there is a very high chance that it will need no migration steps at all. (at least not because of graphics)

Applications using OpenGL directly

What about applications that use OpenGL directly in one way or another, and are not interested in functioning with other graphics APIs? For example, applications that use QQuickFramebufferObject, or connect to signals like QQuickWindow::beforeRendering() to inject their own OpenGL rendering under or above the Qt Quick scene. This is when the setGraphicsApi() function mentioned above comes into play for real: if an application wishes, it can always state that it wants OpenGL (or Vulkan, or Metal, or D3D) only, and nothing else. That way it is guaranteed that Qt Quick is going to use the corresponding QRhi backend (or else it will fail to initialize), so the application can safely assume that going directly to OpenGL is safe, because Qt Quick will also end up rendering through OpenGL. Note that this does not exempt the application from having to do other type of porting steps: for example, if it in addition uses ShaderEffect or creates its own custom materials, it will still need to migrate to the new ways of handling shaders and materials.

QSG* and QQuick* API changes

The API changes mainly fall into 3 categories. This is not going to be an exhaustive list, but rather just a peek at some of the important changes. Detailed change lists and porting guides are expected to be available with the final Qt 6.0 release.

• Different approach to shaders and materials: QSGMaterialShader received a full revamp (matching more or less what the now-removed QSGMaterialRhiShader used to be in 5.14 and 5.15). ShaderEffect no longer allows inline shader strings. Rather, the vertexShader and fragmentShader properties are URLs, similarly to Image.source and others. They can refer to a local .qsb file, or a .qsb file embedded via the Qt resource system (qrc).
  
  

• Removing OpenGL-specifics from QQuickWindow, QSGTexture, and elsewhere. It should come as no surprise that functions like GLuint textureId(), createTextureFromId(GLuint textureId, ...), or setRenderTarget(GLuint fboId) are now gone. Adopting (wrapping) an existing OpenGL texture, Vulkan image, Metal texture, or D3D11 texture, or accessing the underlying native texture for a QSGTexture is still perfectly possible, but now is done via a different set of APIs, such as QSGVulkanTexture and the other similar classes, instances of which are queryable from QSGTexure.

  • Integrating the application's own custom rendering with the graphics API that Qt Quick renders with is fully supported, not just for OpenGL, but also Vulkan, Metal, and D3D11. Due to their nature however, some of these APIs will need more than connecting to one single signal like beforeRendering() or afterRendering(). For example, we now also have beforeRenderPassRecording(). See the relevant section in the scenegraph overview docs for more details and links to examples. Finally, the number of native graphics resources queryable via QSGRendererInterface has been extended, now covering Vulkan, Metal, and Direct 3D too.
    
    

• Extending support for redirecting the Qt Quick scene into an offscreen render target. QQuickRenderControl and the related infrastructure has been heavily enhanced. This was done not just to enable working with graphics APIs other than OpenGL the same way as in Qt 5 (for example, to render a Qt Quick scene into a Vulkan VkImage without an on-screen window), but also to enable integration with AR/VR frameworks and APIs such as OpenXR (in combination with any of Vulkan, D3D11, or OpenGL). Besides the slightly changed QQuickRenderControl interface, we now have a number of helper classes that improve the configurability of a QQuickWindow: QQuickRenderTarget, QQuickGraphicsDevice, and QQuickGraphicsConfiguration. These are essential in scenarios where a more fine grained control is needed: integrating with APIs like OpenXR is not always straightforward when an existing rendering engine is involved, with a number of potential chicken-egg problems when it comes to the creation, initialization, and ownership of instance, device, and other graphics objects: Which Vulkan instance should Qt Quick use, or should it create a new one upon initializing the scenegraph for the first time? Which Vulkan physical device or DXGI adapter should Qt Quick pick, or just stay with the default? Which VkDevice extensions should be enabled in addition to what Qt itself needs? What 2D image/texture should rendering target, who creates that and when? The expectation is that Qt 6.0 will be well-prepared and providing the foundations for further exploring the world of AR/VR during the rest of the Qt 6.x series.

New approach to handling shader code in ShaderEffect

A comprehensive example of the new approach to shader code in ShaderEffect is the Qt 6 port of the classic Qt 5 Cinematic Experience demo. (GitHub repo) This version is ported to CMake and is fully functional with all graphics APIs, including all shader and particle effects.

Looking at the QML source code, for example the code for the curtain effect shows that indeed it has all inline GLSL strings removed.

Instead, the vertex and fragment shaders now live as ordinary files in the source tree, not bundled with the application executable anymore.

It is now up to the build system and Qt Shader Tools to compile, reflect, and translate at build time - with the added benefit of shader compilation errors becoming proper build errors instead of obscure runtime problems! - and then bundle the resulting .qsb files with the application. This is what the qt6_add_shaders() function achieves in the project's CMakeLists.txt

New scenegraph examples

Those interested in some of the lower level topics, such as working directly with the scenegraph, or integrating 3D rendering code with it, are advised to check out the revised list of scenegraph examples that ship with Qt, see the Scene Graph section here. All of these have been updated for Qt 6.0, whereas some of them are brand new.

For instance, the Custom Material example has been introduced specifically to focus on how a custom QQuickItem using its own material can be implemented.

Also noteworthy are the graphics API specific examples, that follow in the vein of Qt 5's openglunderqml example, now demonstrating how to achieve the same with Vulkan, Metal, and Direct3D 11. Naturally, these examples are only functional with the graphics API in question. Looking at their main() function will reveal that they all enforce the relevant RHI backend.

Some of these go further than the classic underlay/overlay approach. For instance, the metaltextureimport and vulkantextureimport examples demonstrate adding a QQuickItem to the scene that effectively is a textured quad, being textured with a MTLTexture or VkImage that was created and rendered to outside the Qt Quick Scenegraph's control.

Direct OpenGL is no more for Qt Quick 3D

In Qt 5.15 the major news was the introduction of Qt Quick 3D, making the 3D world, 3D models, and PBR materials first class citizens of the Qt Quick world. In many ways this was merely a preview of what is to come in Qt 6.0.

While still tied to OpenGL in Qt 5.15, Qt 6.0 ships with renewed internals for Qt Quick 3D, now based on the QRhi-based infrastructure. As the documentation pages describe, the configuration options that apply to Qt Quick also apply to Qt Quick 3D implicitly. If Qt Quick is configured to use Vulkan for example, by setting QSG_RHI_BACKEND=vulkan or using the equivalent C++ APIs, the same will apply to Qt Quick 3D too.

The entire feature set, including materials, lighting, shadows, image-based lighting, the renewed custom material and post-processing effect systems are all fully functional with all supported graphics APIs (OpenGL, Vulkan, Metal, Direct 3D 11).

In line with the general renewal of how shaders are handled in Qt, the concept of custom materials has also undergone a major transformation in Qt 6.0. There is now a whole new extension of the Qt Quick 3D material system that allows creating programmable materials, where the way a mesh is shaded is controlled by application-provided shader code, provided in form of Vulkan-compatible GLSL snippets, going through Qt's standard shader pipeline described above, thus becoming functional with any of the supported graphics APIs at run time, while still being amended by the engine to perform the all the expected lighting, shadowing, occlusion, and other steps, combining all contributions from the scene environment with the application-provided shading logic.

Those interested in looking into the details already now are welcome to check out the CustomMaterial page and the work-in-progress getting started guide from the snapshot documentation.

What about Widgets?

The diagram in the QRhi section mentioned Widgets, albeit placed it in a perhaps odd position at first glance, away from QRhi and Qt Quick. What does this try to indicate?

In general everything that worked in Qt 5 can be expected to work in Qt 6, with the exception of deprecated and now-removed functionality, such as all the Qt 4 era classes with the QGL prefix (most notably, QGLWidget). If a Qt 5 application renders its own OpenGL content into a QWindow, or uses QOpenGLWidget, it will all function as before. (in the worst case with some very minor migration steps, e.g. having to update the application project file due to QOpenGLWidget moving to its own module openglwidgets).

QRhi or the new shader pipeline plays no role here at the moment, in Qt 6.0 at least. The rendering of widgets happens like in Qt 5, whereas OpenGL content in a QOpenGLWindow or QOpenGLWidget continues to use the OpenGL API directly.

QQuickWidget is an interesting hybrid in Qt 6.0, and will require the application to enforce using OpenGL. This is because while Qt Quick could function with other graphics APIs, the composition architecture in widgets (which QOpenGLWidget and QQuickWidget rely on) continues to use OpenGL directly for the time being.

One notable change is the removal of ANGLE, meaning ANGLE is no longer bundled with Qt on Windows. This will not affect the vast majority of applications, with the exception of those that in some form rely on ANGLE translating OpenGL ES to Direct 3D under the hood. Depending on the nature and complexity of their dependency, such applications should consider either enabling functioning with OpenGL proper, or investigate moving to using Direct 3D directly. For applications based on Qt Quick and Qt Quick 3D not having ANGLE around will likely not be a problem in practice since Direct3D 11 is now a first-class rendering option in Qt 6.

That is all for now. There are plenty of more Qt 6 graphics topics to talk about, but hopefully what we have above provides a good starting point to the changes and new features in Qt 6.0. Expect more posts in the near future, especially in the context of Qt Quick 3D.