Let's continue where we left off in the first post. We saw an example of a Qt Quick application running on Linux on top of OpenGL and Vulkan. We also saw a Vulkan frame capture in RenderDoc, which is not just an invaluable tool during Qt development work, but can also be useful to anyone who wants to dig deeper and understand better how Qt Quick renders a frame (or for that matter troubleshoot problems in an application's rendering). Now in this post we are going to focus on what Qt 5.14 offers for macOS and Windows.
Metal on macOS
Quite unsprisingly, running the qt5-cinematic-experience demo with QSG_RHI=1 (and QSG_INFO=1) on macOS 10.13 or 10.14 results in:
qt.scenegraph.general: Using QRhi with backend Metal
graphics API debug/validation layers: 0
QRhi profiling and debug markers: 0
qt.scenegraph.general: threaded render loop
qt.scenegraph.general: Using sg animation driver
qt.scenegraph.general: Animation Driver: using vsync: 16.67 ms
qt.rhi.general: Metal device: Intel(R) HD Graphics 615
qt.scenegraph.general: MSAA sample count for the swapchain is 1. Alpha channel requested = no.
qt.scenegraph.general: rhi texture atlas dimensions: 4096x2048
qt.rhi.general: got CAMetalLayer, size 2560x1440
In line with Qt 6's target of defaulting to the platform's primary, best supported graphics API (while still allowing applications to set their own preferences, if they wish to), enabling the QRhi-based rendering path in Qt Quick on macOS defaults to using Metal.
Similarly to how the Vulkan-based rendering path builds on the Vulkan instance and surface support introduced to the QtGui module, QPA, and some of the platform plugins, the Metal backend of QRhi relies on the cocoa platform plugin's Metal support that got introduced around Qt 5.12. A QWindow with QSurface::MetalSurface gets an NSView backed by a CAMetalLayer under the hood. That is exactly what the Metal backend of QRhi needs. (this all works because QQuickWindow gets the appropriate QSurface::SurfaceType set whenever the QRhi-based code path is active)
One important feature here is the ability to run with the threaded render loop of Qt Quick. This is very welcome because due to threading problems of certain NSOpenGLContext and related APIs in macOS 10.14, Qt disables the threaded loop with OpenGL on macOS, defaulting to the 'basic' loop instead. This leads to a somewhat reduced smoothness in Qt Quick animations. With Metal we do not (according to our current knowledge) have any issues with the threaded setup, so we can once again default to the threaded render loop. (the default choice of the render loop can be overridden by setting the QSG_RENDER_LOOP environment variable; this variable is supported also in combination with QSG_RHI)
We mentioned RenderDoc as a tool for debugging the frame rendering of Qt applications when running on OpenGL, Vulkan, or Direct3D. For Metal, one can use XCode and its built-in GPU frame capture.
One useful way to quickly open a Qt project in XCode ready for debugging is to run make xcodeproj && open *.xcodeproj (or qmake -spec macx-xcode if qmake was not yet run) from the terminal. Hitting Cmd-R then starts the debugger right up. We too use this a lot during Qt development. In case the Metal GPU capture is not available, even though Qt Quick is set to render via Metal, check Product -> Scheme -> Edit scheme..., and change GPU Frame Capture to Metal.
Running a debug build of an application in XCode will have Metal validation enabled, which means XCode will drop a (hopefully) helpful warning and break program execution when a Metal API is used in some incorrect way. Unlike other platforms, the Metal API validation cannot be enabled when not debugging via XCode. (so unlike Vulkan and D3D, setting QSG_RHI_DEBUG_LAYER will have no effect)
Metal on iOS
What about iOS or tvOS?
Well, at the time of writing the Metal-related plumbing in the platform plugin is missing, which also means the Metal backend of QRhi has not yet been tested on these platforms. That is why the Qt 5.14 new features page only mentions macOS in the Qt Quick section. Support is expected to be added in the not too far future, perhaps in 5.15.
Vulkan on macOS
What about Vulkan via MoltenVK?
As mentioned in part 1, requesting rendering via Vulkan and then relying on MoltenVK to translate the API calls and SPIR-V shaders at runtime to Metal and the Metal Shading Language is an option as well. This requires a Vulkan-enabled Qt build which is not the case out of the box on Apple platforms. See this post for details, the key is to pass -I to configure and making sure the libraries can be found at run time by possibly setting QT_VULKAN_LIB.
It is important to note that this approach is going to be supported on a best effort basis only. The primary supported way to render on Apple platforms is going to be via Metal.
Running the demo application with QSG_RHI_BACKEND=vulkan (using an appropriately configured Qt 5.14 build) gives us:
qt.scenegraph.general: Using QRhi with backend Vulkan
graphics API debug/validation layers: 0
QRhi profiling and debug markers: 0
qt.scenegraph.general: threaded render loop
qt.scenegraph.general: Using sg animation driver
qt.scenegraph.general: Animation Driver: using vsync: 16.67 ms
qt.rhi.general: Physical device 0: 'Intel(R) UHD Graphics 630' 0.2.1835 (api 1.0.92 vendor 0x8086 device 0x3E9B type 1)
qt.rhi.general: using this physical device
qt.rhi.general: queue family 0: flags=0x7 count=1
qt.rhi.general: 17 device extensions available
qt.scenegraph.general: MSAA sample count for the swapchain is 1. Alpha channel requested = no.
qt.scenegraph.general: rhi texture atlas dimensions: 2048x1024
qt.rhi.general: Creating new swapchain of 2 buffers, size 1280x720, presentation mode 2
It is worth noting that I had problems getting this running on macOS 10.13, experiencing lock ups and crashes when using the threaded render loop. Upgrading to a newer (1.0.121) Vulkan SDK (that includes MoltenVK) led to a new set of problems and not being able to start the application. (I kept getting something that resembles https://github.com/KhronosGroup/MoltenVK/issues/695) To be investigated later on. Here, on a Mac Mini with 10.14 and a semi-recent Vulkan SDK / MoltenVK, the results are pretty good however, as witnessed on the screenshot.
Windows
Windows is the platform where we have the most options. Out of the 4 main QRhi backends (Vulkan, Metal, D3D11, OpenGL) one can use no fewer than 3 on Windows: Direct3D 11, Vulkan, and OpenGL.
This begs the obvious question right away: Why only 3? Should it not be 4, with Direct3D 12 being the fourth?
There is no D3D12 backend at the moment. This may change later on since this is in the plans, but not currently scheduled. While we are at the topic, it is worth noting that the experimental direct-to-D3D12 backend for Qt Quick added in Qt 5.8 is now architecturally deprecated (since we tackle the problem of multiple graphics APIs in a whole new way), and is subject to removal in Qt 6.
On Windows the default when setting QSG_RHI=1 is Direct3D 11. As usual, override with QSG_RHI_BACKEND, if Vulkan or OpenGL is desired.
qt.scenegraph.general: Using QRhi with backend D3D11
graphics API debug/validation layers: 0
QRhi profiling and debug markers: 0
qt.scenegraph.general: threaded render loop
qt.scenegraph.general: Using sg animation driver
qt.scenegraph.general: Animation Driver: using vsync: 16.67 ms
qt.rhi.general: DXGI 1.2 = true, FLIP_DISCARD swapchain supported = true
qt.rhi.general: Adapter 0: 'NVIDIA GeForce RTX 2060' (flags 0x0)
qt.rhi.general: using this adapter
qt.rhi.general: Adapter 1: 'Microsoft Basic Render Driver' (flags 0x2)
qt.scenegraph.general: MSAA sample count for the swapchain is 1
qt.scenegraph.general: rhi texture atlas dimensions: 2048x1024
It is worth noting that we need Direct3D 11.1, not 11.0. This is mainly because we need VSSetConstantBuffers1
(and the related PS and CS variants). This should not present a problem unless someone wants to run on plain Windows 7 without the Platform Update. Speaking of Windows 7, it is important to note that the D3D11-based rendering does not currently function correctly on Windows 7 as of Qt 5.14, hence mentioning only Windows 10 on the 5.14 new features page. This is something to be remedied later on (but only if Windows 7 is going to stay a supported platform for Qt 6).
To enable the Direct3D debug layer, set the environment variable QSG_RHI_DEBUG_LAYER to 1. This works for Vulkan too, as long as the validation layers are available (from an installed Vulkan SDK for instance). Qt conveniently directs the messages to the debug output (as if they were output via qDebug).
Vulkan and OpenGL should work as expected on Windows, so I will refrain from making the post longer by adding more screenshots.
As with Vulkan and Metal, the OpenGL backend of QRhi builds on some (but not all) of the existing OpenGL enablers and platform plumbing, such as, QOpenGLContext, QOpenGLFunctions, and the underlying plumbing in the windows platform plugin (WGL, EGL). Therefore, everything that applies to Qt Quick applications running directly on OpenGL, applies here as well. (desktop vs. ANGLE vs. software, environment variables like QT_OPENGL)
Speaking of ANGLE, the expectation is that it can be removed as a direct dependency in Qt 6. This needs some further investigation to fully ensure we do not lose support for special use cases, but the plan for now is that having the QRhi-based rendering path with support for Direct3D 11, OpenGL (proper, via WGL), and Vulkan is going to be sufficient for Qt 6 applications on Windows.
And that's it for this post. In part 3 we are going to finally start diving deeper and look into what this QRhi thing is and how shaders are handled.
Commenting for this post has ended.
Say we're targeting web platform(WebAssembly), which API should we use if ANGLE got removed.
Does RHI have any effect on WebAssembly/WebGL?
From Qt's perspective it does not make a difference, there is no effect on the WebAssembly port and WebGL. The WebAssembly port relies on whatever magic Emscripten & co. does to take Qt's OpenGL ES 2.0 calls and generate WebGL from them on the browser side. ANGLE is not used there (in Qt itself). The rest is up to the browser - there it could well be that ANGLE, Swiftshader, or some other solution is used by Chrome, Firefox, etc. to enable and support WebGL, but that is not in Qt's domain anymore.
One case where the new graphics abstraction layer may make a difference also for WebAssembly in the future is the question of what comes after WebGL. If for example the future is WebGPU, then the new QRhi-based rendering path is much better prepared for that than the traditional OpenGL only code path. (but how this would look in practice is not yet known, just speculating for now)
Awesome news ! There is one thing I do not understand, how does it fit with QPAs ? For example if I start an application with '-platform eglfs', this would launch it using EGL, but if I set the RHI environment variables to enable it, would the application start using EGL or Vulkan/Metal/... (or both) ? If not, do you think a '-platform rhi' could become a thing ?
It is on a different level. Think about it, launching with -platform windows could mean using D3D, Vulkan, or OpenGL. It is already fuzzy in Qt 5 since platform plugin != windowing system interface+ graphics API. (we need to deal with GLX and EGL in the xcb plugin, WGL and EGL (for ANGLE) in the windows plugin, etc., what we'll have now simply extends that to include different graphics APIs as well)
So in the architectural stack the RHI sits together with the QtGui module (where in Qt 5 the OpenGL enablers are), above QPA.
Alright so by doing 'app -platform eglfs' in linux, it should start ' app' using RHI (API) and EGL (system interface), isn't it ?
What I am looking for is a way to replace 'EGL' streams around Wayland (server and client) by 'Vulkan' ones in the linux graphic stack, is RHI useful in this case ?
Running with eglfs will only support the OpenGL backend of the RHI. The stack is then something like:
QtGui/QRhi -> QPA -> eglfs -> eglfs backend (f.ex. eglfs_kms) -> libdrm (or fbdev in some legacy systems)
(of course, an EGL/OpenGL implementation, for example Mesa, is in there in the picture too, with arrows pointing to it both from QtGui/QRhi and eglfs)
How we set up Vulkan rendering without a windowing system is something to be seen, there is no platform plugin currently that enables that. (eglfs is not suitable for this, as the name too suggests)
That EGLStream, Wayland, etc. plumbing you mentioned is in the domain of the platform plugin (and QtWayland, if using that), the RHI is on a different level.
Nice blog post as always. What about the Qt 5.14 beta release? The wiki (https://wiki.qt.io/Qt5.14...) states that it should have been released yesterday... I want to test my issue with the new performance improvements of Qt3d and the RHI
https://bugreports.qt.io/br...
Cheers
It is delayed. The alpha is not even out yet, I think that should be available during the next 2 weeks at some point. The beta comes afterwards.
Thanks for the quick reply!
Are we going to get QRhi-backed QPainter, too ?
Most likely, but do not read this as a 100% commitment. There has been some investigation for this already, and a very minimal QRhi-based paint engine has been brought up.
The goal here would be to provide a replacement for the OpenGL paint engine. Running with OpenGL is still an option then, but via the RHI. The aging OpenGL paint engine is then removed in Qt 6, just like the direct OpenGL code path is planned to be removed in Qt Quick. So the story mirrors Qt Quick's pretty much.
What this would enable is then things like supporting the "FBO" approach of QQuickPaintedItem, and, more importantly, potentially introducing the ability to once again render widgets through the accelerated APIs. For example, there has been some interest in having widgets rendered through Metal on macOS, in the hope of getting better performance (than the raster paint engine) for certain type of UIs on high resolution screens. Such a widget/backingstore integration needs further work however, a QRhi-based paint engine is just one enabler for that. However, it is not given this all will happen or makes sense even. Needs further research.
Have you tried to turn off vsync to compare FPS?
No, and while this (setting a swap interval of 0 or switching to a presentation mode different from FIFO) is possible for most (but not all) backends, using this to compare performance would be a somewhat faulty approach since the way the rendering loops work is based on expecting vsync-based throttling. To get proper results we'd need to rework some of the backends to cater for this type of unthrottled benchmarking use case.
The same applies to Qt Quick even in the OpenGL-only Qt 5 world. https://github.com/qt-labs/... does not rely on unthrottled FPS counting either.
It is true however that we will need to get back to this topic, maybe not for 5.14 but later on. There are already some tasks for the future to investigate and get a better idea of how the different backends compare to each other in terms of CPU/GPU usage for instance.