.webp)
As Lars mentioned in his Technical Vision for Qt 6 blog post, we have been researching how we could have a deeper integration between 3D and Qt Quick. As a result we have created a new project, called Qt Quick 3D, which provides a high-level API for creating 3D content for user interfaces from Qt Quick. Rather than using an external engine which can lead to animation synchronization issues and several layers of abstraction, we are providing extensions to the Qt Quick Scenegraph for 3D content, and a renderer for those extended scene graph nodes.
Does that mean we wrote yet another 3D Solution for Qt? Not exactly, because the core spatial renderer is derived from the Qt 3D Studio renderer. This renderer was ported to use Qt for its platform abstraction and refactored to meet Qt project coding style.
"San Miguel" test scene running in Qt Quick 3D
What are our Goals? Why another 3D Solution?
Unified Graphics Story
The single most important goal is that we want to unify our graphics story. Currently we are offering two comprehensive solutions for creating fluid user interfaces, each having its own corresponding tooling. One of these solutions is Qt Quick, for 2D, the other is Qt 3D Studio, for 3D. If you limit yourself to using either one or the other, things usually work out quite fine. However, what we found is that users typically ended up needing to mix and match the two, which leads to many pitfalls both in run-time performance and in developer/designer experience.
Therefore, and for simplicity's sake, we aim have one runtime (Qt Quick), one common scene graph (Qt Quick Scenegraph), and one design tool (Qt Design Studio). This should present no compromises in features, performance or the developer/designer experience. This way we do not need to further split our development focus between more products, and we can deliver more features and fixes faster.
Intuitive and Easy to Use API
The next goal is for Qt Quick 3D is to provide an API for defining 3D content, an API that is approachable and usable by developers without the need to understand the finer details of the modern graphics pipeline. After all, the majority of users do not need to create specialized 3D graphics renderers for each of their applications, but rather just want to show some 3D content, often alongside 2D. So we have been developing Qt Quick 3D with this perspective in mind.
That being said, we will be exposing more and more of the rendering API over time which will make more advanced use cases, needed by power-users, possible.
At the time of writing of this post we are only providing a QML API, but the goal in the future is to provide a public C++ API as well.
Unified Tooling for Qt Quick
Qt Quick 3D is intended to be the successor to Qt 3D Studio. For the time being Qt 3D Studio will still continue to be developed, but long-term will be replaced by Qt Quick and Qt Design Studio.
Here we intend to take the best parts of Qt 3D Studio and roll them into Qt Quick and Qt Design Studio. So rather than needing a separate tool for Qt Quick or 3D, it will be possible to just do both from within Qt Design Studio. We are working on the details of this now and hope to have a preview of this available soon.
For existing users of Qt 3D Studio, we have been working on a porting tool to convert projects to Qt Quick 3D. More on that later.
First Class Asset Conditioning Pipeline
When dealing with 3D scenes, asset conditioning becomes more important because now there are more types of assets being used, and they tend to be much bigger overall. So as part of the Qt Quick 3D development effort we have been looking at how we can make it as easy as possible to import your content and bake it into efficient runtime formats for Qt Quick.
For example, at design time you will want to specify the assets you are using based on what your asset creation tools generate (like FBX files from Maya for 3D models, or PSD files from Photoshop for textures), but at runtime you would not want the engine to use those formats. Instead, you will want to convert the assets into some efficient runtime format, and have them updated each time the source assets change. We want this to be an automated process as much as possible, and so want to build this into the build system and tooling of Qt.
Cross-platform Performance and Compatibility
Another of our goals is to support multiple native graphics APIs, using the new Rendering Hardware Interface being added to Qt. Currently, Qt Quick 3D only supports rendering using OpenGL, like many other components in Qt. However, in Qt 6 we will be using the QtRHI as our graphics abstraction and there we will be able to support rendering via Vulkan, Metal and Direct3D as well, in addition to OpenGL.
What is Qt Quick 3D? (and what it is not)
Qt Quick 3D is not a replacement for Qt 3D, but rather an extension of Qt Quick’s functionality to render 3D content using a high-level API.
Here is what a very simple project with some helpful comments looks like:
import QtQuick 2.12
import QtQuick.Window 2.12
import QtQuick3D 1.0
Window {
id: window
visible: true
width: 1280
height: 720
// Viewport for 3D content
View3D {
id: view
anchors.fill: parent
// Scene to view
Node {
id: scene
Light {
id: directionalLight
}
Camera {
id: camera
// It's important that your camera is not inside
// your model so move it back along the z axis
// The Camera is implicitly facing up the z axis,
// so we should be looking towards (0, 0, 0)
z: -600
}
Model {
id: cubeModel
// #Cube is one of the "built-in" primitive meshes
// Other Options are:
// #Cone, #Sphere, #Cylinder, #Rectangle
source: "#Cube"
// When using a Model, it is not enough to have a
// mesh source (ie "#Cube")
// You also need to define what material to shade
// the mesh with. A Model can be built up of
// multiple sub-meshes, so each mesh needs its own
// material. Materials are defined in an array,
// and order reflects which mesh to shade
// All of the default primitive meshes contain one
// sub-mesh, so you only need 1 material.
materials: [
DefaultMaterial {
// We are using the DefaultMaterial which
// dynamically generates a shader based on what
// properties are set. This means you don't
// need to write any shader code yourself.
// In this case we just want the cube to have
// a red diffuse color.
id: cubeMaterial
diffuseColor: "red"
}
]
}
}
}
}
The idea is that defining 3D content should be as easy as 2D. There are a few extra things you need, like the concepts of Lights, Cameras, and Materials, but all of these are high-level scene concepts, rather than implementation details of the graphics pipeline.
This simple API comes at the cost of less power, of course. While it may be possible to customize materials and the content of the scene, it is not possible to completely customize how the scene is rendered, unlike in Qt 3D via the its customizable framegraph. Instead, for now there is a fixed forward renderer, and you can define with properties in the scene how things are rendered. This is like other existing engines, which typically have a few possible rendering pipelines to choose from, and those then render the logical scene.
A Camera orbiting around a Car Model in a Skybox with Axis and Gridlines (note: stutter is from the 12 FPS GIF )
What Can You Do with Qt Quick 3D?
Well, it can do many things, but these are built up using the following scene primitives:
Node
Node is the base component for any node in the 3D scene. It represents a transformation in 3D space, and but is non-visual. It works similarly to how the Item type works in Qt Quick.
Camera
Camera represents how a scene is projected to a 2D surface. A camera has a position in 3D space (as it is a Node subclass) and a projection. To render a scene, you need to have at least one Camera.
Light
The Light component defines a source of lighting in the scene, at least for materials that consider lighting. Right now, there are 3 types of lights: Directional (default), Point and Area.
Model
The Model component is the one visual component in the scene. It represents a combination of geometry (from a mesh) and one or more materials.
The source property of the Mesh component expects a .mesh file, which is the runtime format used by Qt Quick 3D. To get mesh files, you need to convert 3D models using the asset import tool. There are also a few built-in primitives. These can be used by setting the following values to the source property: #Cube, #Cylinder, #Sphere, #Cone, or #Rectangle.
We will also be adding a programmatic way to define your own geometry at runtime, but that is not yet available in the preview.
Before a Model can be rendered, it must also have a Material. This defines how the mesh is shaded.
DefaultMaterial and Custom Materials
The DefaultMaterial component is an easy to use, built-in material. All you need to do is to create this material, set the properties you want to define, and under the hood all necessary shader code will be automatically generated for you. All the other properties you set on the scene are taken into consideration as well. There is no need to write any graphics shader code (such as, vertex or fragment shaders) yourself.
It is also possible to define so-called CustomMaterials, where you do provide your own shader code. We also provide a library of pre-defined CustomMaterials you can try out by just adding the following to your QML imports:
import QtQuick3D.MaterialLibrary 1.0Texture
The Texture component represents a texture in the 3D scene, as well as how it is mapped to a mesh. The source for a texture can either be an image file, or a QML Component.
A Sample of the Features Available
3D Views inside of Qt Quick
To view 3D content inside of Qt Quick, it is necessary to flatten it to a 2D surface. To do this, you use the View3D component. View3D is the only QQuickItem-based component in the whole API. You can either define the scene as a child of the View3D or reference an existing scene by setting the scene property to the root Node of the scene you want to renderer.
If you have more than one camera, you can also set which camera you want to use to render the scene. By default, it will just use the first active camera defined in the scene.
Also it is worth noting that View3D items do not necessarily need to be rendered to off-screen textures before being rendered. It is possible to set one of the 4 following render modes to define when the 3D content is rendered:
- Texture: View3D is a Qt Quick texture provider and renders content to an texture via an FBO
- Underlay: View3D is rendered before Qt Quick's 2D content is rendered, directly to the window (3D is always under 2D)
- Overlay: View3D is rendered after Qt Quick's 2D content is rendered, directly to the window (3D is always over 2D)
- RenderNode: View3D is rendered in-line with the Qt Quick 2D content. This can however lead to some quirks due to how Qt Quick 2D uses the depth buffer in Qt 5.
2D Views inside of 3D
It could be that you also want to render Qt Quick content inside of a 3D scene. To do so, anywhere where an Texture is taken as a property value (for example, in the diffuseMap property of default material), you can use a Texture with its sourceItem property set, instead of just specifying a file in the source property. This way the referenced Qt Quick item will be automatically rendered and used as a texture.
The diffuse color textures being mapped to the cubes are animated Qt Quick 2D items.
3D QML Components
Due to Qt Quick 3D being built on QML, it is possible to create reusable components for 3D as well. For example, if you create a Car model consisting of several Models, just save it to Car.qml. You can then instantiate multiple instance of Car by just reusing it, like any other QML type. This is very important because this way 2D and 3D scenes can be created using the same component model, instead of having to deal with different approaches for the 2D and 3D scenes.
Multiple Views of the Same Scene
Because scene definitions can exist anywhere in a Qt Quick project, its possible to reference them from multiple View3Ds. If you had multiple cameras in a scene, you could even render from each one to a different View3D.
4 views of the same Teapot scene. Also changing between 3 Cameras in the Perspective view.
Shadows
Any Light component can specify that it is casting shadows. When this is enabled, shadows are automatically rendered in the scene. Depending on what you are doing though, rendering shadows can be quite expensive, so you can fine-tune which Model components cast and receive shadows by setting additional properties on the Model.
Image Based Lighting
In addition to the standard Light components, its possible to light your scene by defining a HDRI map. This Texture can be set either for the whole View3D in its SceneEnvironment property, or on individual Materials.
Animations
Animations in Qt Quick 3D use the same animation system as Qt Quick. You can bind any property to an animator and it will be animated and updated as expected. Using the QtQuickTimeline module it is also possible to use keyframe-based animations.
Like the component model, this is another important step in reducing the gap between 2D and 3D scenes, as no separate, potentially conflicting animation systems are used here.
Currently there is no support for rigged animations, but that is planned in the future.
How Can You Try it Out?
The intention is to release Qt Quick 3D as a technical preview along with the release of Qt 5.14. In the meantime it should be possible to use it already now, against Qt 5.12 and higher.
To get the code, you just need to build the QtQuick3D module which is located here:
https://git.qt.io/annichol/qtquick3d
What About Tooling?
The goal is that it should be possible via Qt Design Studio to do everything you need to set up a 3D scene. That means being able to visually lay out the scene, import 3D assets like meshes, materials, and textures, and convert those assets into efficient runtime formats used by the engine.
A demonstration of early Qt Design Studio integration for Qt Quick 3D
Importing 3D Scenes to QML Components
Qt Quick 3D can also be used by writing QML code manually. Therefore, we also have some stand-alone utilities for converting assets. Once such tool is the balsam asset conditioning tool. Right now it is possible to feed this utility an asset from a 3D asset creation tool like Blender, Maya, or 3DS Max, and it will generate a QML component representing the scene, as well as any textures, meshes, and materials it uses. Currently this tool supports generating scenes from the following formats:
- FBX
- Collada (dae)
- OBJ
- Blender (blend)
- GLTF2
To convert the file myTestScene.fbx you would run:
./balsam -o ~/exportDirectory myTestScene.fbx
Thiswould generate a file called MyTestScene.qml together with any assets needed. Then you can just use it like any other Component in your scene:
import QtQuick 2.12
import QtQuick.Window 2.12
import QtQuick3D 1.0
Window {
width: 1920
height: 1080
visible: true
color: "black"
Node {
id: sceneRoot
Light {
}
Camera {
z: -100
}
MyTestScene {
}
}
View3D {
anchors.fill: parent
scene: sceneRoot
}
}
We are working to improve the assets generated by this tool, so expect improvements in the coming months.
Converting Qt 3D Studio Projects
In addition to being able to generate 3D QML components from 3D asset creation tools, we have also created a plugin for our asset import tool to convert existing Qt 3D Studio projects. If you have used Qt 3D Studio before, you will know it generates projects in XML format to define the scene. If you give the balsam tool a UIP or UIA project generated by Qt 3D Studio, it will also generate a Qt Quick 3D project based on that. Note however that since the runtime used by Qt 3D Studio is different from Qt Quick 3D, not everything will be converted. It should nonetheless give a good approximation or starting point for converting an existing project. We hope to continue improving support for this path to smooth the transition for existing Qt 3D Studio users.
Qt 3D Studio example application ported using Qt Quick 3D's import tool. (it's not perfect yet)
What About Qt 3D?
The first question I expect to get is why not just use Qt 3D? This is the same question we have been exploring the last couple of years.
One natural assumption is that we could just build all of Qt Quick on top of Qt 3D if we want to mix 2D and 3D. We intended to and started to do this with the 2.3 release of Qt 3D Studio. Qt 3D’s powerful API provided a good abstraction for implementing a rendering engine to re-create the behavior expected by Qt Quick and Qt 3D Studio. However, Qt 3D's architecture makes it difficult to get the performance we needed on an entry level embedded hardware. Qt 3D also comes with a certain overhead from its own limited runtime as well as from being yet another level of abstraction between Qt Quick and the graphics hardware. In its current form, Qt 3D is not ideal to build on if we want to reach a fully unified graphics story while ensuring continued good support for a wide variety of platforms and devices ranging from low to high end.
At the same time, we already had a rendering engine in Qt 3D Studio that did exactly what we needed, and was a good basis for building additional functionally. This comes with the downside that we no longer have the powerful APIs that come with Qt 3D, but in practice once you start building a runtime on top of Qt 3D, you already end up making decisions about how things should work, leading to a limited ability to customize the framegraph anyway. In the end the most practical decision was to use the existing Qt 3D Studio rendering engine as our base, and build off of that.
What is the Plan Moving Forward?
This release is just a preview of what is to come. The plan is to provide Qt Quick 3D as a fully supported module along with the Qt 5.15 LTS. In the meantime we are working on further developing Qt Quick 3D for release as a Tech Preview with Qt 5.14.
For the Qt 5 series we are limited in how deeply we can combine 2D and 3D because of binary compatibility promises. With the release of Qt 6 we are planning an even deeper integration of Qt Quick 3D into Qt Quick to provide an even smoother experience.
The goal here is that we want to be able to be as efficient as possible when mixing 2D and 3D content, without introducing any additional overhead to users who do not use any 3D content at all. We will not be doing anything drastic like forcing all Qt Quick apps to go through the new renderer, only ones who are mixing 2D and 3D.
In Qt 6 we will also be using the Qt Rendering Hardware Interface to render Qt Quick (including 3D) scenes which should eliminate many of the current issues we have today with deployment of OpenGL applications (by using DirectX on Windows, Metal on macOS, etc.).
We also want to make it possible for end users to use the C++ Rendering API we have created more generically, without Qt Quick. The code is there now as private API, but we are waiting until the Qt 6 time-frame (and the RHI porting) before we make the compatibility promises that come with public APIs.
Feedback is Very Welcome!
This is a tech preview, so much of what you see now is subject to change. For example, the API is a bit rough around the edges now, so we would like to know what we are missing, what doesn’t make sense, what works, and what doesn’t. The best way to provide this feedback is through the Qt Bug Tracker. Just remember to use the Qt Quick: 3D component when filing your bugs/suggestions.
