Sign up for the KDAB Newsletter
Stay on top of the latest news, publications, events and more.
Go to Sign-up
Continuing our blog post series about the rewrite of Qt3D.
One of the biggest driving factors behind the design of Qt3D 2.0 is the ability to configure the renderer in order to accommodate custom rendering techniques. In this blog post I will explain how to render a scene in Qt3D with shadows.
Shadow mapping in Qt3D. Note the self-shadowing of the plane and of the trefoil knot.
The complete working source code for this blog post is available in the Qt3D repository, under the examples/shadow_map_qml
directory. The entire rendering will be configured using QML (i.e. this is a QML-only example), but it's perfectly possible to also use C++ to achieve the very same result.
Shadows are not directly supported by OpenGL, and these days there are countless techniques that can be employed to generate them. Shadow mapping is one of the oldest; it's still widely used due to its simplicity and ability to generate good-looking shadows, while having a very small performance cost. The Wikipedia entry on shadow mapping has a very good overview of the modern incantations of this technique. However, for our purposes, we are going to stick to a very basic version of it.
Shadow mapping is typically implemented using a two pass rendering. In the first pass we generate the shadow information, and in the second pass we can render the scene "normally" (i.e., using any rendering technique of our choice), while at the same time using the information gathered in the first pass to draw the shadows.
The idea behind shadow mapping is the following: only the closest fragments to the light are the ones lit. Fragments "behind" other fragments are occluded, and therefore in shadow.
Therefore, in the first pass we draw the scene from the point of view of the light. The information that we then store is simply the distance of the closest fragment in this "light space". In OpenGL terms, this corresponds to having a Framebuffer Object, or FBO, with a depth texture attached to it. In fact, the "distance from the eye" is the definition of the depth; and the default depth testing done by OpenGL will actually store only the depth for the closest fragment.
(A color texture attachment is not even needed -- we don't need to shade fragments, only to calculate their depth.)
Exaggerated shadow map texture of the very same scene represented above.
The image above is the shadow map. That is, the depth stored when rendering the scene from the light point of view; darker colours represent a shallow depth (i.e. closer to the camera). In our scene, the light sits somewhere above the objects in the scene, on the right side w.r.t. the main camera (cf. the previous screenshot). This matches with the fact that the toyplane is closer to the camera than the other objects.
Once we have generated the shadow map, we then do the second rendering pass. In this second pass we render using the normal scene's camera; we can use any desired effect here, like for instance Phong shading. The important bit is that in the fragment shader we apply the shadow map algorithm, that is, we ask: is that fragment the closest fragment to the light? If so, then it must be drawn lit; otherwise, it must be drawn in shadow.
How to answer that question is easy once we have the shadow map generated in the first pass. All it suffices is to remap the fragment in light space, therefore calculating its depth from the light point of view, as well as where its coordinates are on the shadow map texture. We can then sample the shadow map texture at the given coordinates and compare the fragment's depth with the result of the sampling: if the fragment is further away, then it's in shadow, otherwise it is lit.
This is the theory about shadow mapping. Let's see how to turn this theory into code using Qt3D.
Let's start from the main.qml
file, where we set up the entire scene.
import Qt3D 2.0
import Qt3D.Render 2.0
Entity {
id: sceneRoot
Camera {
id: camera
projectionType: CameraLens.PerspectiveProjection
fieldOfView: 45
aspectRatio: _window.width / _window.height
nearPlane: 0.1
farPlane: 1000.0
position: Qt.vector3d(0.0, 10.0, 20.0)
viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
upVector: Qt.vector3d(0.0, 1.0, 0.0)
}
Configuration {
controlledCamera: camera
}
Light {
id: light
}
components: [
ShadowMapFrameGraph {
id: framegraph
viewCamera: camera
lightCamera: light.lightCamera
}
]
AdsEffect {
id: shadowMapEffect
shadowTexture: framegraph.shadowTexture
light: light
}
// Trefoil knot entity
Trefoil {
material: AdsMaterial {
effect: shadowMapEffect
specularColor: Qt.rgba(0.5, 0.5, 0.5, 1.0)
}
}
// Toyplane entity
Toyplane {
material: AdsMaterial {
effect: shadowMapEffect
diffuseColor: Qt.rgba(0.9, 0.5, 0.3, 1.0)
shininess: 75
}
}
// Plane entity
GroundPlane {
material: AdsMaterial {
effect: shadowMapEffect
diffuseColor: Qt.rgba(0.2, 0.5, 0.3, 1.0)
specularColor: Qt.rgba(0, 0, 0, 1.0)
}
}
}
The first components we create are a Camera
, which represents the camera used for the final rendering, and a Configuration
element which allows us to control this camera using the keyboard or the mouse. The parameters of the camera are self-explainatory and there isn't much to say about that.
We then create a Light
entity, which represents our light -- a directional spotlight, sitting somewhere above the plane, and looking down at the scene's origin. This light entity is then used by our custom frame graph, ShadowMapFrameGraph
, and our rendering effect, AdsEffect
, whose instances are created just after the light.
Lastly, we create three entities for the meshes in the scene: a trefoil knot, a toy aircraft, and a ground plane. The implementation of these three entities is straightforward and will not be covered here; they simply aggregate a mesh, a transformation and a material that uses the effect defined above. For extra fun, the toyplane and the trefoil knot transformations are actually animated.
The Light
element is defined inside Light.qml
:
import Qt3D 2.0
import Qt3D.Render 2.0
Entity {
id: root
property vector3d lightPosition: Qt.vector3d(30.0, 30.0, 0.0)
property vector3d lightIntensity: Qt.vector3d(1.0, 1.0, 1.0)
readonly property Camera lightCamera: lightCamera
readonly property matrix4x4 lightViewProjection: lightCamera.projectionMatrix.times(lightCamera.matrix)
Camera {
id: lightCamera
objectName: "lightCameraLens"
projectionType: CameraLens.PerspectiveProjection
fieldOfView: 45
aspectRatio: 1
nearPlane : 0.1
farPlane : 200.0
position: root.lightPosition
viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
upVector: Qt.vector3d(0.0, 1.0, 0.0)
}
}
As I said before, the light is a directional spotlight. Since in the first rendering pass we're going to use the light as a camera, I decided to actually put a Camera
sub-entity inside of it, and to expose it as a property. Apart from the camera, the light exposes as properties a position, its colour/intensity, and a 4x4 transformation matrix; we'll see where that matrix gets used, while the rest is straightforward.
In Qt3D 2.0 the frame graph is the data-driven configuration for the rendering. In this example, ShadowMapFrameGraph.qml
contains its implementation, which looks like this:
import Qt3D 2.0
import Qt3D.Render 2.0
import QtQuick 2.2 as QQ2
FrameGraph {
id: root
property alias viewCamera: viewCameraSelector.camera
property alias lightCamera: lightCameraSelector.camera
readonly property Texture2D shadowTexture: depthTexture
activeFrameGraph: Viewport {
rect: Qt.rect(0.0, 0.0, 1.0, 1.0)
clearColor: Qt.rgba(0.0, 0.4, 0.7, 1.0)
RenderPassFilter {
includes: [ Annotation { name: "pass"; value: "shadowmap" } ]
RenderTargetSelector {
target: RenderTarget {
attachments: [
RenderAttachment {
name: "depth"
type: RenderAttachment.DepthAttachment
texture: Texture2D {
id: depthTexture
width: 1024
height: 1024
format: Texture.DepthFormat
generateMipMaps: false
magnificationFilter: Texture.Linear
minificationFilter: Texture.Linear
wrapMode {
x: WrapMode.ClampToEdge
y: WrapMode.ClampToEdge
}
comparisonFunction: Texture.CompareLessEqual
comparisonMode: Texture.CompareRefToTexture
}
}
]
}
ClearBuffer {
buffers: ClearBuffer.DepthBuffer
CameraSelector {
id: lightCameraSelector
}
}
}
}
RenderPassFilter {
includes: [ Annotation { name: "pass"; value: "forward" } ]
ClearBuffer {
buffers: ClearBuffer.ColorDepthBuffer
CameraSelector {
id: viewCameraSelector
}
}
}
}
}
The code defines a FrameGraph
entity, which has a tree of entities as the active frame graph. Any path from the leaves of this tree to the root is a viable frame graph configuration; filter entities can enable or disable such paths, and selector entities can alter the configuration.
In our case, the tree looks like this:
So we have two paths from the topmost Viewport
entity. Each path corresponds to a pass of the shadow map technique; the paths are enabled and disabled using a RenderPassFilter
, an entity that can filter depending on arbitrary values defined in a given render pass (in our case: a string). The actual passes are not defined here, but in the effect (see below); the frame graph simply modifies its configuration when a given pass is rendered.
Now, in the shadow map generation pass, we must render to an offscreen surface (the FBO) which has a depth texture attachment: this in Qt3D is represented by the RenderTarget
entity, which has a number of attachments. In this case, only one attachment is needed: a depth attachment, defined by the RenderAttachment
entity using a type
of RenderAttachment.DepthAttachment
(stating it should store the depth), and a Texture2D
entity which actually configures the texture storage used to store the depth information.
Moreover, in this first pass, we must render using the light's camera; therefore, we have a CameraSelector
entity that sets the camera to the one exported by the Light
.
The second pass is instead way more straightforward, in which we simply render to the screen using the main camera.
The bulk of the magic happens in the AdsEffect.qml file, where our main Effect
entity is defined. As you can imagine from the name, it's an effect implementing the ADS shading model, i.e. Phong, with the addition of shadow mapped generated shadows.
An effect contains the implementation of a particular rendering strategy; in this case, shadow mapping using two passes.
import Qt3D 2.0
import Qt3D.Render 2.0
Effect {
id: root
property Texture2D shadowTexture
property Light light
parameters: [
Parameter { name: "lightViewProjection"; value: root.light.lightViewProjection },
Parameter { name: "lightPosition"; value: root.light.lightPosition },
Parameter { name: "lightIntensity"; value: root.light.lightIntensity },
Parameter { name: "shadowMapTexture"; value: root.shadowTexture }
]
techniques: [
Technique {
openGLFilter {
api: OpenGLFilter.Desktop
profile: OpenGLFilter.Core
majorVersion: 3
minorVersion: 2
}
renderPasses: [
RenderPass {
annotations: [ Annotation { name: "pass"; value: "shadowmap" } ]
shaderProgram: ShaderProgram {
vertexShaderCode: loadSource("qrc:/shaders/shadowmap.vert")
fragmentShaderCode: loadSource("qrc:/shaders/shadowmap.frag")
}
renderStates: [
PolygonOffset { factor: 4; units: 4 },
DepthTest { func: DepthTest.Less }
]
},
RenderPass {
annotations: [ Annotation { name : "pass"; value : "forward" } ]
bindings: [
// Uniforms (those provided by the user)
ParameterMapping { parameterName: "ambient"; shaderVariableName: "ka"; bindingType: ParameterMapping.Uniform },
ParameterMapping { parameterName: "diffuse"; shaderVariableName: "kd"; bindingType: ParameterMapping.Uniform },
ParameterMapping { parameterName: "specular"; shaderVariableName: "ks"; bindingType: ParameterMapping.Uniform }
]
shaderProgram: ShaderProgram {
vertexShaderCode: loadSource("qrc:/shaders/ads.vert")
fragmentShaderCode: loadSource("qrc:/shaders/ads.frag")
}
}
]
}
]
}
The parameters
list defines some default values for the effect. Those values will get mapped to OpenGL shader program uniforms, so that in the shaders we can access them. In this case, we expose some information from the Light
entity (its position, its intensity, its view/projection matrix defined by its internal camera), as well as the shadow map texture exposed by the frame graph.
In general, it's possible to put such parameters
all the way down, from a Material
, to its Effect
, to one of the effect's Technique
s. This allows a Material
instance to override defaults in an Effect
or Technique
. (The bindings
array provides the same thing, except that it also allows us to rename some parameters. In our case, it renames the ambient/diffuse/specular values defined in the material to the actual uniform names used by the shader programs.)
We then have a Technique
element. In order to be able to adapt the implementation to different hardware or OpenGL versions, an Effect
is implemented by providing one or more Technique
elements. In our case, only one technique is provided, targeting OpenGL 3.2 Core (or greater).
Inside that technique, we finally have the definition of our two rendering passes. We "tag" each pass with an Annotation
entity, matching the ones we've set into the frame graph configuration, so that each pass will have different rendering settings.
The first pass is the shadow map generation. To do so, we load a suitable set of GLSL shaders, which are actually extremely simple -- they do nothing except from MVP projection, to bring meshes from their model space into clip space (and, remember, in this first pass, the light is the camera). The fragment shader is totally empty: there's no color to be generated, and the depth will be automatically captured for us by OpenGL. Note that in this first pass, we also set some custom OpenGL state in the form of a polygon offset and depth testing mode.
The second pass is instead a normal forward rendering using Phong shading. The code in the effect entity is extremely simple: we simply configure some parameters (see above) and load a pair of shaders which will be used when drawing.
I will not explain the shader code in too much detail, because that would require a crash course in GLSL. However, I will explain the shadow mapping parts. The first part happens in the vertex shader (ads.vert
), where we output towards the fragment shader the coordinates of each vertex in light space:
positionInLightSpace = shadowMatrix * lightViewProjection
* modelMatrix * vec4(vertexPosition, 1.0);
(Actually, the coordinates get adjusted a little to allow us to easily sample the shadow map texture; that's the purpose of the shadowMatrix
, please refer to a book or to the Wikipedia entry on shadow mapping to understand why that's necessary).
The second part happens in the fragment shader (ads.frag
), where we sample the shadow map, and if the currently processed fragment is behind the one closest to the light, then the current fragment is in shadow (and only gets ambient contribution), otherwise it gets full Phong shading:
float shadowMapSample = textureProj(shadowMapTexture, positionInLightSpace);
vec3 ambient = lightIntensity * ka;
vec3 result = ambient;
if (shadowMapSample > 0)
result += dsModel(position, normalize(normal));
fragColor = vec4(result, 1.0);
And that's it!
In this post I've shown how it's possible to configure Qt3D in order to achieve a custom rendering effect. Although shadow mapping is one of the simplest rendering techniques, the point is demonstrating how Qt3D imposes no particular rendering algorithm or strategy. You can easily experiment with a variety of multipass effects, e.g. introduce stencil shadows, or maybe that effect you've just seen on that SIGGRAPH paper...
About KDAB
The KDAB Group is a globally recognized provider for software consulting, development and training, specializing in embedded devices and complex cross-platform desktop applications. In addition to being leading experts in Qt, C++ and 3D technologies for over two decades, KDAB provides deep expertise across the stack, including Linux, Rust and modern UI frameworks. With 100+ employees from 20 countries and offices in Sweden, Germany, USA, France and UK, we serve clients around the world.
Stay on top of the latest news, publications, events and more.
Go to Sign-up
Upgrade your applications from Qt 5 to Qt 6 with KDAB’s migration services. Get a free migration assessment and join a hands-on workshop to prepare your team for a successful transition!
Learn more
25 Comments
21 - Jan - 2015
Philip
Do you need Qt 5.5 to build Qt3D?
21 - Jan - 2015
Giuseppe D'Angelo
Hi,
yes, you need a checkout from the "dev" branch of Qt (which currently corresponds to 5.5).
You don't need to clone everything, only qtbase and qtdeclarative are really needed, plus optionally qimageformats (to be able to load DDS textures). If you're using the "qt5" repository (i.e. init-repository / git submodules), be sure to checkout "dev" in each submodule instead of in the top level repository! Full instructions are available here.
21 - Jan - 2015
Philip
Cool! I got it working. Is there any documentation that I could generate or look at?
21 - Jan - 2015
Giuseppe D'Angelo
Unfortunately not yet... but you're more than welcome to join the #qt-3d hannel on Freenode and ask around, there's usually always someone around (esp. in CET working hours).
22 - Jan - 2015
Philip
What are the default uniforms that are set in the shader programs? I see you can add uniforms through Effect's parameters property. For example modelMatrix, modelView, modelViewNormal, mvp are all defined in ads.vert but aren't referenced in the qml. I'm assuming these are default uniforms set by qt3d? I'm assuming there's others default uniforms like viewMatrix or projectionMatrix?
23 - Jan - 2015
Giuseppe D'Angelo
You're absolutely correct :) There's a set of builtin uniforms that are set by the engine.
Take a look into RenderView::initializeStandardUniformSetters where you can find the list, it's basically the various combinations of model/view/projection/viewport matrices, their inverses and the elapsed time in the application.
23 - Jan - 2015
Philip
Cool thanks.
24 - Jan - 2015
Sean Harmer
Please let us know if there's other standard uniforms that you would like to see added to the list. Best bet is to file a JIRA task.
26 - Jan - 2015
Philip
I need to try to implement a simple depth pealing algorithm to try test out the Framegraph / Effect / Entry system. I just haven't had time yet.
26 - Jan - 2015
Jim
I would be interested to see the C++ version of this. I plan on using Qt3d for image generation due to Qt's renown documentation. I do all of my programming in C++.
26 - Jan - 2015
Giuseppe D'Angelo
Hi Jim,
a C++ version of this should not be much different, it would essentially be a 1:1 translation of the scene into C++, therefore becoming a bit more verbose.
For comparison, take a look at the various examples that are shipped both in QML and C++ form (e.g. simple-qml / simple-cpp).
26 - Jan - 2015
jiangcaiyang
Hello, I got Qt 5.5 dev branch and compiled Qt statically. All goes well except all the example not linked. It shows:
Looking deep in the code, maybe it is probably due to the namespace of "Qt3D" mobule. I am not sure.
I am using Windows with mingw4.91
26 - Jan - 2015
jiangcaiyang
I notice that all examples requires "play-ground-qml" project, which is the not linked.
26 - Jan - 2015
Sean Harmer
They all require the simple exampleresources project. We'll need to look into this. Last time I tried a static build it was on Linux and the examples linked and executed there. For now, I'd suggest using a dynamic build until we can get this resolved. Patches welcome ;)
26 - Jan - 2015
jiangcaiyang
Is it because of the the export keyword? I noticed that Qt3D Core defines only shared export, not static export:
26 - Jan - 2015
Sean Harmer
If it's a static build there's no need to export anything. That export keyword is actually just a macro for Q_DECL_EXPORT/Q_DECL_IMPORT which in turn are macros for the platform specific export/visibility compiler intrinsics. In a static build everything is in the same executable.
26 - Jan - 2015
jiangcaiyang
In addition, the original exampleresources.pri got not compiled. it is suggested to add "lib" prefix at line 6 and 8 before "exampleresources". the final looks like this:
26 - Jan - 2015
Sean Harmer
Ah looks like a MSVC vs mingw issue in the .pro and .pri files then.
27 - Jan - 2015
jiangcaiyang
Problem solved.
Maybe all the #ifdef directives have to be changed. Taken from another module of Qt, Enginio have long been unsuccessfully linked. I've fired quite a lot post to the developers. Now it is Okay with static build. Looking deep in code, it is written like this:
I've changed all the directives in Qt 3D to look similar to this, and get compiled and linked. I suggest change all the directives to adapt to static build in MinGW. or make a bunch of new directives such as "QT3D_STATIC_EXPORT".
27 - Jan - 2015
Sean Harmer
Thanks, peppe has pushed a patch with this fix in. Should land once gerrit is fixed.
5 - Sept - 2015
Marco
Hi, I have problems running Qt3D shadow map example. After starting the application I can only see a black screen in the app window. This happens with all multi-pass rendering examples(multiview,...). I have tried to run the same example(s) on another PC with other hardware and windows version. Beside Qt3D I have tried many other multi-pass rendering methods. These all run fine. I hope you can give me fast a useful hint. Thanks in advance Marco
7 - Sept - 2015
Giuseppe D'Angelo
Hi Marco, I think the example was accidentally broken, but it should now work in the 5.6 branch.
24 - Nov - 2015
Dominik
hey there,
something im wondering about is: I load a mesh from a .obj file next i want to set a center/origin for this object.
is this possible? for example Transform{ id: .... Rotate{id: ...; origin/center: Qt.vector3d(x, y, z} }
cheers
30 - Nov - 2015
Giuseppe D'Angelo
Hi Dominik,
There are now actually helpers for "rotate around a point" kind of scenarios. Fetch the code from the 5.6 branch and take a look for the QTransform::rotateAround helper (also exposed in QML).
18 - Apr - 2018
Carlos Ranoya
Hi Giuseppe. Congratulations for this article. Very enlightening.
I've been studying this Qt example and found your post. I have one doubt: is it possible to use the Framebuffer Object - the rendered shadow map texture, in this case - as texture source of any QML component instanciated at the root window, above Scene3D?
If so, how to do that? Thanks for any kind of help.