Shadow Mapping in Qt3D 2.0

19.01.2015 Giuseppe D'Angelo 25 comments

Continuing our blog post series about the rewrite of Qt3D.

One of the biggest driving factors behind the design of Qt3D 2.0 is the ability to configure the renderer in order to accommodate custom rendering techniques. In this blog post I will explain how to render a scene in Qt3D with shadows.

Shadow mapping in Qt3D. Note the self-shadowing of the plane and of the trefoil knot.

The complete working source code for this blog post is available in the Qt3D repository, under the examples/shadow_map_qml directory. The entire rendering will be configured using QML (i.e. this is a QML-only example), but it’s perfectly possible to also use C++ to achieve the very same result.

Shadow mapping

Shadows are not directly supported by OpenGL, and these days there are countless techniques that can be employed to generate them. Shadow mapping is one of the oldest; it’s still widely used due to its simplicity and ability to generate good-looking shadows, while having a very small performance cost. The Wikipedia entry on shadow mapping has a very good overview of the modern incantations of this technique. However, for our purposes, we are going to stick to a very basic version of it.

Shadow mapping is typically implemented using a two pass rendering. In the first pass we generate the shadow information, and in the second pass we can render the scene “normally” (i.e., using any rendering technique of our choice), while at the same time using the information gathered in the first pass to draw the shadows.

The idea behind shadow mapping is the following: only the closest fragments to the light are the ones lit. Fragments “behind” other fragments are occluded, and therefore in shadow.

Therefore, in the first pass we draw the scene from the point of view of the light. The information that we then store is simply the distance of the closest fragment in this “light space”. In OpenGL terms, this corresponds to having a Framebuffer Object, or FBO, with a depth texture attached to it. In fact, the “distance from the eye” is the definition of the depth; and the default depth testing done by OpenGL will actually store only the depth for the closest fragment.

(A color texture attachment is not even needed — we don’t need to shade fragments, only to calculate their depth.)

Exaggerated shadow map texture of the very same scene represented above.

The image above is the shadow map. That is, the depth stored when rendering the scene from the light point of view; darker colours represent a shallow depth (i.e. closer to the camera). In our scene, the light sits somewhere above the objects in the scene, on the right side w.r.t. the main camera (cf. the previous screenshot). This matches with the fact that the toyplane is closer to the camera than the other objects.

Once we have generated the shadow map, we then do the second rendering pass. In this second pass we render using the normal scene’s camera; we can use any desired effect here, like for instance Phong shading. The important bit is that in the fragment shader we apply the shadow map algorithm, that is, we ask: is that fragment the closest fragment to the light? If so, then it must be drawn lit; otherwise, it must be drawn in shadow.

How to answer that question is easy once we have the shadow map generated in the first pass. All it suffices is to remap the fragment in light space, therefore calculating its depth from the light point of view, as well as where its coordinates are on the shadow map texture. We can then sample the shadow map texture at the given coordinates and compare the fragment’s depth with the result of the sampling: if the fragment is further away, then it’s in shadow, otherwise it is lit.

This is the theory about shadow mapping. Let’s see how to turn this theory into code using Qt3D.

Getting started

Let’s start from the main.qml file, where we set up the entire scene.

import Qt3D 2.0
import Qt3D.Render 2.0

Entity {
    id: sceneRoot

    Camera {
        id: camera
        projectionType: CameraLens.PerspectiveProjection
        fieldOfView: 45
        aspectRatio: _window.width / _window.height
        nearPlane: 0.1
        farPlane: 1000.0
        position: Qt.vector3d(0.0, 10.0, 20.0)
        viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
        upVector: Qt.vector3d(0.0, 1.0, 0.0)
    }

    Configuration  {
        controlledCamera: camera
    }

    Light {
        id: light
    }

    components: [
        ShadowMapFrameGraph {
            id: framegraph
            viewCamera: camera
            lightCamera: light.lightCamera
        }
    ]

    AdsEffect {
        id: shadowMapEffect

        shadowTexture: framegraph.shadowTexture
        light: light
    }

    // Trefoil knot entity
    Trefoil {
        material: AdsMaterial {
            effect: shadowMapEffect
            specularColor: Qt.rgba(0.5, 0.5, 0.5, 1.0)
        }
    }

    // Toyplane entity
    Toyplane {
        material: AdsMaterial {
            effect: shadowMapEffect
            diffuseColor: Qt.rgba(0.9, 0.5, 0.3, 1.0)
            shininess: 75
        }
    }

    // Plane entity
    GroundPlane {
        material: AdsMaterial {
            effect: shadowMapEffect
            diffuseColor: Qt.rgba(0.2, 0.5, 0.3, 1.0)
            specularColor: Qt.rgba(0, 0, 0, 1.0)
        }
    }
}

The first components we create are a Camera, which represents the camera used for the final rendering, and a Configuration element which allows us to control this camera using the keyboard or the mouse. The parameters of the camera are self-explainatory and there isn’t much to say about that.

We then create a Light entity, which represents our light — a directional spotlight, sitting somewhere above the plane, and looking down at the scene’s origin. This light entity is then used by our custom frame graph, ShadowMapFrameGraph, and our rendering effect, AdsEffect, whose instances are created just after the light.

Lastly, we create three entities for the meshes in the scene: a trefoil knot, a toy aircraft, and a ground plane. The implementation of these three entities is straightforward and will not be covered here; they simply aggregate a mesh, a transformation and a material that uses the effect defined above. Please refer to the previous blog posts for more information about these. For extra fun, the toyplane and the trefoil knot transformations are actually animated.

Light

The Light element is defined inside Light.qml:

import Qt3D 2.0
import Qt3D.Render 2.0

Entity {
    id: root

    property vector3d lightPosition: Qt.vector3d(30.0, 30.0, 0.0)
    property vector3d lightIntensity: Qt.vector3d(1.0, 1.0, 1.0)

    readonly property Camera lightCamera: lightCamera
    readonly property matrix4x4 lightViewProjection: lightCamera.projectionMatrix.times(lightCamera.matrix)

    Camera {
        id: lightCamera
        objectName: "lightCameraLens"
        projectionType: CameraLens.PerspectiveProjection
        fieldOfView: 45
        aspectRatio: 1
        nearPlane : 0.1
        farPlane : 200.0
        position: root.lightPosition
        viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
        upVector: Qt.vector3d(0.0, 1.0, 0.0)
    }
}

As I said before, the light is a directional spotlight. Since in the first rendering pass we’re going to use the light as a camera, I decided to actually put a Camera sub-entity inside of it, and to expose it as a property. Apart from the camera, the light exposes as properties a position, its colour/intensity, and a 4×4 transformation matrix; we’ll see where that matrix gets used, while the rest is straightforward.

Frame graph

In Qt3D 2.0 the frame graph is the data-driven configuration for the rendering. In this example, ShadowMapFrameGraph.qml contains its implementation, which looks like this:

import Qt3D 2.0
import Qt3D.Render 2.0
import QtQuick 2.2 as QQ2

FrameGraph {
    id: root

    property alias viewCamera: viewCameraSelector.camera
    property alias lightCamera: lightCameraSelector.camera
    readonly property Texture2D shadowTexture: depthTexture

    activeFrameGraph: Viewport {
        rect: Qt.rect(0.0, 0.0, 1.0, 1.0)
        clearColor: Qt.rgba(0.0, 0.4, 0.7, 1.0)

        RenderPassFilter {
            includes: [ Annotation { name: "pass"; value: "shadowmap" } ]

            RenderTargetSelector {
                target: RenderTarget {
                    attachments: [
                        RenderAttachment {
                            name: "depth"
                            type: RenderAttachment.DepthAttachment
                            texture: Texture2D {
                                id: depthTexture
                                width: 1024
                                height: 1024
                                format: Texture.DepthFormat
                                generateMipMaps: false
                                magnificationFilter: Texture.Linear
                                minificationFilter: Texture.Linear
                                wrapMode {
                                    x: WrapMode.ClampToEdge
                                    y: WrapMode.ClampToEdge
                                }
                                comparisonFunction: Texture.CompareLessEqual
                                comparisonMode: Texture.CompareRefToTexture
                            }
                        }
                    ]
                }

                ClearBuffer {
                    buffers: ClearBuffer.DepthBuffer

                    CameraSelector {
                        id: lightCameraSelector
                    }
                }
            }
        }

        RenderPassFilter {
            includes: [ Annotation { name: "pass"; value: "forward" } ]

            ClearBuffer {
                buffers: ClearBuffer.ColorDepthBuffer

                CameraSelector {
                    id: viewCameraSelector
                }
            }
        }
    }
}

The code defines a FrameGraph entity, which has a tree of entities as the active frame graph. Any path from the leaves of this tree to the root is a viable frame graph configuration; filter entities can enable or disable such paths, and selector entities can alter the configuration.

In our case, the tree looks like this:

• Viewport
  • RenderPassFilter
    • RenderTargetSelector
      • ClearBuffer
        • CameraSelector
  • RenderPassFilter
    • ClearBuffer
      • CameraSelector

So we have two paths from the topmost Viewport entity. Each path corresponds to a pass of the shadow map technique; the paths are enabled and disabled using a RenderPassFilter, an entity that can filter depending on arbitrary values defined in a given render pass (in our case: a string). The actual passes are not defined here, but in the effect (see below); the frame graph simply modifies its configuration when a given pass is rendered.

Now, in the shadow map generation pass, we must render to an offscreen surface (the FBO) which has a depth texture attachment: this in Qt3D is represented by the RenderTarget entity, which has a number of attachments. In this case, only one attachment is needed: a depth attachment, defined by the RenderAttachment entity using a type of RenderAttachment.DepthAttachment (stating it should store the depth), and a Texture2D entity which actually configures the texture storage used to store the depth information.

Moreover, in this first pass, we must render using the light’s camera; therefore, we have a CameraSelector entity that sets the camera to the one exported by the Light.

The second pass is instead way more straightforward, in which we simply render to the screen using the main camera.

The effect

The bulk of the magic happens in the AdsEffect.qml file, where our main Effect entity is defined. As you can imagine from the name, it’s an effect implementing the ADS shading model, i.e. Phong, with the addition of shadow mapped generated shadows.

An effect contains the implementation of a particular rendering strategy; in this case, shadow mapping using two passes.

import Qt3D 2.0
import Qt3D.Render 2.0

Effect {
    id: root

    property Texture2D shadowTexture
    property Light light

    parameters: [
        Parameter { name: "lightViewProjection"; value: root.light.lightViewProjection },
        Parameter { name: "lightPosition";  value: root.light.lightPosition },
        Parameter { name: "lightIntensity"; value: root.light.lightIntensity },

        Parameter { name: "shadowMapTexture"; value: root.shadowTexture }
    ]

    techniques: [
        Technique {
            openGLFilter {
                api: OpenGLFilter.Desktop
                profile: OpenGLFilter.Core
                majorVersion: 3
                minorVersion: 2
            }

            renderPasses: [
                RenderPass {
                    annotations: [ Annotation { name: "pass"; value: "shadowmap" } ]

                    shaderProgram: ShaderProgram {
                        vertexShaderCode:   loadSource("qrc:/shaders/shadowmap.vert")
                        fragmentShaderCode: loadSource("qrc:/shaders/shadowmap.frag")
                    }

                    renderStates: [
                        PolygonOffset { factor: 4; units: 4 },
                        DepthTest { func: DepthTest.Less }
                    ]
                },

                RenderPass {
                    annotations: [ Annotation { name : "pass"; value : "forward" } ]

                    bindings: [
                        // Uniforms (those provided by the user)
                        ParameterMapping { parameterName: "ambient";  shaderVariableName: "ka"; bindingType: ParameterMapping.Uniform },
                        ParameterMapping { parameterName: "diffuse";  shaderVariableName: "kd"; bindingType: ParameterMapping.Uniform },
                        ParameterMapping { parameterName: "specular"; shaderVariableName: "ks"; bindingType: ParameterMapping.Uniform }
                    ]

                    shaderProgram: ShaderProgram {
                        vertexShaderCode:   loadSource("qrc:/shaders/ads.vert")
                        fragmentShaderCode: loadSource("qrc:/shaders/ads.frag")
                    }
                }
            ]
        }
    ]
}

The parameters list defines some default values for the effect. Those values will get mapped to OpenGL shader program uniforms, so that in the shaders we can access them. In this case, we expose some information from the Light entity (its position, its intensity, its view/projection matrix defined by its internal camera), as well as the shadow map texture exposed by the frame graph.

In general, it’s possible to put such parameters all the way down, from a Material, to its Effect, to one of the effect’s Techniques. This allows a Material instance to override defaults in an Effect or Technique. (The bindings array provides the same thing, except that it also allows us to rename some parameters. In our case, it renames the ambient/diffuse/specular values defined in the material to the actual uniform names used by the shader programs.)

We then have a Technique element. In order to be able to adapt the implementation to different hardware or OpenGL versions, an Effect is implemented by providing one or more Technique elements. In our case, only one technique is provided, targeting OpenGL 3.2 Core (or greater).

Inside that technique, we finally have the definition of our two rendering passes. We “tag” each pass with an Annotation entity, matching the ones we’ve set into the frame graph configuration, so that each pass will have different rendering settings.

The first pass is the shadow map generation. To do so, we load a suitable set of GLSL shaders, which are actually extremely simple — they do nothing except from MVP projection, to bring meshes from their model space into clip space (and, remember, in this first pass, the light is the camera). The fragment shader is totally empty: there’s no color to be generated, and the depth will be automatically captured for us by OpenGL. Note that in this first pass, we also set some custom OpenGL state in the form of a polygon offset and depth testing mode.

The second pass is instead a normal forward rendering using Phong shading. The code in the effect entity is extremely simple: we simply configure some parameters (see above) and load a pair of shaders which will be used when drawing.

The shaders

I will not explain the shader code in too much detail, because that would require a crash course in GLSL. However, I will explain the shadow mapping parts. The first part happens in the vertex shader (ads.vert), where we output towards the fragment shader the coordinates of each vertex in light space:

    positionInLightSpace = shadowMatrix * lightViewProjection
        * modelMatrix * vec4(vertexPosition, 1.0);

(Actually, the coordinates get adjusted a little to allow us to easily sample the shadow map texture; that’s the purpose of the shadowMatrix, please refer to a book or to the Wikipedia entry on shadow mapping to understand why that’s necessary).

The second part happens in the fragment shader (ads.frag), where we sample the shadow map, and if the currently processed fragment is behind the one closest to the light, then the current fragment is in shadow (and only gets ambient contribution), otherwise it gets full Phong shading:

    float shadowMapSample = textureProj(shadowMapTexture, positionInLightSpace);

    vec3 ambient = lightIntensity * ka;
    vec3 result = ambient;

    if (shadowMapSample > 0)
        result += dsModel(position, normalize(normal));

    fragColor = vec4(result, 1.0);

And that’s it!

Conclusions

In this post I’ve shown how it’s possible to configure Qt3D in order to achieve a custom rendering effect. Although shadow mapping is one of the simplest rendering techniques, the point is demonstrating how Qt3D imposes no particular rendering algorithm or strategy. You can easily experiment with a variety of multipass effects, e.g. introduce stencil shadows, or maybe that effect you’ve just seen on that SIGGRAPH paper…