How to integrate OpenGL code with Qt Quick 2 applications (part 2)

In the last blog post we gave a very high level introduction to the Qt Quick 2 renderer. We also showed how various signals are emitted by the renderer during the synchronization and rendering steps. In this blog post we’re going to discuss those signals and show how they can be used to implement overlays and underlays.

Signals emitted by the Qt Quick renderer

Let’s start again by looking at this diagram of the synchronization between the render thread of the renderer and the main thread when a new frame needs to be rendered:


What are exactly the signals emitted by the Qt Quick 2 renderer that we can use? Some of them are visible in the picture above, some of them are not, so I will list all of them here:

  • QQuickWindow::sceneGraphInitialized: emitted when the renderer initializes the OpenGL context and the scene graph resources.
  • QQuickWindow::sceneGraphInvalidated: emitted when the renderer has invalidated the scene graph, and the OpenGL resources should be freed.
  • QQuickWindow::beforeSynchronizing: emitted before the scene graph synchronizes with the main thread. However, the main thread is already blocked at this point.
  • QQuickWindow::afterSynchronizing: emitted after the scene graph synchronized with the main thread. The main thread is still blocked at this point.
  • QQuickWindow::beforeRendering: emitted before any rendering by the Qt Quick renderer has taken place.
  • QQuickWindow::afterRendering: emitted after the Qt Quick renderer has rendered the scene graph.
  • QQuickWindow::frameSwapped: emitted after the Qt Quick renderer has swapped the back and the front buffers.

That is a lot of signals indeed! But it’s extremely good news, because it means we can intervene at any point of the synchronization/render process by simply connecting slots to these signals. An important consideration is that the OpenGL context used by the Qt Quick renderer will be bound when the signals are emitted. This means we can simply have OpenGL calls in our slots, without the problem of finding the “right” OpenGL context to use and make it current.

Also, note that the signals are emitted from the render thread, not the main thread. This typically means that we will need to force the connection type to Qt::DirectConnection, if the receiver object lives in the main thread. It also means we will need to carefully manage shared state between the main thread and the GUI thread, for instance by protecting the state with mutexes.

Remember that after the synchronization is complete, the main thread is unblocked and free to go while the rendering signals are being emitted; we can easily have races between the main thread and slots invoked directly from the render thread.

OpenGL underlays and overlays

So how can we implement, for instance, a simple underlay (that is, drawing custom OpenGL content below a scene)? Here’s a screenshot of an example application:


The trefoil knot is rendered by an external piece of OpenGL code that we are not going to discuss here. Let’s assume is some existing code we don’t want to touch, and treat it as a black box. The small controls on the bottom are instead implemented using Qt Quick. As you can see, the controls are truly over the OpenGL content, and nicely blended with it.

The three sliders control the position of the virtual camera in the world (in spherical coordinates). For simplicity, these coordinates are wrapped by a Camera class, an instance of which gets exposed to the QML engine.

How can we obtain this result using the signals emitted by Qt Quick? Here you are the relevant parts of a possible implementation, done using a subclass of QQuickView.

Let’s start by looking at the constructor:

QuickView::QuickView(QWindow *parent)
    : QQuickView(parent)
    , m_camera(new Camera(this))
    , m_renderer(new MeshRenderer(this))
    connect(this, &QQuickWindow::sceneGraphInitialized,
            this, &QuickView::initializeUnderlay,

    connect(this, &QQuickWindow::beforeSynchronizing,
            this, &QuickView::synchronizeUnderlay,

    connect(this, &QQuickWindow::beforeRendering,
            this, &QuickView::renderUnderlay,

    connect(this, &QQuickWindow::sceneGraphInvalidated,
            this, &QuickView::invalidateUnderlay,

    connect(m_camera, &Camera::azimuthChanged, this, &QQuickWindow::update);
    connect(m_camera, &Camera::elevationChanged, this, &QQuickWindow::update);
    connect(m_camera, &Camera::distanceChanged, this, &QQuickWindow::update);


    rootContext()->setContextProperty("_camera", m_camera);

In the constructor a Camera object gets created, representing the position of our virtual camera, and a MeshRenderer object, which is our black box performing OpenGL rendering (drawing the knot).

We then connect some of the signals emitted by the Qt Quick renderer to some local slots. We’re not interested in all of them; for this example just want to know:

  • when the scene graph is getting created and destroyed, so we can initialize and release the OpenGL resources;
  • when the synchronization round is happening, so we can safely access state in the main thread (which is blocked);
  • when Qt Quick is about to start rendering the next frame, so we can paint before it, therefore realizing an underlay.

Notice the Qt::DirectConnection connection type. Since the receiver (this) is living in the main thread, and the signal is emitted by the render thread, we must force the connection type otherwise it would result in a queued invocation, with disastrous results.

After those connections, we connect the update slot to the various changed signals from the Camera object, because every time the camera moves we want to schedule a redraw.

We then ask Qt Quick not to issue a glClear on the color buffer before starting its own rendering. Since we want to draw something before Qt Quick, clearing would result in not seeing anything of what we had rendered. The setClearBeforeRendering call disables (or enables) the automatic clearing.

Finally, we expose the Camera object to QML (so that it can be manipulated through the Qt Quick sliders) and load a QML file.

What about the slots themselves? Let’s have a look:

void QuickView::initializeUnderlay()

void QuickView::invalidateUnderlay()

void QuickView::renderUnderlay()

void QuickView::synchronizeUnderlay()

Yes, it’s that simple! Since our renderer is a black box, we simply call into it from the slots.

When the scene graph is initialized or invalidated, we respectively initialize or invalidate our renderer.

When Qt Quick is about to render itself, we call our own rendering code, resulting in drawing below Qt Quick.

A bit more interesting is the synchronization slot. Remember that all these slots are called directly from the render thread; as such, concurrent access from the main thread could easily result in a data race. While we could expose the Camera object to our custom OpenGL code, we would then need to implement a thread-safe Camera, which may be tricky or just impossible.

For this reason, we’ve decided to follow what the Qt Quick renderer does: we extract from the objects living in the main thread all the information required for rendering the frame. In this case, the object living in the main thread is m_camera, and the information are the spherical coordinates, which get stored somewhere else (for instance inside our black box renderer). All of this happens during the synchronization stage, while the main thread is blocked, so there is no risk of concurrent access and therefore data races.

Finally, what is the deal with all the resetOpenGLState calls? The fact is that the Qt Quick renderer does not like modifications to the OpenGL state. In other words, if in our slots we modify some OpenGL state (bind textures, change the used shader program, and so on), we must reset the OpenGL state to whatever it was before we touched it.

While this is possible in theory (OpenGL has all the necessary APIs for querying state), it’s also terribly complicated due to the countless settings that OpenGL has; moreover, if we simply call into some third party code, we have no control over which state gets touched by that. For this reason, there is a convenience call that resets the OpenGL state: QQuickWindow::resetOpenGLState. Be sure to call this function every time some OpenGL state gets modified in a slot!

That’s it!

Further steps

In the next blog post we are going to deeply integrate a custom OpenGL item inside a Qt Quick 2 scene. Stay tuned!

About KDAB

KDAB is a consulting company dedicated to Qt and offering a wide variety of services and providing training courses in:

KDAB believes that it is critical for our business to invest into Qt3D and Qt, in general, to keep pushing the technology forward and to ensure it remains competitive.

Share on FacebookTweet about this on TwitterShare on Google+

7 thoughts on “How to integrate OpenGL code with Qt Quick 2 applications (part 2)

  1. Very nice tutorial. I am evaluating this approach for simplifying the development of a desktop application with a good base of pre-existent OpenGL code for visualizing some custom data structures. The only big question marks I have at the moment are:

    (1) Data managment. Assuming that the application many custom 3D data structures to be processed and visualized, where should I store the data? How should I manage updating (or exporting) data to the renderer. In the demo everything is loaded directly by the renderer;

    (2) Mouse and keyboard input. Am I able to send quick and precise mouse events to the renderer to implement, for example, a very custom mouse-keyboard interaction with the 3D models. How would I implement such a thing? How it will perform?

    Thank you again!

    • Hi Marco,

      Regarding (1) there are many possible approaches, depending on the nature of your renderer. For instance, you could have your renderer created and destroyed on demand following the corresponding sceneGraph* signals emitted by QQuickWindow. The data structures in theory should follow the renderer, as when the scene graph context gets destroyed, you will lose your OpenGL objects (buffers, textures, etc.), so they would need to be created again. Now, again, you may want to create “frontend” elements to be manipulated in the GUI thread, and corresponding “backend” elements to be manipulated in the render thread, mimicking what Qt Quick does — so that the renderer has the low-level stuff to just render, while the high level objects stay and live in the main thread. It’s really up to you.

      (Note that if this multithreading becomes a complication, a quick’n’dirty workaround is disable it (by exporting QSG_RENDER_LOOP and setting it to basic) — this basically forces Qt Quick to run single threaded.)

      In some scenarios you may want more control over the OpenGL context creation, threading, and exactly about when rendering should happen. For those scenarios, stay tuned — that’s the subject of the next instalments of these series, involving QQuickRenderControl.

      Regarding (2) there should be no problems at all, just subclass QQuickWindow or QQuickView and override the event handlers — or, catch mouse/keyboard events inside the Qt Quick scene and handle them appropriately.

      For an underlay you can use an approach like:

      class MyQuickView : public QQuickView
          void mousePressEvent(QMouseEvent *e) override {
              QQuickView::mousePressEvent(e); // let Qt Quick handle it first...
              if (!e->isAccepted()) // the click didn't land on anything in Qt Quick, I'll use it then

      Hope this helps,

  2. I think the next blog post isn’t ready yet, but could I have some pointers about creating an item that renders itself with custom opengl inside a QtQuick scene? I need to do it really badly because I need to render something using a MODULATE blending mode.

      • Wow, thanks for such a a quick response.

        I don’t see how it would help me. I want to execute custom code alongside the QSG renderer, to allow usage of blending modes other than AlphaOver. I just want to render acolored rect using the MODULATE blending mode over my existing scene.

        • Hi,

          sorry — I thought you were asking how to render an item with custom GL code.

          To allow blending with the rest of the scene, the only thing that comes to mind is using a custom material node / shader (see QSGMaterialShader). In its activate() function you could set a custom blending function (and remove it in deactivate()). But what do you mean exactly with MODULATE? Do you refer to a specific blend equation / function?

          • Oh thank you so much, that is just what I needed! That allows me to set the GL scene blending mode manually.

            By modulate I meant multiplying the source fragment color with the destination color already in the target. But now I realized that GL_MODULATE only makes sense in old-school multitexturing. It is a bummer that blending isn’t yet programmable…

            I eagerly await the sequel to this post. Cheers!

Leave a Reply

Your email address will not be published. Required fields are marked *