Skip to content

How to integrate OpenGL code with Qt Quick 2 applications (part 2)

In the last blog post we gave a very high level introduction to the Qt Quick 2 renderer. We also showed how various signals are emitted by the renderer during the synchronization and rendering steps. In this blog post we’re going to discuss those signals and show how they can be used to implement overlays and underlays.

Signals emitted by the Qt Quick renderer

Let’s start again by looking at this diagram of the synchronization between the render thread of the renderer and the main thread when a new frame needs to be rendered:

sg-renderloop-threaded

What are exactly the signals emitted by the Qt Quick 2 renderer that we can use? Some of them are visible in the picture above, some of them are not, so I will list all of them here:

  • QQuickWindow::sceneGraphInitialized: emitted when the renderer initializes the OpenGL context and the scene graph resources.
  • QQuickWindow::sceneGraphInvalidated: emitted when the renderer has invalidated the scene graph, and the OpenGL resources should be freed.
  • QQuickWindow::beforeSynchronizing: emitted before the scene graph synchronizes with the main thread. However, the main thread is already blocked at this point.
  • QQuickWindow::afterSynchronizing: emitted after the scene graph synchronized with the main thread. The main thread is still blocked at this point.
  • QQuickWindow::beforeRendering: emitted before any rendering by the Qt Quick renderer has taken place.
  • QQuickWindow::afterRendering: emitted after the Qt Quick renderer has rendered the scene graph.
  • QQuickWindow::frameSwapped: emitted after the Qt Quick renderer has swapped the back and the front buffers.

That is a lot of signals indeed! But it’s extremely good news, because it means we can intervene at any point of the synchronization/render process by simply connecting slots to these signals. An important consideration is that the OpenGL context used by the Qt Quick renderer will be bound when the signals are emitted. This means we can simply have OpenGL calls in our slots, without the problem of finding the “right” OpenGL context to use and make it current.

Also, note that the signals are emitted from the render thread, not the main thread. This typically means that we will need to force the connection type to Qt::DirectConnection, if the receiver object lives in the main thread. It also means we will need to carefully manage shared state between the main thread and the GUI thread, for instance by protecting the state with mutexes.

Remember that after the synchronization is complete, the main thread is unblocked and free to go while the rendering signals are being emitted; we can easily have races between the main thread and slots invoked directly from the render thread.

OpenGL underlays and overlays

So how can we implement, for instance, a simple underlay (that is, drawing custom OpenGL content below a scene)? Here’s a screenshot of an example application:

opengl-underlay-qq2

The trefoil knot is rendered by an external piece of OpenGL code that we are not going to discuss here. Let’s assume is some existing code we don’t want to touch, and treat it as a black box. The small controls on the bottom are instead implemented using Qt Quick. As you can see, the controls are truly over the OpenGL content, and nicely blended with it.

The three sliders control the position of the virtual camera in the world (in spherical coordinates). For simplicity, these coordinates are wrapped by a Camera class, an instance of which gets exposed to the QML engine.

How can we obtain this result using the signals emitted by Qt Quick? Here you are the relevant parts of a possible implementation, done using a subclass of QQuickView.

Let’s start by looking at the constructor:

QuickView::QuickView(QWindow *parent)
    : QQuickView(parent)
    , m_camera(new Camera(this))
    , m_renderer(new MeshRenderer(this))
{
    connect(this, &QQuickWindow::sceneGraphInitialized,
            this, &QuickView::initializeUnderlay,
            Qt::DirectConnection);

    connect(this, &QQuickWindow::beforeSynchronizing,
            this, &QuickView::synchronizeUnderlay,
            Qt::DirectConnection);

    connect(this, &QQuickWindow::beforeRendering,
            this, &QuickView::renderUnderlay,
            Qt::DirectConnection);

    connect(this, &QQuickWindow::sceneGraphInvalidated,
            this, &QuickView::invalidateUnderlay,
            Qt::DirectConnection);

    connect(m_camera, &Camera::azimuthChanged, this, &QQuickWindow::update);
    connect(m_camera, &Camera::elevationChanged, this, &QQuickWindow::update);
    connect(m_camera, &Camera::distanceChanged, this, &QQuickWindow::update);

    setClearBeforeRendering(false);

    setResizeMode(SizeRootObjectToView);
    rootContext()->setContextProperty("_camera", m_camera);
    setSource(QUrl("qrc:///qml/main.qml"));
}

In the constructor a Camera object gets created, representing the position of our virtual camera, and a MeshRenderer object, which is our black box performing OpenGL rendering (drawing the knot).

We then connect some of the signals emitted by the Qt Quick renderer to some local slots. We’re not interested in all of them; for this example just want to know:

  • when the scene graph is getting created and destroyed, so we can initialize and release the OpenGL resources;
  • when the synchronization round is happening, so we can safely access state in the main thread (which is blocked);
  • when Qt Quick is about to start rendering the next frame, so we can paint before it, therefore realizing an underlay.

Notice the Qt::DirectConnection connection type. Since the receiver (this) is living in the main thread, and the signal is emitted by the render thread, we must force the connection type otherwise it would result in a queued invocation, with disastrous results.

After those connections, we connect the update slot to the various changed signals from the Camera object, because every time the camera moves we want to schedule a redraw.

We then ask Qt Quick not to issue a glClear on the color buffer before starting its own rendering. Since we want to draw something before Qt Quick, clearing would result in not seeing anything of what we had rendered. The setClearBeforeRendering call disables (or enables) the automatic clearing.

Finally, we expose the Camera object to QML (so that it can be manipulated through the Qt Quick sliders) and load a QML file.

What about the slots themselves? Let’s have a look:

void QuickView::initializeUnderlay()
{
    m_renderer->initialize();
    resetOpenGLState();
}

void QuickView::invalidateUnderlay()
{
    m_renderer->invalidate();
    resetOpenGLState();
}

void QuickView::renderUnderlay()
{
    m_renderer->render();
    resetOpenGLState();
}

void QuickView::synchronizeUnderlay()
{
    m_renderer->setAzimuth(m_camera->azimuth());
    m_renderer->setElevation(m_camera->elevation());
    m_renderer->setDistance(m_camera->distance());
}

Yes, it’s that simple! Since our renderer is a black box, we simply call into it from the slots.

When the scene graph is initialized or invalidated, we respectively initialize or invalidate our renderer.

When Qt Quick is about to render itself, we call our own rendering code, resulting in drawing below Qt Quick.

A bit more interesting is the synchronization slot. Remember that all these slots are called directly from the render thread; as such, concurrent access from the main thread could easily result in a data race. While we could expose the Camera object to our custom OpenGL code, we would then need to implement a thread-safe Camera, which may be tricky or just impossible.

For this reason, we’ve decided to follow what the Qt Quick renderer does: we extract from the objects living in the main thread all the information required for rendering the frame. In this case, the object living in the main thread is m_camera, and the information are the spherical coordinates, which get stored somewhere else (for instance inside our black box renderer). All of this happens during the synchronization stage, while the main thread is blocked, so there is no risk of concurrent access and therefore data races.

Finally, what is the deal with all the resetOpenGLState calls? The fact is that the Qt Quick renderer does not like modifications to the OpenGL state. In other words, if in our slots we modify some OpenGL state (bind textures, change the used shader program, and so on), we must reset the OpenGL state to whatever it was before we touched it.

While this is possible in theory (OpenGL has all the necessary APIs for querying state), it’s also terribly complicated due to the countless settings that OpenGL has; moreover, if we simply call into some third party code, we have no control over which state gets touched by that. For this reason, there is a convenience call that resets the OpenGL state: QQuickWindow::resetOpenGLState. Be sure to call this function every time some OpenGL state gets modified in a slot!

That’s it!

Further steps

In the next blog post we are going to deeply integrate a custom OpenGL item inside a Qt Quick 2 scene. Stay tuned!

About KDAB

KDAB is a consulting company offering a wide variety of expert services in Qt, C++ and 3D/OpenGL and providing training courses in:

KDAB believes that it is critical for our business to contribute to the Qt framework and C++ thinking, to keep pushing these technologies forward to ensure they remain competitive.

FacebookTwitterLinkedInEmail

Categories: KDAB Blogs / KDAB on Qt / OpenGL

15 thoughts on “How to integrate OpenGL code with Qt Quick 2 applications (part 2)”

  1. Very nice tutorial. I am evaluating this approach for simplifying the development of a desktop application with a good base of pre-existent OpenGL code for visualizing some custom data structures. The only big question marks I have at the moment are:

    (1) Data managment. Assuming that the application many custom 3D data structures to be processed and visualized, where should I store the data? How should I manage updating (or exporting) data to the renderer. In the demo everything is loaded directly by the renderer;

    (2) Mouse and keyboard input. Am I able to send quick and precise mouse events to the renderer to implement, for example, a very custom mouse-keyboard interaction with the 3D models. How would I implement such a thing? How it will perform?

    Thank you again!
    Marco

    1. Giuseppe D'Angelo

      Hi Marco,

      Regarding (1) there are many possible approaches, depending on the nature of your renderer. For instance, you could have your renderer created and destroyed on demand following the corresponding sceneGraph* signals emitted by QQuickWindow. The data structures in theory should follow the renderer, as when the scene graph context gets destroyed, you will lose your OpenGL objects (buffers, textures, etc.), so they would need to be created again. Now, again, you may want to create “frontend” elements to be manipulated in the GUI thread, and corresponding “backend” elements to be manipulated in the render thread, mimicking what Qt Quick does — so that the renderer has the low-level stuff to just render, while the high level objects stay and live in the main thread. It’s really up to you.

      (Note that if this multithreading becomes a complication, a quick’n’dirty workaround is disable it (by exporting QSG_RENDER_LOOP and setting it to basic) — this basically forces Qt Quick to run single threaded.)

      In some scenarios you may want more control over the OpenGL context creation, threading, and exactly about when rendering should happen. For those scenarios, stay tuned — that’s the subject of the next instalments of these series, involving QQuickRenderControl.

      Regarding (2) there should be no problems at all, just subclass QQuickWindow or QQuickView and override the event handlers — or, catch mouse/keyboard events inside the Qt Quick scene and handle them appropriately.

      For an underlay you can use an approach like:

      class MyQuickView : public QQuickView
      {
      protected:
          void mousePressEvent(QMouseEvent *e) override {
              QQuickView::mousePressEvent(e); // let Qt Quick handle it first...
              if (!e->isAccepted()) // the click didn't land on anything in Qt Quick, I'll use it then
                  doMyHandling(e);
          }
      };
      

      Hope this helps,

  2. hi
    thank you for your comprehensive explanations.
    I having a rather unsolvable problem with scenegraph. my renderer class needs to query a database to draw the scene. how should I solve this?
    I tried singleton type – it didn’t work because the qml object is not creatable
    should I read the database in another object and pass it to the renderer through QML?

    1. Giuseppe D'Angelo

      Hello,
      In general this should be perfectly possible. You can query the database during the synchronization step, then give data to render to the renderer during the begin/after rendering phase.

  3. Hello Giuseppe
    Thanks for the post
    It’s so usefull
    I don’t know can get any reply here or not, but I’ll ask my question

    1. The question is related to events on these methods you explained one of them here
    I checked the source code and tried to add mousePressEvent on them,
    in Underlay method, it made the controllers freezed and it was not possible to interact with 3d object(even printing something)

    2. in related to RenderControll method, mousePressEvent worked fine, but have different question about it
    If we want to put the Window inside an Item of QML, we should create a QQuickItem, then register it as an item and use the RenderWindow in it? (I mean it should be connect to SIGNALS like sync and beforeRendering, .. or not?) this is a little bit ambiguous for me

    please reply
    thanks again

    1. Giuseppe D'Angelo

      Hi,
      Unfortunately it’s a bit hard to understand what went wrong. You should be able to subclass QQuickWindow and override the mouse event handlers (although it could be a bit “crude” in terms of a solution, but depends on your needs). Regarding 2, are you talking about adding a GL-drawn QQuickWindow *inside* a Qt Quick scene?

  4. Excellent post, Guiseppe.
    Can I use this overlay/underlay approach when I want the QML to display menu bars, as ApplicationWindow does? QQuickView doesn’t seem to directly support menu bars.

    Thanks!
    Tom

    1. Giuseppe D'Angelo

      Hi,
      You might go for that approach, but it sounds quite complicated (it would require to draw the menu using pure OpenGL, and handle mouse events, etc.). Isn’t a solution using Qt Quick itself not suitable for you? (E.g. by using Qt Quick Controls 2)

  5. So where is the promised next post? I promise I won’t steal your clients, I’m just curious how to integrated custom GL renderers with Qt Quick. Thank you for this blog post tho, very nice of you.

  6. Great Post, Mr.Giuseppe,

    I trying to develop a Qt Quick application that shows the live feed from a camera connected to the device and does some overlay graphics like adding points and drawing lines. Which is the best approach to use?

    1. Giuseppe D'Angelo

      Hello,

      It really depends on your needs. You might get away with simply using Qt Quick APIs: use some element for showing the video feed, then overlay it with Image/Qt Quick Shapes/custom QQPaintedItem etc. elements for the annotations.

  7. very good tutorials and learned a lot!
    my problem is when call QQuickWindow::update it will force to update all items in the window which is too heavy for performance, what is want to update just a image that is feed texture by my own image provider, not update all items, but i cannot find elegant method to do it, call item->update() is useless since its texture is render in background. any good suggestion?

    1. Giuseppe D'Angelo

      Hello Jeff,
      How are you updating your texture and how? There’s “many ways” to achieve an update. If you’re using an image provider, you could just trigger an Image element to reload it (e.g. by altering the URL).

Leave a Reply

Your email address will not be published. Required fields are marked *