Technical – KDAB https://www.kdab.com The Qt Experts Wed, 27 May 2020 19:09:51 +0000 en-US hourly 1 https://wordpress.org/?v=5.4.1 https://www.kdab.com/wp-content/uploads/stories/cropped-Favicon-32x32.png Technical – KDAB https://www.kdab.com 32 32 32184545 Using Visual Studio Code for Qt Applications – Part Two https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-2/ https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-2/#respond Thu, 28 May 2020 09:00:55 +0000 https://www.kdab.com/?p=22669 In the last blog post we saw an essential, C++ oriented, Visual Studio Code setup. That was enough to get going right away, but we can still definitely do more and better. Here I’ll show you how to get a complete setup for your qmake and CMake projects, all this while also wearing a Qt […]

The post Using Visual Studio Code for Qt Applications – Part Two appeared first on KDAB.

]]>
In the last blog post we saw an essential, C++ oriented, Visual Studio Code setup. That was enough to get going right away, but we can still definitely do more and better. Here I’ll show you how to get a complete setup for your qmake and CMake projects, all this while also wearing a Qt hat (on top of my C++ hat) and having a deeper look at the Qt side.

Build qmake Qt projects

Qmake is not integrated with Visual Studio Code the way CMake is, so setting up a qmake project for build is slightly more convoluted than doing the same with CMake. This means we’ll have to define our own build tasks. We’re going to do this in two stages: build steps definition and build steps combination, leveraging the fact that Visual Studio Code implements task dependencies and ordered sequential execution of dependencies.

Create build steps

As far as build steps are concerned, the following are, in a nutshell, the ones that will cover most cases:

  • Create the build directory (in a way that doesn’t fail if the directory already exists)
    {
      "label": "Create build dir",
      "type": "shell",
      "command": "mkdir -Force path/to/build/dir"
    }
    

    Here, -Force is a powershell parameter that will prevent the command to fail if the directory already exists. On Unix based systems, you can use mkdir -p instead.

  • Run qmake
    {
      "label": "Run qmake",
      "type": "shell",
      "command": "qmake",
      "arg": [ ... add your qmake arguments here ... ],
      "options": {
        "cwd": "path/to/build/dir"
      }
    }
    
  • Run make/nmake/jom, depending on the platform
    {
      "label": "Run make",
      "type": "shell",
      "command": "jom",
      "options": {
        "cwd": "path/to/build/dir"
      }
    }
    
  • Clear build folder This can mean different things depending on how the build file is configured. It could be a simple make clean, or a more thorough removal of the whole content of the build folder.

    {
      "label": "Clear build folder",
      "type": "shell",
      "command": "jom clean",
      "options": {
        "cwd": "path/to/build/dir"
      }
    }
    
Combine build steps

Now that our steps are defined, we can go on and define the actual build tasks. We’ll prepare two for this example, one for running a build, and one for running a clean build. Let’s see the task for a regular build:

{
  "label": "Build",
  "dependsOn": [
    "Create build dir",
    "Run qmake",
    "Run make"
  ],
  "dependsOrder": "sequence"
}

There are two new properties here: "dependsOn" is a list of task labels, and it means that those tasks need to be executed before the current task is built, while "dependsOrder", when set to "sequence", will tell Visual Studio Code to run all dependent tasks sequentially and in the given order.

The task for a clean build is very similar and will only have an extra step where the project is cleaned before being built again:

{
  "label": "Clean build",
  "dependsOn": [
    "Clear build folder",
    "Create build dir",
    "Run qmake",
    "Run make"
  ],
  "dependsOrder": "sequence"
}

And that’s it, now it’s just a matter to open the command palette (Ctrl+Shift+P), select “Task: Run Task” and then “Build”.

Use a default task

As an alternative (or better, an addition) to selecting manually the build task from a list every time, Visual Studio Code also allows to run a default task with a key combination (Ctrl+Shift+B). To mark a task as default, you need to add a few extra lines to the task configuration:

{
  // ... task configuration
  "group": {
    "kind": "build",
    "isDefault": true
  }
}
Use your own Qt

If Qt is not configured at a system level, or you want to use a Qt version other than the default one installed and configured in your system, you need to explicitly configure the environment so that every task is run with the right Qt version in the path. Visual Studio Code allows you to do this every time a terminal is launched for running a task, so our environment customizations are set before running the task command.

This is done in the settings file (or in the workspace settings if you’re working with a workspace), and the property name for this setting is system dependent: either "terminal.integrated.env.windows", "terminal.integrated.env.linux", or "terminal.integrated.env.osx". The property requires an object, where each property is the name of an environment variable, and the associated value is the value for the variable. Below is an example configuration for Windows:

{
  // All other settings...
  "terminal.integrated.env.windows": {
    "PATH": "C:/Qt/5.12.4/msvc2017_64/bin;${env:PATH}"
  }
}

Build CMake Qt projects

Setting up a CMake based project using the CMake extension doesn’t require any settings manipulation if Qt is already configured on your system. What you will need is to select a CMake kit (the CMake extension finds them automatically), a build variant, and launch the build with F7.

Short video showing how to launch a CMake build with Visual Studio Code

However, you may want to use extra arguments in the configuration step or specify your build directory so for instance it doesn’t end up being inside the source directory. You can customize CMake configuration arguments by setting the property "cmake.configureSettings" in your settings file. This property expects a list of string arguments that will be passed to CMake during the configuration step:

"cmake.configureSettings": {
  "CMAKE_PREFIX_PATH": "my/prefix/path",
  "ENABLE_FEATURE": "1",
  "ENABLE_OTHER_FEATURE": "0"
}

To customize the build directory, just set "cmake.buildDirectory" to the desired path. This value may contain variables, so it can be configured, for instance, to point a path relative to the project folder:

"cmake.buildDirectory": "${workspaceFolder}/../build-cmake-project"

If you want to use a custom Qt version, or Qt is not configured system-wide (as is the case on Windows) it’s enough to set CMAKE_PREFIX_PATH properly in the "cmake.configureSettings" property in the settings file. For example:

"cmake.configureSettings": {
  "CMAKE_PREFIX_PATH": "otherprefixpath;C:/Qt/5.12.4/msvc2017_64"
  // ... other args
]

You can find a complete documentation for the CMake Tools extension here, featuring a guide on how to use CMake Tools from the UI, and a documentation for all available settings.

Running and debugging our Qt application

Now that your application has been built, let’s see how we can launch it and, most importantly, debug it.

Running qmake projects

For projects built with qmake, we don’t have any help from extensions and the only option we have is to bake our own launch configurations in the way we’ve seen in the last blog post. This is done in the launch configurations file (launch.json) or in the workspace file, and this is how a launch configuration looks:

{
  "name": "My application",
  "type": "cppvsdbg",
  "request": "launch",
  "program": "path/to/application",
  "stopAtEntry": false,
  "cwd": "${workspaceFolder}",
  "environment": [],
  "externalConsole": false
}

You can run launch configurations both with or without debugger, using “Debug: Start Debugging” (F5) or “Run: Start Without Debugging” (Ctrl+F5) respectively. If Qt is not configured at a system level, or you want to use a custom Qt version, the corresponding launch configuration will need to be explicitly configured to include Qt in its path.

You can do this by updating the "environment" property in the launch configuration. Below is an example for Windows, setting only the "PATH" environment variable. Configurations for other systems need to be adjusted but are essentially similar.

"environment": [
  {
    "name": "PATH",
    "value": "C:/Qt/5.12.4/msvc2017_64/bin;${env:PATH}"
  }
]

Side note: here ${env:PATH} means whaterever value the environment variable PATH has before the launch configuration is run. In general, the syntax ${env:VARNAME} can be used to get an environment variable in a task or a launch configuration.

Running CMake projects

Working with CMake is easier in principle, as the CMake extension provides the commands “CMake: Run Without Debugging” and “CMake: Debug”, allowing you to respectively launch and debug CMake targets.

However, this approach has a number of shortcomings:

  • It’s not possible to specify per-target run arguments for debug runs.
  • It’s not possible at all to specify run arguments for non-debug runs.
  • Some debugging options such as source mapping or custom views with natvis are not configurable using cmake settings.

So in conclusion using the CMake extension for running targets is not really convenient if you want a comprehensive debugging experience, and the best way to go is still to create your own launch configurations.

The CMake extension provides a few convenient variables for launch configurations:

  • ${command:cmake.launchTargetPath}: resolves to the full path of the executable for the target selected from the launch target menu.
  • ${command:cmake.launchTargetDirectory}: resolves to the directory containing the executable for the target selected from the launch target menu.

Qt aware debugging

What we’ve seen until now will let you build and run your Qt applications, using either your system provided Qt or your own. Debugging will work out of the box already, as long as the application code is involved. But wouldn’t it be great to also be able to peek inside Qt’s source code while debugging? Or if we had a better visualization for Qt specific types?

Turns out we can do both with little manipulation on launch configurations. Let’s see how.

Configure debug symbols

Usually Qt debug symbols are distributed alongside libraries, so there’s no real need to explicitly configure debug symbols paths. If that’s not the case, you can configure the debug symbols path by setting the "symbolSearchPath" property on a launch configuration. This property is a string and contains a list of paths separated by a semicolon.

"symbolSearchPath": "otherSearchPath;C:/Qt/5.12.4/msvc2017_64/bin"

This of course can be useful for adding debug symbols for other libraries too.

Source mapping

If the source directory for your Qt differs from the actual source directory (or directories) used while building it, you can configure the debugger to resolve those paths correctly. This happens for instance with binary Qt releases on Windows. You can enable source mapping in launch configurations by adding the "sourceFileMap" property. This property requires an object where each key is the source folder as it’s provided by the debug symbols, and the corresponding value is the path where the source code is in your system. This is how it can be configured for a binary Qt release on Windows:

"sourceFileMap": {
    "C:/work/build/qt5_workdir/w/s": "C:/Qt/5.12.4/Src",
    "Q:/qt5_workdir/w/s": "C:/Qt/5.12.4/Src",
    "C:/Users/qt/work/install": "C:/Qt/5.12.4/Src",
    "C:/Users/qt/work/qt": "C:/Qt/5.12.4/Src"
}
Using Natvis for Qt aware objects visualization

Natvis is a Visual Studio framework that allows you to customize how native C++ objects are visualized in the debugger. Natvis visualization rules are specified through xml files with a specific schema. A natvis file lists visualization rules for each C++ type, and every visualization rule consists in a series of properties. Such properties are meant to be user friendly and will be displayed on the debug window when visualizing objects of the corresponding type.

To name a few examples, a QString is visualized as the string it contains and has a size property and a number of items corresponding to its characters, and QRect will have a width and a height property instead of just the bare (and less intuitive) internal representation of the top left and bottom right points (x1, y1, x2, y2).

If you want to enable natvis in a debug run, just set the "visualizerFile" property in your launch configuration so that it points to the natvis file.

"visualizerFile": "path/to/qt5.natvis"

Debug pane before and after configuring natvis

You can find a ready to use natvis file for Qt 5 at this link.

Updating the code model

In order to be able to navigate Qt headers and enable IntelliSense for the Qt API, it’s enough to adjust the C++ settings for our project (c_cpp_properties.json) by adding the Qt include folder (and all its subfolders):

{
  // ...
  "includePath": [
    // ...
    "C:/Qt/5.12.4/msvc2017_64/include/**"
  ]
}

If you’re working on a CMake project, it’s also possible to use the CMake plugin as a configuration provider. Doing so, include paths and defines will be bound to the currently configured CMake build, and won’t need to be specified manually. This simplifies the C++ properties file considerably, as it’s shown in the example below:

{
  "configurations": [
    {
      "name": "Win32",
      "intelliSenseMode": "msvc-x64",
      "configurationProvider": "vector-of-bool.cmake-tools"
    }
  ],
  "version": 4
}

A note about using Visual Studio compilers on Windows

Visual Studio provides batch files that automate the environment setup necessary to use their C++ compiler and linker. In the last post we saw how it’s possible to configure a task so that it sets up the environment through the vcvars.bat script before running a command.

However, if you need to configure the environment with vcvars.bat for most of your build steps, it is also possible to configure Visual Studio Code so that it runs the batch file for every task. To do so, you need to tweak the configured shell (which is powershell by default on windows) and pass a few args. The setting name for doing this is “terminal.integrated.shellArgs.windows” and it’s set as follows:

"terminal.integrated.shellArgs.windows": [
  "Invoke-BatchFile 'C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Auxiliary/Build/vcvars64.bat' amd64",
  "; powershell"
]

What’s going on here is this: Visual Studio Code will launch by default every shell task by calling this command:

powershell <terminal.integrated.shellargs.windows> -Command <task command> <task argument list>

So, if you set “terminal.integrated.shellArgs.windows” this way, the final command will look like this:

powershell Invoke-BatchFile 'path/to/vcvars' ; powershell -Command <task command> <task argument list>

As a result, task commands will be effectively run in a powershell with the right environment set.

And that’s it for now. Many new things on the table, and some advanced features too. Hopefully this will help you with your workflow.

But there is still more to say, so make sure you don’t miss the next post!

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Using Visual Studio Code for Qt Applications – Part Two appeared first on KDAB.

]]>
https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-2/feed/ 0 22669
Using Modern CMake with Qt https://www.kdab.com/using-modern-cmake-with-qt/ https://www.kdab.com/using-modern-cmake-with-qt/#respond Mon, 25 May 2020 15:52:03 +0000 https://www.kdab.com/?p=24101 KDAB’s Kevin Funk presented Using Modern CMake with Qt at Qt Virtual Tech Con last month. He reported that the Qt Company did a great job moderating the sessions at this event, and there was a lively Q&A at the end – Kevin had to pick from about 60 questions, so this is a hot […]

The post Using Modern CMake with Qt appeared first on KDAB.

]]>
KDAB’s Kevin Funk presented Using Modern CMake with Qt at Qt Virtual Tech Con last month.

He reported that the Qt Company did a great job moderating the sessions at this event, and there was a lively Q&A at the end – Kevin had to pick from about 60 questions, so this is a hot topic.

Now the event is over you can access Kevin’s talk here, including the answers he had time for, and also his slides, below the abstract.

Using Modern CMake with Qt with Kevin Funk

Prerequisite: No prior CMake experience required

CMake is a cross-platform build system, with powerful APIs for finding dependencies of various or specific versions, and with many abstractions for platforms, compilers, other build systems, and dependencies.

The next major Qt version, Qt6, will be using CMake internally as its build system, so the CMake integration with Qt will likely get tighter and more versatile in the long-term.

In this talk, we’ll be introducing Qt specific CMake functionalities, in order to find and use Qt5 inside your personal CMake-based project, using modern CMake capabilities. We are going to discuss how to find Qt installs using CMake’s find_package function and how to find specific Qt versions when multiple versions are installed.

Further than that, useful CMake variables such as CMAKE_INCLUDE_CURRENT_DIR, CMAKE_AUTOMOC, CMAKE_AUTORCC, and CMAKE_AUTOUIC will be explained in detail and how the use of the CMake integrations can speed up the build drastically.

Last but not least some of the additional supplied Qt related CMake functions, such as for big resources or translation support will be discussed.

Target audience: Build Engineers or Software Engineers who would like to know more about using Qt under CMake.

Download Kevin’s slides: QTVTC20 – Using Modern CMake – Kevin Funk

About Kevin Funk

Kevin has actively developed with Qt/C++ since 2006 and has a special interest in tooling and profiling. He’s an active contributor to KDAB’s GammaRay analyzer (a high-level Qt application debugger) and has a strong emphasis on state machine tooling. He is a co-maintainer of the KDevelop IDE, a powerful C/C++ development environment backed by Clang, and is pushing for cross-platform success inside KDE. Kevin holds a Masters Degree in Computer Science.

Download Kevin’s whitepaper on CMake and Qt…

The post Using Modern CMake with Qt appeared first on KDAB.

]]>
https://www.kdab.com/using-modern-cmake-with-qt/feed/ 0 24101
Automating tasks in Qt Creator https://www.kdab.com/automating-tasks-in-qt-creator/ https://www.kdab.com/automating-tasks-in-qt-creator/#comments Thu, 21 May 2020 09:00:23 +0000 https://www.kdab.com/?p=23489 My first project when I entered KDAB was the migration of a multi-million lines Motif application to Qt… feels quite scary said like that. Fortunately, migrations from any toolkit to Qt is something KDAB has been doing from the beginning, and has lots of experience with. You may be wondering what this has to do […]

The post Automating tasks in Qt Creator appeared first on KDAB.

]]>
My first project when I entered KDAB was the migration of a multi-million lines Motif application to Qt… feels quite scary said like that. Fortunately, migrations from any toolkit to Qt is something KDAB has been doing from the beginning, and has lots of experience with.

You may be wondering what this has to do with automating tasks in Qt Creator, or why I have a new entry in my Qt Creator locator… keep up with me and everything will be clear shortly.

Qt Creator Locator showing the Run Script selected

Automate all things

There are several rules you have to follow when doing a migration project, but the most important ones are probably those two:

  • Do not mix migration and refactoring
  • Automate all things

The second one is true for everything (remember, a good developer is a lazy developer), but it’s particularly important when doing a migration, as you don’t want to rewrite all single lines in a multi-million lines application by hand.

At that time, I was introduced to the wonderful world of XEmacs and Lisp scripting. During the course of the migration, I spent hours writing Lisp scripts to automate some parts of the migration, some with general purpose (used everywhere), others with a very narrow focus (only used once for one file). Coming from Windows and Microsoft Visual Studio 6 at the time, this was quite a new way to work for me.

Then slightly after this migration project, Qt Creator was released. As much as I like XEmacs and vim (and we all know which one is the best 🙂 ), having a real IDE makes development so much easier, and Qt Creator was adopted quickly inside KDAB.

The road to automation in Qt Creator

Even if we are using Qt Creator, we still need to automate as much as possible, particularly when doing migration projects.

The Macro plugin

My first try was the creation of the macro plugin, contributed to Qt Creator in version 2.2.0. For those who know it, the name is inspired by the macro feature in Notepad++ (also a very nice editor at the time – Windows only). The plugin allows you to save and replay user inputs:

  • Start recording using Alt+[
  • Navigate, search, write code
  • Stop recording using Alt+]
  • Then replay the macro using Alt+R

If you want to reuse a macro between session, it’s possible to save it, call it using the locator (the rm shortcut – “run macro” – probably not really used) or even assign a shortcut.

Though a nice step toward automation, it’s still far from perfect, as a macro doesn’t understand code semantics (it’s only user inputs), and not everything can be recorded: for example completion is not recorded in the macro, as it’s not a user input.

The Scripting plugin

Following the macro plugin, our second iteration was the creation of a plugin allowing us to write and run scripts, like you could do with Lisp and XEmacs. At the time, Qt Script was still nice and shiny, so it was based on it and the scripts were written in javascript. Like for macros, you can run scripts using the locator, or even assign shortcuts to useful scripts.

Here is a small script commenting out the function the cursor is in, while keeping the cursor at the same position:

var editor = editors.current()
var start = editor.createMark()

editor.gotoPosition(editor.currentFunction.start)
editor.find("{")

var startOfFunction = editor.position()
editor.gotoBlockEnd()
editor.gotoLineStart();
editor.insert("*/\n")
editor.gotoPosition(startOfFunction)
editor.insert("\n/*")

editor.gotoPosition(start)

As you can see, it’s straightforward, and it has some semantic information of the code (editor.currentFunction for example). You have access to a wide range of APIs for scripting:

  • Access and handling of editors (text, C++ editors)
  • Code navigation
  • Files, file infos, directories
  • Semantic data
  • Git actions

It’s actually way better than macros, and to be honest I now only use macros locally, I never save them anymore.

This plugin is used for our Photon to Qt or MFC to Qt migrations.

The Scripting plugin… but better

Lately, we reviewed the scripting plugin and checked what could be improved:

  • It is based on the dying Qt Script module
  • It’s not possible to interact with the script

Back to the drawing board, coming from Qt Script/javascript the natural way forward is Qt Declarative/QML. Our new scripting plugin is now based on the QQmlEngine, and now supports user interactions.

A glimpse into our Qt Creator Scripting plugin

As said, the new plugin is now based on the QQmlEngine, opening a wide array of new features to us.

Script types

There are now two different script types one can use, depending on the needs:

  • javascript (*.js): automation without user interactions
  • QML (*.qml): automation, which allows us to add user interaction and visual UI

For the javascript scripts, it’s close to what was possible with the previous scripting plugin based on Qt Script, the only difference is that we require a main function.

function main() {
    message.log("This is a log line: message.log(...)")
}

A QML script uses the same API, but allows more freedom, as you can now add a UI to it (using QtQuick) and/or add user interaction with the mouse and keyboard. Here is a small example of a script adding a header to the current file (it could have been done with a javscript script too, it’s just an example):

import QtQml 2.2
import Script 1.0

Script {
    property var mark

    function init() {
        mark = editor.createMark()
        editor.gotoDocumentStart()
        editor.insert("/****************************************************************************\n")
        editor.insert("/* Copyright (C) " + Qt.formatDate(new Date(), "yyyy") + " Klaralvdalens Datakonsult AB. All rights reserved.\n")
        editor.insert("** This file is part of the ACME project.")
        editor.insert("\n**********************************************************************/\n\n")
        m.restore()
        Qt.quit()
    }
    TextEditor {
        id: editor
        fileName: Editors.currentFileName
    }
    Component.onCompleted: init()
}

User interaction

One of the benefits of having QML scripting is the availability of handling user interaction on an editor: as a script creator, you can easily handle key or mouse inputs on an editor:

import QtQml 2.2import Script 1.0
TextEditor {
    id: editor
    fileName: Util.mktemp("foo_XXXXXX.txt")

    KeyHandler {
        key: Qt.Key_Left
        filter: true
        onKeyPressed: doSomething()
    }
    MouseHandler {
        onClicked: doSomethingElse()
    }
    // ...
}

Which ultimately leads you to write a game (actually 2: whack-a-mole and follow-path):

Conclusion

As someone said once: developers are lazy, and that’s (usually) a good thing. It certainly took us some time to develop the different plugins, but it’s paying off. It’s also more fun to write a tool to automate the work than doing the work itself.

For now the scripting plugin is internal to KDAB, let us know if that’s something you would be interested in.

You can download a KDAB whitepaper on Migrations here…

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Automating tasks in Qt Creator appeared first on KDAB.

]]>
https://www.kdab.com/automating-tasks-in-qt-creator/feed/ 4 23489
Why is my screen still black https://www.kdab.com/why-is-my-screen-still-black/ https://www.kdab.com/why-is-my-screen-still-black/#respond Mon, 18 May 2020 09:00:55 +0000 https://www.kdab.com/?p=23340 Part 2 If you are here from Part 1, welcome back. If you are wondering why we started at Part 2, go to Part 1. So, you tried everything from the first part (that was relative to you), and your screen is still a backlighting test? No problem. (Well.. I mean, clearly there is but […]

The post Why is my screen still black appeared first on KDAB.

]]>
Part 2

If you are here from Part 1, welcome back. If you are wondering why we started at Part 2, go to Part 1.

Black Screen

Uncanny Valley!

So, you tried everything from the first part (that was relative to you), and your screen is still a backlighting test? No problem. (Well.. I mean, clearly there is but you know what I mean) We’ve gathered another five reasons this could be happening and how to go about fixing them.

Issue 6: Directional light is pointing in the wrong direction.

There are a few choices when it comes to choosing a light source for your scene. You can even add more than one. So we have Point lights, Directional lights, Spot lights and Area lights. You can then also have emissive textures that will act as a light source. Some of these get tricky… say, if you have a Spot or Directional light, you have to make sure they are pointing at something. For testing, you can always have a point light in the scene but even with these, you have to make sure they won’t be ‘inside’ an object, aren’t black and have a decent brightness.

Issue 7: It’s so dark in here. Did you not turn any lights on?

If using a lighting model, ensure the lights are actually enabled. A good test here is to set a temporary glClearColor (eg pink) – this will usually show un-lit geometry as black. Another solution is to set an ambient colour term and/or changing the background colour to something lighter.

Issue 8: Are you supplying enough power to your Raspberry Pi 4?

The Raspberry Pi 4 has dual HDMI outputs. Using the Qt EGLFS backend, these are available as different displays – so can run two different Qt applications, one full-screen on each. But if your system is near the limits of its power supply (and the onboard power monitor is quite picky, on the Pi) – the second display might fail to initialise. When debugging this problem, we initially tested different OpenGL backends and QPA plugins, but none of these made any difference. Very occasionally both displays would work, but mostly the second one would fail to initialise with an EGL error. A beefier power supply fixed the problem immediately. (Sometimes, it’s really not a software problem)

Issue 9: The GL context won’t create.

This can happen for instance if you upgrade your drivers (especially common with Linux and NVidia cards) and don’t reboot, there’s a high chance that your system won’t be able to create a GL context anymore. To make sure that this is the issue, start a simple 3D program such as glxgears. If it does not work, only one solution : reach for that restart button.

For more info, see: Checkliste Allgemein (German)

Issue 10: Return of the mac. – You’re using the wrong profile

OpenGL has been around for a while and has many version. Maybe you are using something that requires a more recent version of OpenGL? One thing that is more subtle, especially when updating old code to more modern practices, is the concept of profile. As of 3.2, contexts can be created with a Core profile or a Compatibility profile. The latter preserves compatibility with older fixed function pipeline code and settings. However, it is optional for drivers to implement that profile. Apple, in its wisdom, has decided not to do so. So if you ask for a Compatibility profile, it will create a 2.1 context, and you will not be able to use 3.0 or later features.

So, make sure Core profile is enabled, on the default QSurfaceFormat instance.

Other cross-platform issues are quite common. For example, NVIDIA drivers tend to be forgiving and accept using texture2D() in shaders even though it should not be allowed in a Core profile. So test on as many platforms and driver setups you can lay your hands on.

Once you’ve double checked the camera settings, shaders and your model settings, 6, 7, 8, 9, and 10, you should be good to go! If not, why not comment your issue below and we’ll try to get it in the next part.

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Why is my screen still black appeared first on KDAB.

]]>
https://www.kdab.com/why-is-my-screen-still-black/feed/ 0 23340
Debugging and Profiling Qt 3D applications https://www.kdab.com/debugging-profiling-qt-3d-apps/ https://www.kdab.com/debugging-profiling-qt-3d-apps/#respond Tue, 24 Mar 2020 10:00:53 +0000 https://www.kdab.com/?p=22490 Qt 3D, being a retained mode high level graphic API abstraction, tries to hide most of the details involved in rendering the data provided by applications. It makes a lot of decisions and operations in the background in order to get pixels on the screen. But, because Qt 3D also has very rich API, developers […]

The post Debugging and Profiling Qt 3D applications appeared first on KDAB.

]]>
Qt 3D, being a retained mode high level graphic API abstraction, tries to hide most of the details involved in rendering the data provided by applications. It makes a lot of decisions and operations in the background in order to get pixels on the screen. But, because Qt 3D also has very rich API, developers can have a lot of control on the rendering by manipulating the scene graph and, more importantly, the frame graph. It is however sometimes difficult to understand how various operations affect performance.

In this article, we look at some of the tools, both old and new, that can be used to investigate what Qt 3D is doing in the back end and get some insight into what is going on during the frame.

 

Built in Profiling

The first step in handling performance issues is, of course, measuring where time is spent. This can be as simple as measuring how long it took to render a given frame. But to make sense of these numbers, it helps to have a notion of how complex the scene is.

In order to provide measurable information, Qt 3D introduces a visual overlay that will render details of the scene, constantly updated in real time.

 

The overlay shows some real time data:

  • Time to render last frame and FPS (frames per second), averaged and plotted over last few seconds. As Qt 3D is by default locking to VSync, this should not exceed 60fps on most configurations.
  • Number of Jobs: these are the tasks that Qt 3D executes on every frame. The number of jobs may vary depending on changes in the scene graph, whether animations are active, etc.
  • Number of Render Views: this matches loosely to render pass, see below discussion on the frame graph.
  • Number of Commands: this is total number of draw calls (and compute calls) in the frame.
  • Number of Vertices and Primitives (triangles, lines and points combined).
  • Number of Entities, Geometries and Textures in the scene graph. For the last two, the overlay will also show the number of geometries and textures that are effectively in use in the frame.

As seen in the screen shots above, the scene graph contains two entities, each with one geometry. This will produce two draw calls when both objects are in frame. But as the sphere rotates out of the screen, you can see the effect of the view frustum culling job which is making sure the sphere doesn’t get rendered, leaving a single draw call for the torus.

This overlay can be enabled by setting the showDebugOverlay property of the QForwardRenderer to true.

 

Understanding Rendering Steps

To make sense of the numbers above, it helps to understand the details of the scene graph and frame graph.

In the simple case, as in the screen shots, an entity will have a geometry (and material, maybe a transform). But many entities may share the same geometry (a good thing if appropriate!). Also, entities may not have any geometry but just be used for grouping and positioning purposes.

So keeping an eye on the number of entities and geometries, and seeing how that effects the number of commands (or draw calls), is valuable. If you find one geometry drawn one thousand times in a thousand separate entities, if may be a good indication that you should refactor your scene to use instanced rendering.

In order to provide more details, the overlay has a number of buttons that can be used to dump the current state of the rendering data.

For a deeper understanding of this, you might consider our full Qt 3D Training course.

Scene Graph

Dumping the scene graph will print data to the console, like this:

Qt3DCore::Quick::Quick3DEntity{1} [ Qt3DRender::QRenderSettings{2}, Qt3DInput::QInputSettings{12} ]
  Qt3DRender::QCamera{13} [ Qt3DRender::QCameraLens{14}, Qt3DCore::QTransform{15} ]
  Qt3DExtras::QOrbitCameraController{16} [ Qt3DLogic::QFrameAction{47}, Qt3DInput::QLogicalDevice{46} ]
  Qt3DCore::Quick::Quick3DEntity{75} [ Qt3DExtras::QTorusMesh{65}, Qt3DExtras::QPhongMaterial{48},
                                       Qt3DCore::QTransform{74} ]
  Qt3DCore::Quick::Quick3DEntity{86} [ Qt3DExtras::QSphereMesh{76}, Qt3DExtras::QPhongMaterial{48}, 
                                       Qt3DCore::QTransform_QML_0{85} ]

This prints the hierarchy of entities and for each of them lists all the components. The id (in curly brackets) can be used to identify shared components.

Frame Graph

Similar data can be dumped to the console to show the active frame graph:

Qt3DExtras::QForwardRenderer
  Qt3DRender::QRenderSurfaceSelector
    Qt3DRender::QViewport
      Qt3DRender::QCameraSelector
        Qt3DRender::QClearBuffers
          Qt3DRender::QFrustumCulling
            Qt3DRender::QDebugOverlay

This is the default forward renderer frame graph that comes with Qt 3D Extras.

As you can see, one of the nodes in that graph is of type QDebugOverlay. If you build your own frame graph, you can use an instance of that node to control which surface the overlay will be rendered onto. Only one branch of the frame graph may contain a debug node. If the node is enabled, then the overlay will be rendered for that branch.

The frame graph above is one of the simplest you can build. They may get more complicated as you build effects into your rendering. Here’s an example of a Kuesa frame graph:

Kuesa::PostFXListExtension
  Qt3DRender::QViewport
    Qt3DRender::QClearBuffers
      Qt3DRender::QNoDraw
    Qt3DRender::QFrameGraphNode (KuesaMainScene)
      Qt3DRender::QLayerFilter
        Qt3DRender::QRenderTargetSelector
          Qt3DRender::QClearBuffers
            Qt3DRender::QNoDraw
          Qt3DRender::QCameraSelector
            Qt3DRender::QFrustumCulling
              Qt3DRender::QTechniqueFilter
                Kuesa::OpaqueRenderStage (KuesaOpaqueRenderStage)
                  Qt3DRender::QRenderStateSet
                    Qt3DRender::QSortPolicy
            Qt3DRender::QTechniqueFilter
              Kuesa::OpaqueRenderStage (KuesaOpaqueRenderStage)
                Qt3DRender::QRenderStateSet
                  Qt3DRender::QSortPolicy
            Qt3DRender::QFrustumCulling
              Qt3DRender::QTechniqueFilter
                Kuesa::TransparentRenderStage (KuesaTransparentRenderStage)
                  Qt3DRender::QRenderStateSet
                    Qt3DRender::QSortPolicy
            Qt3DRender::QTechniqueFilter
              Kuesa::TransparentRenderStage (KuesaTransparentRenderStage)
                Qt3DRender::QRenderStateSet
                  Qt3DRender::QSortPolicy
          Qt3DRender::QBlitFramebuffer
            Qt3DRender::QNoDraw
    Qt3DRender::QFrameGraphNode (KuesaPostProcessingEffects)
      Qt3DRender::QDebugOverlay
        Qt3DRender::QRenderStateSet (ToneMappingAndGammaCorrectionEffect)
          Qt3DRender::QLayerFilter
            Qt3DRender::QRenderPassFilter

If you are not familiar with the frame graph, it is important to understand that each path (from root to leaf) will represent a render pass. So the simple forward renderer will represent a simple render pass, but the Kuesa frame graph above contains eight passes!

It is therefore often easier to look at the frame graph in term of those paths. This can also be dumped to the console:

[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QClearBuffers, Qt3DRender::QNoDraw ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene),
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QClearBuffers, Qt3DRender::QNoDraw ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene), 
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QCameraSelector, Qt3DRender::QFrustumCulling, 
  Qt3DRender::QTechniqueFilter, Kuesa::OpaqueRenderStage (KuesaOpaqueRenderStage), Qt3DRender::QRenderStateSet, 
  Qt3DRender::QSortPolicy ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene), 
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QCameraSelector, Qt3DRender::QTechniqueFilter, 
  Kuesa::OpaqueRenderStage (KuesaOpaqueRenderStage), Qt3DRender::QRenderStateSet, Qt3DRender::QSortPolicy ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene),
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QCameraSelector, Qt3DRender::QFrustumCulling,
  Qt3DRender::QTechniqueFilter, Kuesa::TransparentRenderStage (KuesaTransparentRenderStage), Qt3DRender::QRenderStateSet,
  Qt3DRender::QSortPolicy ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene),
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QCameraSelector, Qt3DRender::QTechniqueFilter,
  Kuesa::TransparentRenderStage (KuesaTransparentRenderStage), Qt3DRender::QRenderStateSet, Qt3DRender::QSortPolicy ]
[ Kuesa::PostFXListExtension, Qt3DRender::QViewport, Qt3DRender::QFrameGraphNode (KuesaMainScene),
  Qt3DRender::QLayerFilter, Qt3DRender::QRenderTargetSelector, Qt3DRender::QBlitFramebuffer, Qt3DRender::QNoDraw ]

Hopefully this is a good way of finding out issues you may have when building your custom frame graph.

Draw Commands

On every pass of the frame graph, Qt 3D will traverse the scene graph, find entities that need to be rendered, and for each of them, issue a draw call. The number of objects drawn in each pass may vary, depending on whether the entities and all of their components are enabled or not, or whether entities get filtered out by using QLayers (different passes may draw different portions of the scene graph).

The new profiling overlay also gives you access to the actual draw calls.

So in this simple example, you can see that two draw calls are made, both for indexed triangles. You can also see some details about the render target, such as the viewport, the surface size, etc.

That information can also be dumped to the console which makes it easier to search in a text editor.

 

Built in Job Tracing

The data above provides a useful real time view on what is actually being processed to render a particular frame. However, it doesn’t provide much feedback as to how long certain operations take and how that changes during the runtime of the application.

In order to track such information, you need to enable tracing.

Tracing tracks, for each frame, what jobs are executed by Qt 3D’s backend. Jobs involve updating global transformations and the bounding volume hierarchy, finding objects in the view frustum, layer filtering, picking, input handling, animating, etc. Some jobs run every frame, some only run when internal state needs updating.

If your application is slow, it may be because jobs are taking a lot of time to complete. But how do you find out which jobs take up all the time?

Qt 3D has had tracing built in since a few years already, but it was hard to get to. You needed to do your own build of Qt 3D and enable tracing when running qmake. From thereon, every single run of an application linked against that build of Qt 3D would generate a trace file.

In 5.15, tracing is always available. It can be enabled in two ways:

  • By setting the QT3D_TRACE_ENABLED environment variable before the application starts (or at least before the aspect engine is created). This means the tracing will happen for the entire run of the application.
  • If you’re interested in tracing for a specific part of your application’s life time, you can enable the overlay and toggle tracing on and off using the check for Jobs. In this case, a new trace file will be generated every time the tracing is enabled.

For every tracing session, Qt 3D will generate one file in the current working directory. So how do you inspect the content of that file?

KDAB provides a visualisation tool but it is not currently shipped with Qt 3D. You can get the source and build it from GitHub here. Because jobs change from one version of Qt 3D to the next, you need to take care to configure which version was used to generate the trace files. Using that tool, you can open the trace files. It will render a time line of all the jobs that were executed for every frame.

In the example above, you can see roughly two frames worth of data, with jobs executed on a thread pool. You can see the longer running jobs, in this case:

  • RenderViewBuilder jobs, which create all the render views, one for each branch in the frame graph. You can see some of them take much longer that others.
  • FrameSubmissionPart1 and FrameSubmissionPart2 which contain the actual draw calls.

Of course, you need to spend some time understanding what Qt 3D is doing internally to make sense of that data. As with most performance monitoring tools, it’s worth spending the time experimenting with this and seeing what gets affected by changes you make to your scene graph or frame graph.

Job Dependencies

Another important source of information when analysing performance of jobs is looking at the dependencies. This is mostly useful for developers of Qt 3D aspects.

Using the profiling overlay, you can now dump the dependency graph in GraphViz dot format.

Other Tools

Static capabilities

Qt 3D 5.15 introduces QRenderCapabilities which can be used to make runtime decisions based on the actual capabilities of the hardware the application is running on. The class supports a number of properties which report information such as the graphics API in use, the card vendor, the supported versions of OpenGL and GLSL. It also has information related to the maximum number of samples for MSAA, maximum texture size, if UBOs and SSBOs are supported and what their maximum size is, etc.

Third Party Tools

Of course, using more generic performance tools is also a good idea.

perf can be used for general tracing, giving you insight where time is spent, both for Qt 3D and for the rest of your application. Use it in combination with KDAB’s very own hotspot to get powerful visualisation of the critical paths in the code.

Using the flame graph, as show above (captured on an embedded board), you can usually spot the two main sections of Qt 3D work, the job processing and the actual rendering.

Other useful tools are the OpenGL trace capture applications, either the generic ones such as apitrace and renderdoc, or the ones provided your hardware manufacturer, such as nVidia or AMD.

 

Conclusion

We hope this article will help you get more performance out of your Qt 3D applications. The tools, old and new, should be very valuable to help find bottlenecks and see the impact of changes you make to your scene graph or frame graph. Furthermore, improvements regarding performance are in the works for Qt 6, so watch this space!

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Debugging and Profiling Qt 3D applications appeared first on KDAB.

]]>
https://www.kdab.com/debugging-profiling-qt-3d-apps/feed/ 0 22490
Using Visual Studio Code for Qt Applications – Part One https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-1/ https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-1/#respond Wed, 18 Mar 2020 10:00:12 +0000 https://www.kdab.com/?p=21910 A few weeks ago, we published an article with an overview of Visual Studio Code through the eyes of a Qt developer. In this short blog series, I will show you how to get up to speed with a Qt project using Visual Studio Code more in detail. Before digging into the Qt parts, I’d […]

The post Using Visual Studio Code for Qt Applications – Part One appeared first on KDAB.

]]>
A few weeks ago, we published an article with an overview of Visual Studio Code through the eyes of a Qt developer.

In this short blog series, I will show you how to get up to speed with a Qt project using Visual Studio Code more in detail. Before digging into the Qt parts, I’d like to go through an overview of the bits and pieces that are needed to configure any C++ project.

Setup a workspace

With Visual Studio Code you can start hacking on your code pretty much right away, just by opening the folder where your code resides. Sometimes if you’re just skimming through a codebase and you just want to be able to look for specific strings on the whole project folder that is all you need.

You can also manage multiple folders at the same time on a single Visual Studio Code window, search for strings on all files on all folders, and if you set up your workspace properly, also cross navigate between files in different folders with the help of a C++ code model.

Screenshot of a visual studio code windows on a workspace with two folders

A Visual Studio Code workspace is just a bundle of opened folders

In any case, you can add a folder to your workspace by selecting the “File” menu and then “Add folder to workspace…”. Removing folders from a workspace is done by right clicking on a folder and then selecting “Remove Folder from Workspace”.

Doing anything more than just reading and searching through your code will require you to save and possibly adjust your workspace. Let’s start from saving your workspace. To do so, click on the “File” menu and select “Save Workspace As…”.

Once you have saved your workspace file, you can start editing it. That can be done through Visual Studio Code itself since it is basically a JSON file listing all folders belonging to the project and workspace level (that is, global) settings. If you have multiple folders, you may want to assign a name to each one as it might be helpful later on. To do so, edit the workspace file, and add a “name” attribute to the object relative to your folder, alongside the “path” attribute.

Short video showing how to update folder names on Visual Studio Code workspaces

Get the right extensions

Putting the C++ developer hat on, other than just being able to search through your source code, you may also want some code model to be available, so that you can easily detect for instance wrong includes, typos in variables names and the like.

Visual Studio Code is language agnostic by nature, but lets you do this on C/C++ code bases by means of a C++ extension, which you can find in the marketplace under the unsurprising name “C/C++” (full extension identifier: ms-vscode.cpptools).

If you plan to use CMake in your projects, another handy extension is “CMake Tools” (ms-vscode.cmake-tools).

You can install them both as shown below, after opening the command list by pressing “Ctrl+Shift+P”.

Configuration files

Now that the workspace is configured we can finally go on and set up build configurations, run configurations and C++ specific settings. We will see these operations in detail below, but before doing that it’s worth spending a few extra words on some common concepts.

If you don’t have any configuration file yet, be it a run, build, or C++ settings file, Visual Studio Code will create them for you as soon as you try to update your configuration. By default, it will do so by creating a .vscode subfolder under your first workspace folder, and placing all your configuration files there. You can also choose to have one .vscode subfolder for each one of your workspace folders and add configuration files in each of them. This will let you adjust settings on a per-folder basis.

In all configuration files, you can use ${workspaceFolder} to refer to the current source code folder. It is important to keep in mind that ${workspaceFolder} will always refer to the workspace folder where the configuration files reside. If you want to access other workspace folders, you can do it by explicitly referring to their name in the workspace configuration file, so if the folder is named “folder1” in the workspace file, you can access it in the configuration file using ${workspaceFolder:folder1}.

Add and run build configurations

The first step to add a new build configuration is to open the command line (Ctrl+Shift+P) and select “Tasks: Run Task”. You will be prompted to create a “tasks.json” file if you don’t have one already. For a simple build task, just select “Others” for the task type. This will create an example “shell” build configuration which lets you run a command from your shell with a number of arguments.

From now on you can start setting up your build configurations as in the example below.

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Build target1",
      "type": "shell"
      "command": "make /f ${workspaceFolder:folder1}/Makefile target1"
      "options": {
        "cwd": "${workspace:folder1}"
      }
    }
    // ... add more tasks here
  ]
}

Once builds are configured Visual Studio code will allow you to:

  • mark a run configuration as default (Ctrl+Shift+P -> “Tasks: Configure Default Build Task”)
  • run a specific build configuration (Ctrl+Shift+P -> “Tasks: Run Task”, then select from the list of available build configurations)
  • run the default build configuration (Ctrl+Shift+P -> “Tasks: Run Build Task” or just Ctrl+Shift+B)
How to use MSVC in a build configuration

When launching a build configuration on Windows, you will not have access to the MSVC compiler and all its associated environment variables by default. So if you try to run for instance nmake to build your project your build will fail.

This is usually solved by calling vcvarsall.bat (or its moral equivalent for later Visual Studio versions) before launching a build. We can’t do that straight away here because a shell run configuration command will be run through Windows’ PowerShell console and not through the classic command prompt, and vcvarsall.bat won’t run on PowerShell.

One way to fix this is to install the pscx module on your PowerShell. You can do it by running a PowerShell with administrator rights and typing:

Install-Module pscx

Once the module is installed you can run any batch file and retain the resulting environment by running Invoke-BatchFile. You can now use nmake (or any other MSVC binary) in a build configuration as in the example below:

{
  "label": "Build target1",
  "type": "shell"
  "command": "Invoke-BatchFile 'PATH_TO_YOUR_VCVARS' ; nmake /f ${workspaceFolder:folder1}/Makefile target1"
  "options": {
     "cwd": "${workspace:folder1}"
  }
}

Add and run launch configurations

The next step once your app is built is to add launch configuration(s) so you can debug (or just run) your executable(s).

First, let’s get Visual Studio Code to create a launch configuration file for you by opening the command line (Ctrl+Shift+P), selecting “Debug: Open launch.json”, and then “C++ (Windows)”. This will create a new launch.json configuration file. From there, it’s enough to update the “program” and “cwd” parameters on the autogenerated file to get a working configuration.

Once you’re done your launch configuration should look like this:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "target1",
      "type": "cppvsdbg",
      "request": "launch",
      "program": "${workspaceFolder:folder1}/build/target1",
      "args": [],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder:folder1}/build",
      "environment": []
    }
    // ... add more targets here
  ]
}

Similarly to what happens with the build configuration file, you can add a new launch configurations by adding a new JSON object under the “configurations” list.

Once this is done, you can see a list of run configurations on the debug tab, which you can access by clicking on the small bug on the left side of the window or by pressing “Ctrl+Shift+D” on windows.

Workspace level configurations

All the build and run configurations we saw until now can also be set at a workspace level. This could be useful, for instance, if you want to access the same folder from multiple workspaces and different build configurations depending on the workspace.

The syntax is the same as for the folder level configuration files, you just have to put all the settings in your workspace file as in the following example:

{
  "folders": [ .... ],
  "settings": [ .... ],
  "tasks": [
    { ... first build conf ... },
    { ... second build conf ... }
  ],
  "launch": [
    { ... first run conf ... },
    { ... second run conf ... }
  ]
}

Set up the code model

With the steps described up to now, it will be possible to navigate across all files within the project which is currently opened inside the editor. It will also be possible to use code completion (IntelliSense) for all the types inside said project.

But what if your project is using third party libraries (say, Qt) and you want to be able to navigate to library headers and have code completion for library types (say, QObject, or QWidget)?

This can be configured thanks to the C/C++ extension installed before. To get it to work, we need to edit the configuration file for this extension by going to the command prompt (Ctrl+Shift+P) and selecting “C/C++: Edit configurations (JSON)”.

This is what the default configuration will look like on Windows:

{
    "configurations": [
        {
            "name": "Win32",
            "includePath": [
                "${workspaceFolder}/**"
            ],
            "defines": [
                "_DEBUG",
                "UNICODE",
                "_UNICODE"
            ],
            "windowsSdkVersion": "10.0.17763.0",
            "compilerPath": "C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.22.27905/bin/Hostx64/x64/cl.exe",
            "cStandard": "c11",
            "cppStandard": "c++17",
            "intelliSenseMode": "msvc-x64"
        }
    ],
    "version": 4
}

Here it is possible to tweak with the C and C++ standard versions (cStandard and cppStandard), but the most important fields if you want to include third party libraries to the code model properly are includePath and defines.

includePath is a JSON list and will tell the code model builder where to look for header files, so, for instance, if you are using Qt in your project, you will need to add something like “C:/Qt/5.12.7/msvc2017_64/include/**”. Note the trailing **, that tells the code model to look for headers in all the subfolders.

defines is a list that will tell the code model builder which preprocessor defines to use for building the code model. They should ideally reflect the ones that will be used at build time, as the C/C++ plugin can’t infer them from build tasks.

This concludes the first round of notes about using Visual Studio Code in your C++/Qt projects. It includes all the fundamental pieces and should be enough to get started using Visual Studio Code effectively. In the next posts I will go more into details on how to add Qt, qmake and cmake to the scene.

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

 

The post Using Visual Studio Code for Qt Applications – Part One appeared first on KDAB.

]]>
https://www.kdab.com/using-visual-studio-code-for-qt-apps-pt-1/feed/ 0 21910
Getting rid of “volatile” in (some of) Qt https://www.kdab.com/getting-rid-of-volatile-in-some-of-qt/ https://www.kdab.com/getting-rid-of-volatile-in-some-of-qt/#comments Wed, 04 Mar 2020 13:00:18 +0000 https://www.kdab.com/?p=21592 The upcoming version of the C++ Standard (C++2a) is proposing to deprecate certain usages of the volatile keyword, by adopting the P1152 proposal (Deprecating volatile). Despite the somewhat “flamboyant” title, the actual deprecated parts are very limited and indeed the paper limits the deprecation to somehow language/library corner cases and/or dangerous antipatterns. For instance certain […]

The post Getting rid of “volatile” in (some of) Qt appeared first on KDAB.

]]>
The upcoming version of the C++ Standard (C++2a) is proposing to deprecate certain usages of the volatile keyword, by adopting the P1152 proposal (Deprecating volatile).

Despite the somewhat “flamboyant” title, the actual deprecated parts are very limited and indeed the paper limits the deprecation to somehow language/library corner cases and/or dangerous antipatterns.

For instance certain read-modify-write operations on a variable declared volatile are now deprecated, like:

volatile int i = 42;
++i;     // deprecated
i *= 2;  // also deprecated

In any case, I took the occasion for digging a bit into Qt’s own usages of volatile. I didn’t do a thorough search through the entirety of Qt because I’ve also tried to fix the mis-usages…

In this blog post I am going to share some of my discoveries.

volatile foo used in place of std::atomic<foo>

As usual, the problem here is that volatile in C++ has nothing to do with synchronization/atomicity. (Ok, that’s not entirely true on all systems, but it’s true enough.)

There are at least two big offenders I found this way.

1: QQmlIncubationController::incubateWhile(volatile bool *)

The incubateWhile call takes a pointer to volatile bool. The idea is that the pointed boolean gets periodically checked by the incubator controller; another thread can tell the controller to stop incubating by flipping that bool.

To be honest, I’m not 100% convinced by the soundness of this API. For instance, it could’ve accepted a function object to poll, moving the problem of the eventual synchronization from another thread to the user’s domain. When in doubt, making it someone else’s problem is usually a good choice.

But I don’t want to redesign the QML engine APIs here, so I’ve just attempted a fix by porting to std::atomic<bool> instead.

The fix will appear in Qt 5.15.

2: QObjectPrivate::threadData

QObjectPrivate::threadData is a raw pointer to QThreadData (i.e. QThreadData *), basically storing the thread affinity of a given QObject, by holding a pointer to that thread’s private data.

The problem is that that variable is read and written by multiple threads without synchronization. For instance it’s written from here:

void QObjectPrivate::setThreadData_helper(QThreadData *currentData, QThreadData *targetData)
{
    ...
   
    // set new thread data
    targetData->ref();
    threadData->deref();
    threadData = targetData; // <-- the write
    
    ...
}

and potentially read from here:

void QCoreApplication::postEvent(QObject *receiver, QEvent *event, int priority)
{
    ...
    
    QThreadData * volatile * pdata = &receiver->d_func()->threadData; // <-- the read
    ...
}

These calls can happen from different threads, without synchronization — postEvent is documented to be thread-safe. Therefore, we have a data race, and data races are undefined behavior.

This ended up being super-tricky to fix (and I’m still not sure of it).

Sure, the idea is to make QObjectPrivate::threadData an atomic pointer to QThreadData, but then how to fix all the usage points?

The big struggle is that QObject is merely reentrant, and not thread safe, so most cases don’t require any synchronization (and can be turned into relaxed loads) because you’re not supposed to be modifying that variable by multiple threads without synchronization. Big exception: the cases that involve code paths that are documented to be thread-safe, like the postEvent code path above, which thus require acquire loads.

The current fix draft is sitting here.

The most interesting part is probably the old loop in postEvent(), trying to lock receiver‘s thread event queue:

    QThreadData * volatile * pdata = &receiver->d_func()->threadData;
    QThreadData *data = *pdata;
    if (!data) {
        // posting during destruction? just delete the event to prevent a leak
        delete event;
        return;
    }

    // lock the post event mutex
    data->postEventList.mutex.lock();

    // if object has moved to another thread, follow it
    while (data != *pdata) {
        data->postEventList.mutex.unlock();

        data = *pdata;
        if (!data) {
            // posting during destruction? just delete the event to prevent a leak
            delete event;
            return;
        }

        data->postEventList.mutex.lock();
    }

This loop used volatile once more to “cheat” synchronization: the code needs to get the threadData of an object and lock its event queue. Before that’s done, however, the affinity of the object could’ve been changed, so we need to unlock and try again. The idea is sound, but volatile does not help at all here as far as C++ is concerned; it seems to work because the compiler actually reloads the value of receiver‘s threadData, since its access happens through a volatile pointer.

As a side note: the same loop was completely missing from other places, like QCoreApplication::removePostedEvents, that also need to lock an object’s event queue (oops!). So I fixed that too, as a drive-by change…

volatile as a way to tell the compiler to… stop thinking

The QLibraryInfo class stores, amongst other things, the path to Qt’s own installed “parts” (plugins, translations, etc.). This path is hardcoded at build time; it’s one of the reasons why a Qt installation cannot be easily relocated.

In QLibraryInfo there’s a mysteriousconst char * volatile path” variable:

#ifndef QT_BUILD_QMAKE_BOOTSTRAP
    if (!fromConf) {
        const char * volatile path = 0;
        if (loc == PrefixPath) {
            path = getPrefix(
#ifdef QT_BUILD_QMAKE
                        group
#endif
                   );
        } else if (unsigned(loc) <= sizeof(qt_configure_str_offsets)/sizeof(qt_configure_str_offsets[0])) {
            path = qt_configure_strs + qt_configure_str_offsets[loc - 1];
#ifndef Q_OS_WIN // On Windows we use the registry
        } else if (loc == SettingsPath) {
            path = QT_CONFIGURE_SETTINGS_PATH;
#endif
# ifdef QT_BUILD_QMAKE
        } else if (loc == HostPrefixPath) {
            static const QByteArray hostPrefixPath = getHostPrefixFromHostBinDir().toLatin1();
            path = hostPrefixPath.constData();
# endif
        }

        if (path)
            ret = QString::fromLocal8Bit(path);
    }
#endif

What’s it for? It turned out to be a complete hack.

In order to achieve relocation of Qt libraries (for instance, to allow the binary Qt installers to install Qt in any path specified by the user), Qt can be built with a dummy installation path. The installer will then binary patch Qt, replacing this dummy path with the actual installation path.

What’s the hack, then? The hack is to prevent the compiler to aggressively inline the call to QString::fromLocal8Bit below, which involves a call to strlen(path).

With path set to a compile-time string literal, strlen(path) would also be fully evaluated at compile time, and that would make the binary patch not work — it would end up with path and its length not matching, and making fromLocal8Bit return broken data or crash horribly. This has actually happened and has been reported as QTBUG-45307.

How to stop the compiler from calling strlen at compile-time? There you have it, don’t use a “const char *“, use a “const char * volatile” (a volatile pointer to const char). I have no idea why it actually works!

Didn’t fix it, but left a comment for the future reader.

volatile for setjmp/longjmp protection

Now we go into the legitimate use cases for volatile. C (and thus C++) define the setjmp and longjmp library functions. They do non-local jumps: longjmp jumps to a setjmp done somewhere before in the call stack.

In C they’re used as a poor man’s exception handling mechanism — return from a deep call stack up directly to a caller, unwinding the stack in the process. Since it’s C, every object is trivially destructible, so there’s nothing special to do to perform the unwinding. In C++ performing a longjmp that would destroy non-trivially destructible objects yields undefined behavior.

volatile has an interesting interaction with setjmp/longjmp: it’s used to give well-defined semantics when using function local variables across the jump.

For instance, a snippet like this:

volatile int i = 0;
jmp_buf buf;

if (!setjmp(buf)) {
   // first time, enter here
   i = 42;
   longjmp(buf, 1);
}

// then go here after the longjmp
std::cout << i;

exhibits well-defined behavior (and prints 42) specifically because i is volatile. Without the volatile, i would have indeterminate value, and producing such an indeterminate value in this context is undefined behavior (I’m sensing a pattern here…). In other words: any local variable that is set after a setjmp and read after the longjmp must be declared volatile.

In layman’s terms: this volatile is telling the compiler something like “do not put i in a register and don’t optimize it out in any way; always keep it spilled on the stack, so it’s safe in case of a longjmp“.

Qt has a couple of places where there are setjmp/longjmp calls, namely its image handlers. PNG and JPEG use the respective C libraries (libpng and libjpeg), which seem to be designed with using setjmp/longjmp as a way to notify users of errors. That’s also where there are more usages of volatile.

PNG handler

In the case of the PNG image handler, a local variable was declared volatile, but that local was actually never accessed after the longjmp; so volatile wasn’t even needed in the first place. This has been fixed here.

JPEG handler

For the JPEG image handler, the situation was a bit more spicy. Sure thing, a local variable was again unnecessarily declared volatile (fix).

However the same code also was showing undefined behavior by not protecting some locals with volatile. For instance here, simplifying the code, we have this pattern:

JSAMPROW row_pointer[1];
row_pointer[0] = 0;
...
if (!setjmp(jerr.setjmp_buffer)) {
    ...
    row_pointer[0] = new uchar[cinfo.image_width*cinfo.input_components];
    ...
}
delete [] row_pointer[0];

The row_pointer local variable was not volatile; it was written after the setjmp (line 615) and then read again after the longjmp (line 716).

The fix in this case couldn’t be to simply mark it as volatile: the object in question is passed to libjpeg’s API, which simply want a pointer-to-object, and not a pointer-to-volatile-object (d’oh!). Marking an object with volatile, then const_cast‘ing volatileness away and accessing the object is an immediate undefined behavior, so don’t even think about doing it…

Also, libjpeg’s error handling requires the user to use setjmp/longjmp; therefore, the code had to keep the same structure (including the jumps). Basically, in libjpeg, one installs a custom error handling function; once libjpeg calls that handling function, it expects it not to return. So it must either exit the entire process, or longjmp out.

My fix was to make those variables non-local to the function calling setjmp, thus recovering well-defined behavior. You can read more about it in the fix’ commit message here.

longjmps, longjmps everywhere

Now the really interesting bit is that the very same code in the very same shape (and with the same very bug) is present everywhere.

For instance, it’s present in the equivalent GTK code in GDK that loads JPEG images here:

struct jpeg_decompress_struct cinfo;
...
if (sigsetjmp (jerr.setjmp_buffer, 1)) {
    ...
    jpeg_destroy_decompress (&cinfo);
    ...
}
...
jpeg_create_decompress (&cinfo);

Or in ImageMagick, here:

  struct jpeg_compress_struct
    jpeg_info;
    
  ...
  
  if (setjmp(error_manager.error_recovery) != 0)
    {
      jpeg_destroy_compress(&jpeg_info);
    }
    
  ...
  
  jpeg_create_compress(&jpeg_info);

It turns out that this kind of code comes all the way back from libjpeg version 6b’s own examples, dated 1998!

Those examples showcased the non-local return via setjmp/longjmp, and were affected by the same undefined behavior. You can still download them here (check out the sources for libjpeg 6b).

The historical libjpeg-turbo’s example also had the same problem. Comments are mine:

// 1. declare a non-volatile object
struct jpeg_decompress_struct cinfo;


if (setjmp(jerr.setjmp_buffer)) {
  // 3: and access it after the longjmp. 
  jpeg_destroy_decompress(&cinfo);
}

// 2. access it after the setjmp
jpeg_create_decompress(&cinfo);

I opened an issue asking for this to be fixed; the fix landed in this commit.

Conclusions

To conclude, let’s celebrate the longevity of this problem in all the codebases in the world. In perfect security-drama design: the most important part of an issue is its logo.

Therefore, here’s the official logo of the volatile handling in JPEG fiasco, of course made in 1998 style:

Thanks for reading!

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Getting rid of “volatile” in (some of) Qt appeared first on KDAB.

]]>
https://www.kdab.com/getting-rid-of-volatile-in-some-of-qt/feed/ 3 21592
KDSoap 1.9.0 released https://www.kdab.com/kdsoap-1-9-0-released/ https://www.kdab.com/kdsoap-1-9-0-released/#respond Mon, 24 Feb 2020 15:44:51 +0000 https://www.kdab.com/?p=22822 KD SOAP is a tool for creating client applications for web services. The release of 1.9.0 brings a fair number of improvements and fixes, as the following text describes. General C++11 is now required. Qt4 is still supported, but this is the last release with support for Qt4. rpath for Unix/Linux/macOS (#181) is now fixed. […]

The post KDSoap 1.9.0 released appeared first on KDAB.

]]>
KD SOAP is a tool for creating client applications for web services.

The release of 1.9.0 brings a fair number of improvements and fixes, as the following text describes.

General

  • C++11 is now required. Qt4 is still supported, but this is the last release with support for Qt4. rpath for Unix/Linux/macOS (#181) is now fixed.
  • cmake usage is improved by setting the install interface directories on the library targets (#196). You can just link to KDSoap::kdsoap and you’ll get the correct include paths now.
  • Conan has a new installing file (WIP – see conan folder).

Client-side

  • Support for selecting WS-Addressing namespace for send messages (#176) has been added
  • WS-Addressing spec compliance (#193) is fixed
  • Support for implementing SOAP-over-UDP clients (see KDSoapUdpClient) is fixed.

Server-side

WSDL parser / code generator changes, applying to both client and server side:

  • An override indicator to generated files has been added. This requires C++11 for users of generated files.
  • An option for loading PKCS12 certificates (#190) has been added.
  • All special handling of soap/encoding namespace is removed, which fixes the use of soapenc:string and soapenc:integer for instance (#179).
  • A compilation error due to missing QSharedPointer include in generated code (#170) is now fixed.

Examples

  • The holidays examples have been replaced with bank-lookup examples. Less fun, but the holidays web service no longer exists…

Download KDSoap

KDSoap on Github

KDABSoap Releases

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post KDSoap 1.9.0 released appeared first on KDAB.

]]>
https://www.kdab.com/kdsoap-1-9-0-released/feed/ 0 22822
How not to lose the alpha channel https://www.kdab.com/how-not-to-lose-the-alpha-channel/ https://www.kdab.com/how-not-to-lose-the-alpha-channel/#respond Thu, 20 Feb 2020 14:03:32 +0000 https://www.kdab.com/?p=22742 Working on color imagery for QiTissue recently, I realized we were accidentally losing the alpha channel in multiple places. For efficiency reasons, we keep colors in memory using the QRgb type, and convert that to/from QString for serialization purposes via QColor as needed. Here’s what I discovered about why that doesn’t work, and some ways […]

The post How not to lose the alpha channel appeared first on KDAB.

]]>
Working on color imagery for QiTissue recently, I realized we were accidentally losing the alpha channel in multiple places. For efficiency reasons, we keep colors in memory using the QRgb type, and convert that to/from QString for serialization purposes via QColor as needed. Here’s what I discovered about why that doesn’t work, and some ways I fixed it.

Firstly, be aware there is no QRgba in the Qt API . There is only QRgb, for 8bit color channels. It can hold an alpha value too, despite the lack of a trailing a in the type name. Then there is QRgba64 which uses 16bit per color channel. For our purposes, 8bit per channel is sufficient. So where do we lose the alpha channel, when QRgb can store it in principle?

The first pitfall is QColor(QRgb), which calls QColor::fromRgb internally. Both of these functions silently ignore the alpha channel of the passed QRgb value and assume the color is fully opaque. To get around this, you have to use QColor::fromRgba instead.

QColor toColor(QRgb colorWithAlpha)
{
    // all bad: input alpha ignored, set to fully opaque
    return QColor(colorWithAlpha);
    return colorWithAlpha;
    return QColor::fromRgb(colorWithAlpha);
    // good:
    return QColor::fromRgba(colorWithAlpha);
}

Then you’ll find that QColor::name() also ignores the alpha channel. Here, you have to use QColor::name(QColor::HexArgb) instead.

QString toString(const QColor &colorWithAlpha)
{
    // bad: input alpha ignored, output is e.g. #112233
    return colorWithAlpha.name();
    // good: output is e.g. #00112233
    return colorWithAlpha.name(QColor::HexArgb);
}

Thankfully, the QColor constructors and QColor::setName function, which parse a string color name, won’t ignore the alpha channel when you pass an ‘#AARRGGBB’ string.

However, there is one more pitfall: When you have a QColorDialog with QColorDialog::ShowAlphaChannel, and listen to its currentColorChanged signal and set a color with an alpha channel on it, a stray change signal notification will be emitted with a fully opaque color. The reason is that, internally, the color dialog will first set the RGB color components, and then in a second step will set the alpha component. But both will trigger a change notification, where the first step holds a fully opaque color. This should be fixed upstream, but thankfully for now it is easy to workaround by ignoring the signal while setting a color from the outside.

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post How not to lose the alpha channel appeared first on KDAB.

]]>
https://www.kdab.com/how-not-to-lose-the-alpha-channel/feed/ 0 22742
Qt World Summit 2019 talk videos are online https://www.kdab.com/qt-world-summit-2019-talk-videos-are-online/ https://www.kdab.com/qt-world-summit-2019-talk-videos-are-online/#respond Thu, 13 Feb 2020 10:00:07 +0000 https://www.kdab.com/?p=22295 Did you miss the past Qt World Summit? Were you there, but you couldn’t attend that talk or two that you really wanted to see because the conference was so, so packed with awesome content? Fear no more! We are glad to announce that the talks at the past Qt World Summit 2019 in Berlin […]

The post Qt World Summit 2019 talk videos are online appeared first on KDAB.

]]>
Did you miss the past Qt World Summit?

Were you there, but you couldn’t attend that talk or two that you really wanted to see because the conference was so, so packed with awesome content?

Fear no more! We are glad to announce that the talks at the past Qt World Summit 2019 in Berlin (or QtWS19, for the friends) have been video recorded and are now available online! You can now catch up with the latest news, improvements and best practices around Qt and its ecosystem, all from the comfort of your sofa office chair.

We have gathered all the talks given by KDAB engineers on this summary page, where you can find also more information about the contents of each talk and download the slides. For your convenience, we have also collected all of KDAB talks in a YouTube playlist:

The talks by other speakers are available for viewing in the Resource Center on www.qt.io.

Happy hacking!

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post Qt World Summit 2019 talk videos are online appeared first on KDAB.

]]>
https://www.kdab.com/qt-world-summit-2019-talk-videos-are-online/feed/ 0 22295