How to – KDAB https://www.kdab.com The Qt Experts Mon, 08 Jun 2020 12:42:26 +0000 en-US hourly 1 https://wordpress.org/?v=5.4.2 https://www.kdab.com/wp-content/uploads/stories/cropped-Favicon-32x32.png How to – KDAB https://www.kdab.com 32 32 32184545 Writing a Custom Qt 3D Aspect – part 2 https://www.kdab.com/writing-custom-qt-3d-aspect-part-2/ https://www.kdab.com/writing-custom-qt-3d-aspect-part-2/#comments Wed, 13 Dec 2017 10:49:45 +0000 https://www.kdab.com/?p=16605 Introduction In the previous article we gave an overview of the process for creating a custom aspect and showed how to create (most of) the front end functionality. In this article we shall continue building our custom aspect by implementing the corresponding backend types, registering the types and setting up communication from the frontend to […]

The post Writing a Custom Qt 3D Aspect – part 2 appeared first on KDAB.

]]>
Introduction

In the previous article we gave an overview of the process for creating a custom aspect and showed how to create (most of) the front end functionality. In this article we shall continue building our custom aspect by implementing the corresponding backend types, registering the types and setting up communication from the frontend to the backend objects. This will get us most of the way there. The next article will wrap up by showing how to implement jobs to process our aspect’s components.

As a reminder of what we are dealing with, here’s the architecture diagram from part 1:

Creating the Backend

One of the nice things about Qt 3D is that it is capable of very high throughput. This is achieved by way of using jobs executed on a threadpool in the backend. To be able to do this without introducing a tangled web of synchronisation points (which would limit the parallelism), we make a classic computer science trade-off and sacrifice memory for the benefit of speed. By having each aspect work on its own copy of the data, it can schedule jobs safe in the knowledge that nothing else will be trampling all over its data.

This is not as costly as it sounds. The backend nodes are not derived from QObject. The base class for backend nodes is Qt3DCore::QBackendNode, which is a pretty lightweight class. Also, note that aspects only store the data that they specifically care about in the backend. For example, the animation aspect does not care about which Material component an Entity has, so no need to store any data from it. Conversely, the render aspect doesn’t care about Animation clips or Animator components.

In our little custom aspect, we only have one type of frontend component, FpsMonitor. Logically, we will only have a single corresponding backend type, which we will imaginatively call FpsMonitorBackend:

class FpsMonitorBackend : public Qt3DCore::QBackendNode
{
public:
    FpsMonitorBackend()
       : Qt3DCore::QBackendNode(Qt3DCore::QBackendNode::ReadWrite)
       , m_rollingMeanFrameCount(5)
    {}

private:
    void initializeFromPeer(const Qt3DCore::QNodeCreatedChangeBasePtr &change) override
    {
        // TODO: Implement me!
    }

    int m_rollingMeanFrameCount;
};

The class declaration is very simple. We subclass Qt3DCore::QBackendNode as you would expect; add a data member to mirror the information from the frontend FpsMonitor component; and override the initializeFromPeer() virtual function. This function will be called just after Qt 3D creates an instance of our backend type. The argument allows us to get at the data sent from the corresponding frontend object as we will see shortly.

Registering the Types

We now have simple implementations of the frontend and backend components. The next step is to register these with the aspect so that it knows to instantiate the backend node whenever a frontend node is created. Similarly for destruction. We do this by way of an intermediary helper known as a node mapper.

To create a node mapper, just subclass Qt3DCore::QNodeMapper and override the virtuals to create, lookup and destroy the backend objects on demand. The manner in which you create, store, lookup and destroy the objects is entirely up to you as a developer. Qt 3D does not impose any particular management scheme upon you. The render aspect does some fairly fancy things with bucketed memory managers and aligning memory for SIMD types, but here we can do something much simpler.

We will store pointers to the backend nodes in a QHash within the CustomAspect and index them by the node’s Qt3DCore::QNodeId. The node id is used to uniquely identify a given node, even between the frontend and all available aspect backends. On Qt3DCore::QNode the id is available via the id() function, whereas for QBackendNode you access it via the peerId() function. For the two corresponding objects representing the component, the id() and peerId() functions return the same QNodeId value.

Let’s get to it and add some storage for the backend nodes to the CustomAspect along with some helper functions:

class CustomAspect : public Qt3DCore::QAbstractAspect
{
    Q_OBJECT
public:
    ...
    void addFpsMonitor(Qt3DCore::QNodeId id, FpsMonitorBackend *fpsMonitor)
    {
        m_fpsMonitors.insert(id, fpsMonitor);
    }

    FpsMonitorBackend *fpsMonitor(Qt3DCore::QNodeId id)
    {
        return m_fpsMonitors.value(id, nullptr);
    }

    FpsMonitorBackend *takeFpsMonitor(Qt3DCore::QNodeId id)
    {
        return m_fpsMonitors.take(id);
    }
    ...

private:
    QHash<Qt3DCore::QNodeId, FpsMonitorBackend *> m_fpsMonitors;
};

Now we can implement a simple node mapper as:

class FpsMonitorMapper : public Qt3DCore::QBackendNodeMapper
{
public:
    explicit FpsMonitorMapper(CustomAspect *aspect);

    Qt3DCore::QBackendNode *create(const Qt3DCore::QNodeCreatedChangeBasePtr &change) const override
    {
        auto fpsMonitor = new FpsMonitorBackend;
        m_aspect->addFpsMonitor(change->subjectId(), fpsMonitor);
        return fpsMonitor;
    }

    Qt3DCore::QBackendNode *get(Qt3DCore::QNodeId id) const override
    {
        return m_aspect->fpsMonitor(id);
    }
    
    void destroy(Qt3DCore::QNodeId id) const override
    {
        auto fpsMonitor = m_aspect->takeFpsMonitor(id);
        delete fpsMonitor;
    }

private:
    CustomAspect *m_aspect;
};

To finish this piece of the puzzle, we now need to tell the aspect about how these types and the mapper relate to each other. We do this by calling QAbstractAspect::registerBackendType() template function, passing in a shared pointer to the mapper that will create, find, and destroy the corresponding backend nodes. The template argument is the type of the frontend node for which this mapper should be called. A convenient place to do this is in the constructor of the CustomAspect. In our case it looks like this:

CustomAspect::CustomAspect(QObject *parent)
    : Qt3DCore::QAbstractAspect(parent)
{
    // Register the mapper to handle creation, lookup, and destruction of backend nodes
    auto mapper = QSharedPointer<FpsMonitorMapper>::create(this);
    registerBackendType<FpsMonitor>(mapper);
}

And that’s it! With that registration in place, any time an FpsMonitor component is added to the frontend object tree (the scene), the aspect will lookup the node mapper for that type of object. Here, it will find our registered FpsMonitorMapper object and it will call its create() function to create the backend node and manage its storage. A similar story holds for the destruction (technically, it’s the removal from the scene) of the frontend node. The mapper’s get() function is used internally to be able to call virtuals on the backend node at appropriate points in time (e.g. when properties notify that they have been changed).

The Frontend-Backend Communications

Now that we are able to create, access and destroy the backend node for any frontend node, let’s see how we can let them talk to each other. There are 3 main times the frontend and backend nodes communicate with each other:

  1. Initialisation — When our backend node is first created we get an opportunity to initialise it with data sent from the frontend node.
  2. Frontend to Backend — Typically when properties on the frontend node get changed we want to send the new property value to the backend node so that it is operating on up to date information.
  3. Backend to Frontend — When our jobs process the data stored in the backend nodes, sometimes this will result in updated values that should be sent to the frontend node.

Here we will cover the first two cases. The third case will be deferred until the next article when we introduce jobs.

Backend Node Initialisation

All communication between frontend and backend objects operates by sending sub-classed Qt3DCore::QSceneChanges. These are similar in nature and concept to QEvent but the change arbiter that processes the changes has the opportunity to manipulate them in the case of conflicts from multiple aspects, re-order them into priority, or any other manipulations that may be needed in the future.

For the purpose of initialising the backend node upon creation, we use a Qt3DCore::QNodeCreatedChange which is a templated type that we can use to wrap up our type-specific data. When Qt 3D wants to notify the backend about your frontend node’s initial state, it calls the private virtual function QNode::createNodeCreationChange(). This function returns a node created change containing any information that we wish to access in the backend node. We have to do it by copying the data rather than just dereferencing a pointer to the frontend object because by the time the backend processes the request, the frontend object may have been deleted – i.e. a classic data race. For our simple component our implementation looks like this:

struct FpsMonitorData
{
    int rollingMeanFrameCount;
};
Qt3DCore::QNodeCreatedChangeBasePtr FpsMonitor::createNodeCreationChange() const
{
    auto creationChange = Qt3DCore::QNodeCreatedChangePtr<FpsMonitorData>::create(this);
    auto &data = creationChange->data;
    data.rollingMeanFrameCount = m_rollingMeanFrameCount;
    return creationChange;
}

The change created by our frontend node is passed to the backend node (via the change arbiter) and gets processed by the initializeFromPeer() virtual function:

void FpsMonitorBackend::initializeFromPeer(const Qt3DCore::QNodeCreatedChangeBasePtr &change)
{
    const auto typedChange = qSharedPointerCast<Qt3DCore::QNodeCreatedChange<FpsMonitorData>>(change);
    const auto &data = typedChange->data;
    m_rollingMeanFrameCount = data.rollingMeanFrameCount;
}

Frontend to Backend Communication

At this point, the backend node mirrors the initial state of the frontend node. But what if the user changes a property on the frontend node? When that happens, our backend node will hold stale data.

The good news is that this is easy to handle. The implementation of Qt3DCore::QNode takes care of the first half of the problem for us. Internally it listens to the Q_PROPERTY notification signals and when it sees that a property has changed, it creates a QPropertyUpdatedChange for us and dispatches it to the change arbiter which in turn delivers it to the backend node’s sceneChangeEvent() function.

So all we need to do as authors of the backend node is to override this function, extract the data from the change object and update our internal state. Often you will then want to mark the backend node as dirty in some way so that the aspect knows it needs to be processed next frame. Here though, we will just update the state to reflect the latest value from the frontend:

void FpsMonitorBackend::sceneChangeEvent(const Qt3DCore::QSceneChangePtr &e)
{
    if (e->type() == Qt3DCore::PropertyUpdated) {
        const auto change = qSharedPointerCast<Qt3DCore::QPropertyUpdatedChange>(e);
        if (change->propertyName() == QByteArrayLiteral("rollingMeanFrameCount")) {
            const auto newValue = change->value().toInt();
            if (newValue != m_rollingMeanFrameCount) {
                m_rollingMeanFrameCount = newValue;
                // TODO: Update fps calculations
            }
            return;
        }
    }
    QBackendNode::sceneChangeEvent(e);
}

If you don’t want to use the built in automatic property change dispatch of Qt3DCore::QNode then you can disable it by wrapping the property notification signal emission in a call to QNode::blockNotifications(). This works in exactly the same manner as QObject::blockSignals() except that it only blocks sending the notifications to the backend node, not the signal itself. This means that other connections or property bindings that rely upon your signals will still work.

If you block the default notifications in this way, then you need to send your own to ensure that the backend node has up to date information. Feel free to subclass any class in the Qt3DCore::QSceneChange hierarchy and bend it to your needs. A common approach is to subclass Qt3DCore::QStaticPropertyUpdatedChangeBase, which handles the property name and in the subclass add a strongly typed member for the property value payload. The advantage of this over the built-in mechanism is that it avoids using QVariant which does suffer a little in highly threaded contexts in terms of performance. Usually though, the frontend properties don’t change too frequently and the default is fine.

Summary

In this article we have shown how to implement most of the backend node; how to register the node mapper with the aspect to create, lookup and destroy backend nodes; how to initialise the backend node from the frontend node in a safe way and also how to keep its data in sync with the frontend.

In the next article we will finally make our custom aspect actually do some real (if simple) work, and learn how to get the backend node to send updates to the frontend node (the mean fps value). We will ensure that the heavy lifting parts get executed in the context of the Qt 3D threadpool so that you get an idea of how it can scale. Until next time.

The post Writing a Custom Qt 3D Aspect – part 2 appeared first on KDAB.

]]>
https://www.kdab.com/writing-custom-qt-3d-aspect-part-2/feed/ 5 16605
Writing a Custom Qt 3D Aspect – part 1 https://www.kdab.com/writing-custom-qt-3d-aspect/ https://www.kdab.com/writing-custom-qt-3d-aspect/#comments Sat, 25 Nov 2017 09:00:48 +0000 https://www.kdab.com/?p=16399 Introduction Qt 3D has a flexible and extensible architecture that allows us to easily add our own new functionality to it without disrupting the existing features. The functionality of Qt 3D is divided among so-called aspects, each of which encapsulates a particular subject domain such as rendering, input, or animation. This short series of articles […]

The post Writing a Custom Qt 3D Aspect – part 1 appeared first on KDAB.

]]>
Introduction

Qt 3D has a flexible and extensible architecture that allows us to easily add our own new functionality to it without disrupting the existing features. The functionality of Qt 3D is divided among so-called aspects, each of which encapsulates a particular subject domain such as rendering, input, or animation.

This short series of articles will walk you through the process of adding a new aspect that provides component types and behaviour for a new domain not covered by Qt 3D out of the box. For this example we have chosen to implement an aspect that allows calculating running means of the frame rate. Of course this could legitimately be added to the renderer, but it’s simple enough that it makes a nice example for our purposes today. The full source code for the example is available for download.

Overview

The application that we will build looks like this. It’s a very simple Qt 3D scene and the window shows the current mean frame rate.

Custom Aspect Screenshot

There are a few parts that we need to consider when adding such new functionality (see the diagram):

  • Aspect — Responsible for orchestrating any jobs that need executing each frame and for managing the storage of any backend objects.
  • Components — Provided by your library/application are what an Entity will aggregate to give it new behaviour. The components are the main API you will need to create.
  • Nodes — Just as for components except that subclasses of QNode typically provide supporting data required by a component. For example QAnimationClipLoader is a node that provides animation key frame data to a QClipAnimator component.
  • Backend Nodes — The backend counterparts to any frontend components or nodes provided by your library/application. These are the objects typically processed by jobs running on the threadpool as dictated by the aspect itself.
  • Mapper — Custom mappers are registered with the aspect and are responsible for creating, fetching, and destroying backend nodes on demand. The mapper is used by QAbstractAspect and QAspectEngine to synchronise lifetimes of the frontend and backend objects.
  • Jobs — Created and scheduled by the aspect and which process the backend nodes. Jobs may also send events to the frontend nodes and components if properties change.
  • Change Arbiter — Responsible for delivering events between and among the frontend and backend objects. No need to do anything with this but be aware of its existence.

Adding the Aspect

Writing the initial aspect itself is really trivial. Just subclass QAbstractAspect and register it with the QAspectEngine:

class CustomAspect : public Qt3DCore::QAbstractAspect
{
    Q_OBJECT
public:
    explicit CustomAspect(QObject *parent = nullptr)
        : Qt3DCore::QAbstractAspect(parent)
    {}

protected:
    QVector<Qt3DCore::QAspectJobPtr> jobsToExecute(qint64 time) override
    {
        qDebug() << Q_FUNC_INFO << "Frame time =" << time;
        return {};
    }
};
int main(int argc, char **argv)
{
    QGuiApplication app(argc, argv);
    Qt3DExtras::Quick::Qt3DQuickWindow view;

    view.engine()->aspectEngine()->registerAspect(new CustomAspect);

    view.setSource(QUrl("qrc:/main.qml"));
    view.show();
    return app.exec();
}

QAbstractAspect has a few more virtuals that you can override to do initialisation and cleanup should you need them. However, for this simple example all we need to do is to implement the jobsToExecute() virtual. Later we will use this to schedule a job to execute on the threadpool each frame, but for now we just output some debug text to return an empty vector (no jobs to run). Note that these virtual functions will only be called once you have registered the aspect with the QAspectEngine (see above) so that it is part of the simulation. We will return and complete the aspect a little later.

The FpsMonitor Component

We wish to add functionality to report on the mean frame rate averaged over a given number of frames. With this is mind we might come up with an API something like this:

class FpsMonitor : public Qt3DCore::QComponent
{
    Q_OBJECT
    Q_PROPERTY(int rollingMeanFrameCount READ rollingMeanFrameCount WRITE setRollingMeanFrameCount NOTIFY rollingMeanFrameCountChanged)
    Q_PROPERTY(float framesPerSecond READ framesPerSecond NOTIFY framesPerSecondChanged)

public:
    explicit FpsMonitor(Qt3DCore::QNode *parent = nullptr);

    float framesPerSecond() const;
    int rollingMeanFrameCount() const;

public slots:
    void setRollingMeanFrameCount(int rollingMeanFrameCount);

signals:
    void framesPerSecondChanged(float framesPerSecond);
    void rollingMeanFrameCountChanged(int rollingMeanFrameCount);

private:
    float m_framesPerSecond;
    int m_rollingMeanFrameCount;
};

Note that we use a declarative, property-based API so that this class can easily be used from QML as well as from C++. The property rollingMeanFrameCount is a regular read-write property and the implementation of the setter and getter functions are completely standard. This property will be used to control the number of frames over which we calculate the moving average frame rate. The framesPerSecond property is a read-only property which will later be set from a job that we will write to process FpsMonitor components on the Qt 3D threadpool.

Creating a Small Test Application

Before we can utilise our custom component from QML, we must register the type with the QML type system. This is a simple one-liner that we can add to our main function:

qmlRegisterType<FpsMonitor>("CustomModule", 1, 0, "FpsMonitor");
rootContext->setContextProperty("_window", &view);

where we also took the opportunity to export the window to the QML context too (we’ll need that in a moment).

Once that is done, we can import the CustomModule QML module and make use of the FpsMonitor component just as we would for any of the built in types.

    Entity {
        components: [
            FpsMonitor {
                rollingMeanFrameCount: 20
                onFramesPerSecondChanged: {
                    var fpsString = parseFloat(Math.round(framesPerSecond * 100) / 100).toFixed(2);
                    _window.title = "CustomAspect: FPS = " + fpsString
                            + " (" + rollingMeanFrameCount + " frame average)"
                }
            }
        ]
    }

Of course at this point, the FpsMonitor doesn’t do much except for setting the value of the rollingMeanFrameCount property. Once we complete the backend, the above code will update the window title to show the current mean frame rate and the number of frames used to calculate it.

In the next part we will implement the corresponding backend node for FpsMonitor and make sure it gets created and destroyed on demand and set up communications between the frontend and backend.

The post Writing a Custom Qt 3D Aspect – part 1 appeared first on KDAB.

]]>
https://www.kdab.com/writing-custom-qt-3d-aspect/feed/ 1 16399
Lots of lights: Generating cities https://www.kdab.com/lots-lights-generating-cities/ https://www.kdab.com/lots-lights-generating-cities/#respond Thu, 28 Sep 2017 11:35:05 +0000 https://www.kdab.com/?p=15781 Sometimes data visualization might call for more than a graph – you need to visualize complex data, such as that generated by city lighting, in three dimensions to get the full effect. KDAB decided to put together a showcase for the Qt World Summit that allowed us to demonstrate Qt 3D’s capabilities as a performant […]

The post Lots of lights: Generating cities appeared first on KDAB.

]]>
Sometimes data visualization might call for more than a graph – you need to visualize complex data, such as that generated by city lighting, in three dimensions to get the full effect.

KDAB decided to put together a showcase for the Qt World Summit that allowed us to demonstrate Qt 3D’s capabilities as a performant next-generation graphics engine, which can draw thousands of lights and objects simultaneously. This also enabled us to show what modern technologies like Qt 3D can achieve when paired with OpenGL, C++ and custom tooling.

What would show a huge amount of dynamic lighting in action better than a city at night?  We couldn’t answer that, so, to avoid the tedium of building a virtual 3D world to get us going, we imported data from the freely accessible OpenStreetMap service. OpenStreetMap is a great community and a source of free and open mapping information. Additionally, it allows for the export of street and building data to XML format which can be further post-processed.

Buildings

Our data was filtered according to OpenStreetMap keys, and building locations were extracted as polygonal shapes. From these shapes – according to specific rules – Houdini was used to generate a geometric three-dimensional object out of a single polygon per house. Along with that, texture coordinates were created automatically, too. Texture coordinates tell the graphics card how to map an image – commonly referred to as a “texture” – onto a geometric object much like a present is wrapped in paper.

The OpenStreetMap data contains a wealth of information, such as road surface type, speed limits and the number of floors per building. Our algorithm we used calculates the building’s height according to this data and produces an approximation. We used additional properties to provide rooftop variations and create random materials. To keep the performance high, we decided that all buildings should share the same, automatically generated texture atlas. This means that all images we used are combined into a single one so the graphics card has to load and hold just one image in memory for the whole scene.

Traffic

What would a city be without a little traffic? As a challenge, cars should roam the streets of our virtual city, we decided, and of course provide moving light sources. For that to work, we exported OpenStreetMap path data to a custom format and uploaded it to the GPU using a texture. Cars were then animated, instanced and drawn purely on the system’s graphics card.

 

Putting it together

Finally, we combined traffic, buildings and a massive amount of lights into a single scene using Qt 3D. In the final demo, a simple QML interface provides the ability to switch different features on and off as well as switch between a real-world city and a completely computer-generated one. Very little effort was necessary to connect the C++ logic to the interfaces, as QML makes those kind of data bindings very easy.

 

Make sure to see for yourself and visit our booth at the 2017 Qt World Summit!

The post Lots of lights: Generating cities appeared first on KDAB.

]]>
https://www.kdab.com/lots-lights-generating-cities/feed/ 0 15781
Clang Tidy, part 2: Integrate qmake and other build systems using Bear https://www.kdab.com/clang-tidy-part-2-integrate-qmake-and-other-build-systems-using-bear/ https://www.kdab.com/clang-tidy-part-2-integrate-qmake-and-other-build-systems-using-bear/#comments Tue, 25 Jul 2017 09:20:13 +0000 https://www.kdab.com/?p=15420 Clang-Tidy, part 2: Modernize your source code using C++11/C++14. The blog series introduces you to the clang-tidy utility and how to use it to automatically refactor C++ source code and integrate it with your build system.

The post Clang Tidy, part 2: Integrate qmake and other build systems using Bear appeared first on KDAB.

]]>
Introduction

This article is part of a blog series about Clang Tidy. In the previous article we learned about the general usage of Clang Tidy to automatically refactor source code for projects using the CMake build system. In this particular episode we’ll discuss using Clang Tooling on projects using different build systems with the help of Bear.

Motivation: So you want to use Clang Tooling on your project — but what if your particular project of interest is using another build system such as qmake, a build system predominantly used in the Qt world? There is a way to leverage Clang Tooling on any build system out there by just using a small helper tool.

Introducing Bear

Bear (Build EAR) is a tool that generates a compilation database for clang tooling. As we’ve learned in the previous article in this blog series, all that Clang Tooling needs to know about your project is a JSON Compilation Database, which is basically a replay of all the compilations (i.e. compiler invocations) happening during a project build. While the CMake build system can generate one out of the box, other build systems usually require additional work to get one.

Bear is a generic way to generate the compilation database during the build process, i.e. when running make for any build system. The concept behind Bear is that it intercepts the exec calls done by the build tool (i.e. make) and uses the command-line arguments passed to the compiler and stores them in the compilation database. Think of a strace-like tool filtering for exec system calls.

Unfortunately, Bear currently only works on Unix-based systems (FreeBSD, GNU/Linux and macOS), since it’s heavily relying on the LD_PRELOAD mechanism on *BSD/Linux and its macOS counterpart to intercept the system calls.

Getting Bear

On Ubuntu (since at least 14.04 LTS) or Debian (since at least jessie):

sudo apt-get install bear

On any other distributions not providing a package:

git clone https://github.com/rizsotto/Bear
cd Bear
cmake .; make; make install

Refer to the official documentation for more information.

Using Bear

This section will show how to use Bear with different build systems. Always remember that Bear is just a helper tool used to intercept the build tool used. The end result of Bear is a compilation database which can be consumed by Clang Tooling later.

Using Bear with qmake

The qmake build system is predominantly used in the Qt world. While this little tutorial is specific to qmake, the same workflow can be applied to any other build system.

Let’s use Bear on an example project called qtremoteobjects, which is a Qt module hosted on Qt Git.

<checkout qtremoteobjects>
cd qtremoteobjects
mkcd build

PS: Tip: mkcd just creates a directory and cd’s into it — source.

At this point we want to set up the build system. Note we cannot just call qmake .. alone, since on Linux this will use the linux-gcc spec by default. The compile flags used in the Makefiles generated using this spec won’t be compatible with clang, thus we need to resort to using the linux-clang mkspec.

qmake -spec linux-clang ..

This generated the root Makefile. Now we have to run make, but since we want to generate the compilation database on the fly as well, we have to “preload” bear.

bear make

After bear make finished, you’ll end up with a file called compile_commands.json in the current working directory. Similarly as if you would have let CMake generate this compilation database for you.

Let’s check what’s inside:

head compile_commands.json
[
{
"command": "c++ -c -pipe -g -std=c++1z -fvisibility=hidden -fvisibility-inlines-hidden -fno-exceptions -Wall -W -D_REENTRANT -fPIC -DQT_BUILD_REMOTEOBJECTS_LIB ... -I/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects ... -o .obj/qconnection_local_backend.o /home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_local_backend.cpp",
"directory": "/home/kfunk/devel/src/qt5.8/qtremoteobjects/build/src/remoteobjects",
"file": "/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_local_backend.cpp"
},
{
"command": "c++ ... /home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_tcpip_backend.cpp",
...

Looking good.

Note: If you interrupt bear make, a consecutive run of bear make will just *override this compile_commands.json file. In other words, you’ll lose the information inside that file from the previous make run, of targest which have been compiled already. Thus in order to get a complete compile_commands.json, clear the build directory first, then invoke bear ... as before again.

Using Bear with any other build tool (for completeness sake)

This is what you need:

bear <build tool>

The build tool could be anything — it can even be a totally different build system which does not leverage traditional build tools such as make/ninja to invoke the compiler.

Running clang-tidy as usual

Next, as soon as we have a valid compile_commands.json file, we can run run clang-tidy as we learned in part 1 of this blog series, using the run-clang-tidy script:

run-clang-tidy-3.9.py -clang-tidy-binary clang-tidy-3.9 -clang-apply-replacements-binary clang-apply-replacements-3.9 -header-filter='.*' -checks='-*,modernize-use-auto' -fix
...
/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectnode.cpp:419:5: warning: use auto when initializing with new to avoid duplicating the type name [modernize-use-auto]
QConnectedReplicaPrivate *rp = new QConnectedReplicaPrivate(name, meta, q);
^
auto
/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectnode.cpp:440:9: warning: use auto when initializing with new to avoid duplicating the type name [modernize-use-auto]
QInProcessReplicaPrivate *rp = new QInProcessReplicaPrivate(name, meta, q);
^
auto
...

We’ll end up with a couple of changes in the source directory which we can commit afterwards.

Troubleshooting

Unfortunately there are a couple of issues when running Bear which one needs to take into account when using it.

Problem with PCHs from a different compiler

If we had used qmake -spec linux-gcc ... before, the clang-tidy invocation would have failed. You’d run into issues such as this one here:

error: no suitable precompiled header file found in directory '.pch/Qt5RemoteObjects.gch' [clang-diagnostic-error]

In other words: Clang tries to read a precompiled header (PCH) generated by GCC which it isn’t capable of => compilation fails.

Solution: Use qmake -spec linux-clang ... in order to use the correct compiler from the start.

Problem with PCH from a different compiler version

If we had used qmake ... without CONFIG-=precompile_header and clang++ is of a different version than the clang-tidy executable, we would have run into this issue:

error: PCH file built from a different branch ((tags/RELEASE_400/rc1)) than the compiler ((tags/RELEASE_391/rc2)) [clang-diagnostic-error]

Solution: Either make sure the clang-tidy version and the clang version match, or just disable precompiled headers in qmake by passing CONFIG-=precompile_header to it.

clang-apply-replacements executable segfaulting

Problem: When running clang-apply-replacements on a set of suggested fixes (stored as .yml files in a tmp directory) — which is done as part of the run-clang-tidy script — one may encounter invalid file references. When encountering the same invalid file references a second time, any currently released version of clang-apply-replacements will crash.

Example invocation:

% clang-apply-replacements /tmp/tmpIqtp7m
...
Described file '.moc/../../../../../../src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectabstractitemmodelreplica_p.h' doesn't exist. Ignoring...
Described file '.moc/../../../../../../src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectpendingcall.h' doesn't exist. Ignoring...
zsh: segmentation fault  clang-apply-replacements /tmp/tmpIqtp7m

The clang-apply-replacements executable crashes. Note I haven’t investigated why the invalid file references happen in the first place, but I’ve fixed the crash in clang-apply-replacements upstream for good.

Solution: Apply this patch when you have a source checkout of the Clang Tooling.

Unfortunately that means you’ll have to compile the clang-apply-replacements project yourself in order to take advantage of the patch for now or wait for a new Clang release.

Conclusion

With Bear one can use Clang Tooling on projects based on other build systems such as CMake. It’s a simple strace-like tool which intercepts the build tool and logs the way the compiler is invoked in a compilation database file, which can later be consumed by clang-tidy and other Clang Tooling helpers.

Any thoughts, questions?

The post Clang Tidy, part 2: Integrate qmake and other build systems using Bear appeared first on KDAB.

]]>
https://www.kdab.com/clang-tidy-part-2-integrate-qmake-and-other-build-systems-using-bear/feed/ 7 15420
A tale of 2 curves https://www.kdab.com/tale-2-curves/ https://www.kdab.com/tale-2-curves/#respond Tue, 20 Jun 2017 12:58:26 +0000 https://www.kdab.com/?p=15236 As my first subject for this animation blog series, we will be taking a look at Animation curves. Curves, or better, easing curves, is one of the first concepts we are exposed to when dealing with the subject of animation in the QML space. What are they? Well, in simplistic terms, they are a description […]

The post A tale of 2 curves appeared first on KDAB.

]]>
As my first subject for this animation blog series, we will be taking a look at Animation curves.

Curves, or better, easing curves, is one of the first concepts we are exposed to when dealing with the subject of animation in the QML space.

What are they?

Well, in simplistic terms, they are a description of an X position over a Time axis that starts in (0 , 0) and ends in (1 , 1). These curves are to be applied to a numerical property. For example, say we want to move the value of X from 0 to 250 over a time period of, say, 5000 ms in a linear fashion (constant speed), we would do something like:

NumberAnimation {
        .....
        property: "x"
        duration: 5000
        easing.type: Easing.Linear
        from:0
        to:250
        .....
    }

And it works relatively well – for reference you can see most of them here.

However there is a limited set of curves, and that poses a problem when trying to achieve more complex animation patterns, probably because no single curve in the above list reflects that complex animation we are trying to describe. Usually the most common solution will be to reach those by using sequential and parallel animations. This creates, in my experience, a lot of trial and error code that we tend not to be particularly proud of. This is mostly because even describing simple enough animations as, say, kicking a ball that jumps, rolls, bounces and stops, requires a lot of code.  We need to describe the y movement in a sequential animation, an x movement in another parallel animation and the rolling phase in yet another animation stage. The code required to generate this is big, clunky and hard to read, given the nature of multiple parallel and sequential animations running under the same master animation.

On top of that, sequential animations suffer from another, more fundamental limitation: they mostly rely on a “stop” of the motion in order to “start” the next movement when jumping from one curve to the next. Not using this will result in non continuities in the speed variation, spikes, and this will look unnatural (a non smooth speed variation indicates an infinite acceleration, and only infinitely rigid materials can achieve that when colliding with other similar materials, which doesn’t happen in nature).   

The speed can be seen as the angle of the curve on each point in the chosen easing curve. The steeper the curve, the faster the movement, so when we have a sequential animation, it is important that, when transitioning from one curve to the next, the end angle of one curve is the same as the start angle for the next. Now, given the limited set of curves we are offered, we will have a hard time matching those, other than for curves that fade/start into a flat angle, aka the motion stops and restarts from one motion to the next.

How can we circumvent that limitation?

Can’t I just use my custom curves?

 

The reality is that you can, and it is relatively simple to do. Let me introduce you to the QML easing Curve Editor: one simple application that can be found in your bin Qt folder, named qmleasing. This helps you create such curves.

 

Here you can see an example of such a curve being created. Notice the ticked smooth flag that enforces that the curve is smooth at the junction of the 2 bezierCurves. This allows us to create far more complex animations, as natural as we can achieve, using this UI. The practical usage of this is achieved by copy pasting the simplified bezier description “[0.33,-0.00625,0.34,1.05,0.52,0.916,0.69,0.778,0.66,0.997,1,1]” into an easing.bezierCurve:

 NumberAnimation {
        .....
        property: "x"
        duration: 5000
        easing.bezierCurve: [0.33,-0.00625,0.34,1.05,0.52,0.916,0.69,0.778,0.66,0.997,1,1]
        from:0
        to:250
        .....
    }

The result will be something like this:

 

The first red square is moving according to a linear animation, and the second blue one is moving according to a, still relativity simple, new curve that we made on the qmleasing application. However, as you explore this, it quickly becomes obvious that what you win in freedom, you lose in control. The nature of the curve time/position is a bit counter-intuitive to your (at least my) thinking of position/speed.

The curve is and needs to be a description of a variable (numerical property) from an initial value (0) to a final value (1) over a time period also (0) to (1).  This curve gets scaled and applied to whatever the time frame you chose in milliseconds and the start and end of the value that you are going to change, so when and if you do loop-like bezier curves, they will make no sense, as time “tends” to just move forwards… You need to think in terms of time and not speed, which means you look at the curve and divide it into multiple columns where the points in the curve represent the scaled position according to your to: and from:.  In fact any animation coordination must be thought of in these terms: you look at the time an “event” occurs and map that as you see fit in the curve. Unfortunately this software lacks the visual tools to do this in a precise fashion .

but…

Hmmm, beziers I can do those.

 

As I was investigating this, it became obvious to me that this opened up room for serious exploitation and manipulation of mathematically/physics-based generated curves, and this will be the subject of the next post. I leave a teaser here in the form of the expected end result…

So, see you soon in the continuation of this specific subject of animation curve manipulation.

 

As usual leave your comments below.

The post A tale of 2 curves appeared first on KDAB.

]]>
https://www.kdab.com/tale-2-curves/feed/ 0 15236
Clang-Tidy, part 1: Modernize your source code using C++11/C++14 https://www.kdab.com/clang-tidy-part-1-modernize-source-code-using-c11c14/ https://www.kdab.com/clang-tidy-part-1-modernize-source-code-using-c11c14/#comments Thu, 16 Mar 2017 09:00:00 +0000 https://www.kdab.com/?p=14309 This blog series will introduce the clang-tidy utility from the Clang/LLVM project and show how to use it to automatically refactor C++ source code and integrate with your build system, as well as how to use the tool on other platforms than Unices. Motivation: The joy of legacy code bases C++11 added a significant amount of new C++ language features […]

The post Clang-Tidy, part 1: Modernize your source code using C++11/C++14 appeared first on KDAB.

]]>
This blog series will introduce the clang-tidy utility from the Clang/LLVM project and show how to use it to automatically refactor C++ source code and integrate with your build system, as well as how to use the tool on other platforms than Unices.

Motivation: The joy of legacy code bases

C++11 added a significant amount of new C++ language features which to date still aren’t used to their full extent. The most visual ones are for sure auto, override, Lambda expressions, range-based for, uniform initialization syntax, you name it. While C++11 is now several years old already, there’s still lots of code bases which don’t use any of the new language features, be it by policy from management, or by pure laziness to avoid the porting effort from the developer side. Clang-Tidy from the LLVM compiler infrastructure project is here to at least overcome the latter, to allow automatic refactoring of your source code so it uses the new language features.

Use override, anyone?

Now, what happens if you already maintain a fairly large code base which is still compiles under C++03 mode but you do want to embrace using C++11 in the future, with all the helpful features it has? You’ll likely have lots and lots of code similar to this:

struct Base {
    virtual void reimplementMe(int a) {}
};
struct Derived : public Base  {
    virtual void reimplementMe(int a) {}
};

So far you of course always re-implemented the correct base-class method, because you’ve extensively tested your code via unit tests! Of course you did. Thus the code is fine but you would now like to move on. You’d like to add the override specifier to every single re-implemented method in your code base, but of course without hiring a trainee which goes through the code line-by-line and adds them manually.

Just to stress that adding override indeed serves a purpose, we’ve just recently fixed a bug in Qt 3D where we were not overloading the correct base-class method. With the specifier added earlier, we would have noticed instantly, after recompilation.

We’ll take the missing-override example further to explain basic usage of clang-tidy.

Clang-Tidy to the rescue!

Clang-Tidy is a clang-based C++ linter tool which provides a shell executable called clang-tidy as the main entry point. It is an extensible framework for diagnosing typical programming errors, or style issues — generally anything which can be detected during static analysis of the code. The real benefit of the tool is that it additionally allows to automatically refactor the source code by applying fixits each individual issue may provide. It is heavily plugin-based and comes with a useful set of plugins out of the box, which we’re going to discuss in the next paragraph.

LLVM logo -- home of clang-tidy

Clang-Tidy is a tool developed and maintained by the Clang/LLVM community.

Setup

When running Linux, clang-tidy is usually easy to get via your distribution’s package manager. On Ubuntu Linux for instance, installing it is as easy as running the following commands:

% sudo apt-get install clang-tidy

We’ll be discussing installing the tool on other platforms than Linux in one of the upcoming blog posts.

Note: We recommend you to always install the latest version (at the time of writing, the version based on Clang/LLVM 3.9 is recommended), as the number of available plugins/checkers varies greatly from version to version and grows constantly.

Introduction

Note: In this blog post, clang-tidy-3.9 was used

A typical invocation of the command-line tool looks like this:

% clang-tidy test.cpp -- -Imy_project/include -DMY_DEFINES ...

Executing it like this, the tool will print a bunch of warnings and notes (if applicable), in exactly the same way Clang/GCC provide diagnostics, too.

Clang-Tidy is a useful static analysis tool on its own with lots of different available checkers, this, however, is not the focus of this blog post. We’d rather want to leverage the tool’s powerful refactoring capabilities to modernize our source code.

Listing available checkers

Running the tool without any specific command-line parameters will run the default set of checkers enabled by the utility. Let’s check what other checkers it has to offer (by passing –checks=’*’ to see them all), and specifically grep for the ones with modernize in their names. Those checkers advocate usage of modern language constructs:

$ clang-tidy --list-checks -checks='*' | grep "modernize"
    modernize-avoid-bind
    modernize-deprecated-headers
    modernize-loop-convert
    modernize-make-shared
    modernize-make-unique
    modernize-pass-by-value
    modernize-raw-string-literal
    modernize-redundant-void-arg
    modernize-replace-auto-ptr
    modernize-shrink-to-fit
    modernize-use-auto
    modernize-use-bool-literals
    modernize-use-default
    modernize-use-emplace
    modernize-use-nullptr
    modernize-use-override
    modernize-use-using

Impressive list of options already, isn’t it? Clang-Tidy indeed ships some interesting checkers out of the box (as of Clang/LLVM 3.9), with the list growing constantly from release to release.

The names of the checkers are pretty much self-explanatory (e.g. modernize-use-auto will embrace using auto where applicable), but if you want to explore what each of them means, please consult the list of checkers on the clang-tidy homepage:

To show how the tool is being used let’s focus on the modernize-use-override checker, as it’s the most applicable and most uncontroversial checker.

Refactoring a single file

Our override example again:

struct Base {
    virtual void reimplementMe(int a) {}
};
struct Derived : public Base  {
    virtual void reimplementMe(int a) {}
};

Running clang-tidy on the example (this time with the modernize-use-override checker enabled):

% clang-tidy-3.9 -checks='modernize-use-override' test.cpp -- -std=c++11
1 warning generated.
/home/kfunk/test.cpp:5:18: warning: prefer using 'override' or (rarely) 'final' instead of 'virtual' [modernize-use-override]
    virtual void reimplementMe(int a) {}
                 ^
                                      override

Alright. So it noticed that Derived::reimplementMe(int) overrides a base-class method but is lacking the override specifier! Now we could add that manually… or just let the tool do it for us by passing -fix!

Running it on the example (with modernize-use-override checker & fix-its enabled):

% clang-tidy-3.9 -checks='modernize-use-override' -fix test.cpp -- -std=c++11
1 warning generated.
/home/kfunk/test.cpp:5:18: warning: prefer using 'override' or (rarely) 'final' instead of 'virtual' [modernize-use-override]
    virtual void reimplementMe(int a) {}
                 ^
                                      override
/home/kfunk/test.cpp:5:5: note: FIX-IT applied suggested code changes
    virtual void reimplementMe(int a) {}
    ^
/home/kfunk/test.cpp:5:38: note: FIX-IT applied suggested code changes
    virtual void reimplementMe(int a) {}
                                     ^
clang-tidy applied 2 of 2 suggested fixes.

Clang-tidy applied the fix-it, and inserted override after the method declaration in line 5. Done!

A couple of notes

There are a few things worth mentioning:

  • Not all checkers of clang-tidy actually carry fix-its, but the ones starting with modernize all do.
  • You can use fix-its from multiple checkers at the same time (consider -checks=’modernize-use-override,modernize-use-auto’ -fix)
  • Running clang-tidy invokes the complete Clang compiler frontend, thus will need some time to complete
  • Refactoring results from clang-tidy are perfectly accurate, due to the fact it’s backed by a fully-fledged C++ parser

Refactoring a complete project (CMake-based)

So far we’ve run clang-tidy on a single, standalone file only. What happens if you have a more complex project setup, with lots of files, all with custom compile flags? Clang-tidy always operates on a single file, or rather, translation unit. We can help the tool to figure out the correct compile flags for each translation unit we compile in our project. The most convenient way to run it is with a compile command database. CMake can automatically generate one, and once a compile_commands.json is in place and a working version of clang-tidy is in PATH the entire code base can be analyzed with the run-clang-tidy.py script (usually shipped as part of the installation). If not you can simply download it here.

Note: It is highly recommended to use run-clang-tidy.py to run clang-tidy on a whole project, since it’ll run the tool multiple times in parallel and makes sure concurrent executions don’t interfere with each other (e.g. by avoiding modifying a shared header in parallel and in turn generating broken code).

Generating a compile_commands.json file

For generating the compile_commands.json file in a CMake-based project, just run:

% cd my-cmake-based-project
% mkdir build
% cd build
% cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..

Use script to run clang-tidy

Now for running the tool with the default checks on each translation unit in the project, simply invoke the run-clang-tidy script inside the directory with the compile_commands.json file:

% run-clang-tidy.py

As seen before, this will not modify anything so far, as we’ve run clang-tidy with just the default checks enabled. To e.g. run the modernize-use-override check on all translations units and actually refactor all your code, this invocation is needed:

% run-clang-tidy.py -header-filter='.*' -checks='-*,modernize-use-override' -fix

That’s it. clang-tidy will now be invoked on each translation unit in your project and will add overrides where missing. The parameter -header-filter=’.*’ makes sure clang-tidy actually refactors code in the headers being consumed in the translation unit. The parameter checks=’-*,…’ makes sure all default checks are disabled.

Note that the fixes are only applied in once run-clang-tidy has finished! The script will only record the changes to-be-performed and applies them all at once at the end.

Running other checkers

Again, the modernize-use-override is just an example, clang-tidy has lots of other checkers which are useful. Another super useful one is the modernize-use-nullptr checker, which transforms 0, or e.g. NULL literals into modern C++11 nullptr version. To refactor all uses of the old-style literals in your project, simply run:

% run-clang-tidy.py -header-filter='.*' -checks='-*,modernize-use-nullptr' -fix

It’s usually a good idea to perform one checker after another, committing intermediate results (think of “Port towards C++11 nullptr”, “Port towards C++11 override”, …) into your version-control system.

Some real world examples

I’ve personally used clang-tidy on a lot of different projects already, with positive results. Remember, this tool has perfect knowledge of your code (as it is in fact using the Clang compiler frontend) and thus will refactor your code without ever introducing broken code.

Examples:

  • This patch for instance ports all of KDE’s Frameworks libraries towards C++11 nullptr, by touching around 9000 different code locations
  • This patch ports the KDE Marble code base to C++11 override, by touching around 2300 different code locations

Conclusion

Clang-Tidy is a powerful tool which makes porting your legacy code base towards C++11 a matter of running a one-liner. It comes with a great set of default checkers and the list of additional ones grows constantly. The modernize- checkers can be used to modernize/refactor your source code to use new C++ language features.

In the next part of this series we will discuss how to use clang-tidy project-wide with other build systems.

The post Clang-Tidy, part 1: Modernize your source code using C++11/C++14 appeared first on KDAB.

]]>
https://www.kdab.com/clang-tidy-part-1-modernize-source-code-using-c11c14/feed/ 16 14309
Writing Code That Won’t Kill https://www.kdab.com/functional_safety_sourcebook/ https://www.kdab.com/functional_safety_sourcebook/#comments Wed, 21 Dec 2016 16:27:29 +0000 https://www.kdab.com/?p=13744 Functional Safety is the term used for systems designed to minimize risk to human health. But there’s a dilemma in today’s tech-driven world. We are more reliant than ever on software to control our planes, trains, cars, and boats, to operate our medical equipment, to process our food, to clean our water, and to power […]

The post Writing Code That Won’t Kill appeared first on KDAB.

]]>
Functional Safety is the term used for systems designed to minimize risk to human health. But there’s a dilemma in today’s tech-driven world.

We are more reliant than ever on software to control our planes, trains, cars, and boats, to operate our medical equipment, to process our food, to clean our water, and to power our homes. If these systems don’t function properly, they have the ability to injure, or even kill us. Meanwhile software complexity continues to increase exponentially making it impossible to exhaustively test the millions of individual states of a product, even for a relatively simple embedded system. Even when exhaustive state testing is possible, that testing may not reveal faults for software that interfaces with external systems (including humans), is multi-threaded, or runs on multi-core hardware.

If you’re a programmer who needs to work on functionally safe software, this state of affairs might make you panic. What to do? You need an education in process, in methods, and in best practices. While there’s a lot of great material out there, sorting out the valuable nuggets from the sales pitches and the impractical academic takes a lot of time.

We’re concerned about making safe software too, so we’ve done some research for you. Here are a few whitepapers, blogs, and articles on functional safety that are worth a read. If you’ve got other great functional safety references that you would recommend, please post them in the comments for everyone.

 

NASA Software Safety Guidebook Very comprehensive overview of safe software, terminology, processes, requirements, design, implementation, testing and integration

C/C++ Coding Standard Recommendations for IEC 61508 Mike Medoff, John Grebe, exida Consulting. Whitepaper on developing new C/C++ code as well as integrating with other modules, outlining principles and coding guidelines for writing safe C/C++ software

The Power of Ten – Rules for Developing Safety Critical Code Gerard J. Holzmann, NASA/JPL Laboratory for Reliable Software. Whitepaper on ten strict but straightforward rules that assist in building safe software with C

Implementing Functional Safety, IEEE Computer Society Christof Ebert, Vector Consulting Services. Article on the different ISO safety standards, high-level benefits of bringing process to the organization

Building Functional Safety into Complex Systems, Part I Building Functional Safety into Complex Systems, Part II Chris Hobbs, QNX Software Systems. Vendor whitepaper (behind an email wall) on the challenges faced in certifying complex software and the methods on properly validating software, with many examples and references

A Causal Model of Human Error for Safety Critical User Interface Design Julia Galliers, Shailey Minocha and Alistair G. Sutcliffe Centre, School of Informatics, City University, London, United Kingdom. University whitepaper that explains human causes of errors when building safety critical UIs, with detailed case study

ISO 26262 : 2011 Rustam Rakhimov, College of Information and Communication, Konkuk University, South Korea. Presentation with detailed overview of Automotive Safety standard ISO 26262, delving into each section of the standard

Static Verification of Non-Functional Software Requirements in the ISO-26262 Daniel Kästner, Christian Ferdinand, AbsInt GmbH. Whitepaper that expands static analysis to check for timing constraints, runtime errors, and stack overflows

Analysis of ISO26262 standard application in development of steer-by-wire systems Aliaksandr Marchanka, Masters Thesis, Chalmers University of Technology, University of Gothenburg, Sweden. Academic whitepaper that takes existing safety software and applies ISO26262 process to it, including use of Agile development and fault tree analysis tools

A quick guide to ISO 26262 Feabhas Ltd, United Kingdom. High-level summary of crucial points for ISO 26262, including FAQ and several case studies

 

See more whitepapers from KDAB…

The post Writing Code That Won’t Kill appeared first on KDAB.

]]>
https://www.kdab.com/functional_safety_sourcebook/feed/ 2 13744
Qt on Android: How to create an Android service using Qt https://www.kdab.com/qt-android-create-android-service-using-qt/ https://www.kdab.com/qt-android-create-android-service-using-qt/#comments Wed, 28 Sep 2016 12:26:37 +0000 https://www.kdab.com/?p=10812 Starting with Qt 5.7, we added the ability to create Android services using Qt. In this article we’re going to see how to get started and also how to communicate between the two. Before we get started I want to add a big bold WARNING about the performance! Because the services are run in the […]

The post Qt on Android: How to create an Android service using Qt appeared first on KDAB.

]]>
Starting with Qt 5.7, we added the ability to create Android services using Qt. In this article we’re going to see how to get started and also how to communicate between the two.

Before we get started I want to add a big bold WARNING about the performance! Because the services are run in the background for a very long time, make sure your service doesn’t drain the device battery!

Update for Qt 5.10+

Starting with Qt 5.10, you must use QAndroidService instead of QCoreApplication on the server side.

Getting started

Step I: Extend QtService

Every single Qt Android Service must have its own Service java class which extends QtService, so the first step is to create such a service:

// java file goes in android/src/com/kdab/training/MyService.java
package com.kdab.training;
import org.qtproject.qt5.android.bindings.QtService;

public class MyService extends QtService
{
}

Step II: Add the service section(s) to your AndroidManifest.xml file

The next step is to add the service section(s) to your AndroidManifest.xml file. To do that you first need to copy & paste the template from https://wiki.qt.io/AndroidServices to your AndroidManifest.xml file, then set android:name attribute with your service class name, as shown in the following snippet:

<application ... >
  <!-- .... -->
  <service android:process=":qt" android:name=".MyService">
  <!-- android:process=":qt" is needed to force the service to run on a separate
                                                        process than the Activity -->

    <!-- .... -->

    <!-- Background running -->
    <meta-data android:name="android.app.background_running" android:value="true"/>
    <!-- Background running -->
  </service>
  <!-- .... -->
</application>

BE AWARE: Every single Qt service/activity MUST run in it’s own process! Therefore for each service you must set a different android:process attribute value.

Step III: How to start the service ?

Now you need to decide how to start the service. There are two ways to do it:

  • on demand
  • at boot time

We’re going to check them both:

Start the service on demand

This is the most common way to start your service(s). To start the service you just need to call Context.startService(Intent intent) method. The easiest way is to add a static method to your MyService:

// java file goes in android/src/com/kdab/training/MyService.java
package com.kdab.training;

import android.content.Context;
import android.content.Intent;
import org.qtproject.qt5.android.bindings.QtService;

public class MyService extends QtService
{
    public static void startMyService(Context ctx) {
        ctx.startService(new Intent(ctx, MyService.class));
    }
}

Then simply call it from Qt to start it:

QAndroidJniObject::callStaticMethod<void>("com/kdab/training/MyService",
                                              "startMyService",
                                              "(Landroid/content/Context;)V",
                                              QtAndroid::androidActivity().object());

Start the service at boot time

This method is used quite seldom and is useful ONLY when you really need to run the service at boot time, otherwise I do recommend you to start it on demand.

First you need to add android.permission.RECEIVE_BOOT_COMPLETED permission to your AndroidManifest.xml file:

<application ... >

  <!-- .... -->
  <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />
</application>

Then you need to add a receiver element to your AndroidManifest.xml file:

<application ... >
    <!-- .... -->
    <receiver android:name=".MyBroadcastReceiver">
        <intent-filter>
            <action android:name="android.intent.action.BOOT_COMPLETED" />
        </intent-filter>
    </receiver>
    <!-- .... -->
</application>

And finally, you need to implement MyBroadcastReceiver class, as shown in the following snippet:

public class MyBroadcastReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        Intent startServiceIntent = new Intent(context, MyService.class);
        context.startService(startServiceIntent);
    }
}

Step IV: Where to put your Qt Service code?

Next you need to decide where you’re going to put your service code. Qt (and qmake) has two options for you:

  • in the same .so file with the application
  • in a separate .so file

We’re going to check them both:

Same .so for app & service(s)

Because you’ll have one big .so file, you need a way to know when it will run as an activity or as a service. To do that you just need pass some arguments to your main function. AndroidManifest.xml allows you to easily do that:

<service ... >
    <!-- ... -->
    <!-- Application arguments -->
    <meta-data android:name="android.app.arguments" android:value="-service"/>
    <!-- Application arguments -->
    <!-- ... -->
</service>

Then make sure you set the same android.app.lib_name metadata for both service(s) & activity elements:

<service ... >
    <!-- ... -->
    <meta-data android:name="android.app.lib_name"
                android:value="-- %%INSERT_APP_LIB_NAME%% --"/>
    <!-- ... -->
</service>

I recommend you to use this method only if your activity and your service(s) share a large piece of code.

Separate .so files for app & service(s)

The second option is to create separate .so files for your app & service(s). First you need to create a separate server .pro file(s):

TEMPLATE = lib
TARGET = server
CONFIG += dll
QT += core
SOURCES += \
    server.cpp

The server .so main entry is the main function:

#include <QDebug>

int main(int argc, char *argv[])
{
    qDebug() << "Hello from service";
    return 0
}

Last you need to load the server .so file:

<service ... >
    <!-- ... -->
    <meta-data android:name="android.app.lib_name" android:value="server"/>
    <!-- ... -->
</service>

Use QtRemoteObject for communication

We’ve seen how to create and how to start a Qt on Android service, now let’s see how to do the communication between them. There are lots of solutions out there, but for any Qt project, I do recommend you use QtRemoteObject, because it will make your life so easy!

QtRemoteObjects is a playground Qt module led by Ford, for object remoting between processes/devices:

  • exports QObjects remotely (properties, signals & slots)
  • exports QAbstractItemModels remotely
  • creates a replicant on the client side you can interface with
  • repc generates source & replica (server & client) source files from .rep files
    • .rep file is the QtRemoteObjects IDL (interface description language)

As you can see it’s very Qt specific! Let’s see how to add it to your projects and use it.

Get QtRemoteObjects

QtRemoteObjects project is located at http://code.qt.io/cgit/qt/qtremoteobjects.git/, to get it you need to run the following commands:

$ git clone git://code.qt.io/qt/qtremoteobjects.git
$ cd qtremoteobjects
$ ~/Qt/5.10.1/android_armv7/bin/qmake -r && make && make install

If needed, replace ~/Qt/5.10.1/android_armv7 with your Qt version and android ABI of choice.

Use QtRemoteObjects

Using QtRemoteObjects is pretty easy, you need to do a few easy steps:

– add QtRemoteObjects to your .pro files

# ...
QT += androidextras
QT += remoteobjects
# ...

– create .rep file(s)

class PingPong {
    SLOT(void ping(const QString &msg));
    SIGNAL(pong(const QString &msg));
}

– add .rep file(s) to the server .pro file

# ...
REPC_SOURCE += pingpong.rep
# ...

– add .rep file(s) to the client .pro file

# ...
REPC_REPLICA += pingpong.rep
# ...

– QtRemoteObjects source(server) side implementation

#include <QAndroidService>
#include "rep_pingpong_source.h"

class PingPong : public PingPongSource {
public slots:
    // PingPongSource interface
    void ping(const QString &msg) override {
        emit pong(msg + " from server");
    }
};

int main(int argc, char *argv[])
{
    QAndroidService app(argc, argv);

    QRemoteObjectHost srcNode(QUrl(QStringLiteral("local:replica")));
    PingPong pingPongServer;
    srcNode.enableRemoting(&pingPongServer);

    return app.exec();
}

Let’s check the code a little bit. First you need to implement all .rep interfaces (PingPongSource), then export PingPong object using enableRemoting.

– QtRemoteObjects replica(client) side implementation

#include "rep_pingpong_replica.h"

// ....
    QRemoteObjectNode repNode;
    repNode.connectToNode(QUrl(QStringLiteral("local:replica")));
    QSharedPointer<PingPongReplica> rep(repNode.acquire<PingPongReplica>());
    bool res = rep->waitForSource();
    Q_ASSERT(res);
    QObject::connect(rep.data(), &PingPongReplica::pong, [](const QString &msg){
        qDebug() << msg;
    });
    rep->ping("Hello");
// ....

Let’s check the code:

  • use QRemoteObjectNode to connect to QRemoteObjectHost
  • use QRemoteObjectNode:acquire to link the local object to the remote one
  • use the acquired object as its local (call slots, connect to signals, etc.)

As you can see, using Qt + QtRemoteObject is (much?) easier and more straight forward than Android’s Java services + AIDL 😉

Limitations

  • the activities & service(s) must run on a different process.
  • it is not possible (yet) to use QtCreator to easily add a service section to your AndroidManifest.xml file check QTCREATORBUG-16884
  • it is not possible (yet) to use QtCreator to easily generate a service subproject for us, check QTCREATORBUG-16885
  • it is not possible (yet) to see the services logs in QtCreator. You’ll need to use
     $ adb logcat 

    to see it, check QTCREATORBUG-16887

  • it is not possible (yet (hopefully)) to debug the services in QtCreator. This feature will take some time to implement it, therefore I’ll not hold my breath for it, check QTCREATORBUG-16886

Please use the above bug report links to vote for your favorite tasks, the ones that have more votes (usually) are implemented first!

You can find the full source code of this article here: https://github.com/KDAB/android

The post Qt on Android: How to create an Android service using Qt appeared first on KDAB.

]]>
https://www.kdab.com/qt-android-create-android-service-using-qt/feed/ 44 12848
Qt on Android: How to restart your application https://www.kdab.com/qt-on-android-how-to-restart-your-application/ https://www.kdab.com/qt-on-android-how-to-restart-your-application/#comments Wed, 22 Jun 2016 10:13:01 +0000 http://www.kdab.com/?p=10032 Some time ago, I wrote a code to restart the running application on Android. You might well ask why such a thing was needed. It was needed because there are cases where, whenever a user changes the theme, the application has to restart to apply the change (it can’t be applied on the fly). In […]

The post Qt on Android: How to restart your application appeared first on KDAB.

]]>
Some time ago, I wrote a code to restart the running application on Android.

You might well ask why such a thing was needed. It was needed because there are cases where, whenever a user changes the theme, the application has to restart to apply the change (it can’t be applied on the fly). In my example I used it to restart Qt Quick Controls 2 gallery.

Sadly my fix was not accepted, because other platforms (iOS, IIRC) have this missing feature too, so it was decided to remove Android code so it didn’t stand out.

Anyway, because I think this piece of code might help other people, I’m going to share it with you today.

Our goal is to use AlarmManager.set() method to set an alarm (a PendingIntent) that will start our application in the near future (in our case 100ms from now), then quits the application. When the alarm timeouts it will start our application. Be aware that the AlarmManager is anything but accurate, so it might take more than 100ms to restart the application (in some cases it takes up to 5s to restart it).

Here is the code that does all the magic:

auto activity = QtAndroid::androidActivity();
auto packageManager = activity.callObjectMethod("getPackageManager",
                                                "()Landroid/content/pm/PackageManager;");

auto activityIntent = packageManager.callObjectMethod("getLaunchIntentForPackage",
                                                      "(Ljava/lang/String;)Landroid/content/Intent;",
                                                      activity.callObjectMethod("getPackageName",
                                                      "()Ljava/lang/String;").object());

auto pendingIntent = QAndroidJniObject::callStaticObjectMethod("android/app/PendingIntent", "getActivity",
                                                               "(Landroid/content/Context;ILandroid/content/Intent;I)Landroid/app/PendingIntent;",
                                                               activity.object(), jint(0), activityIntent.object(),
                                                               QAndroidJniObject::getStaticField<jint>("android/content/Intent",
                                                                                                       "FLAG_ACTIVITY_CLEAR_TOP"));

auto alarmManager = activity.callObjectMethod("getSystemService",
                                              "(Ljava/lang/String;)Ljava/lang/Object;",
                                              QAndroidJniObject::getStaticObjectField("android/content/Context",
                                                                                      "ALARM_SERVICE",
                                                                                      "Ljava/lang/String;").object());

alarmManager.callMethod<void>("set",
                              "(IJLandroid/app/PendingIntent;)V",
                              QAndroidJniObject::getStaticField<jint>("android/app/AlarmManager", "RTC"),
                              jlong(QDateTime::currentMSecsSinceEpoch() + 100), pendingIntent.object());

qApp->quit();

Let’s have a closer look to the code:

  • 1st line gets the activity packageManager object.
  • 2nd line gets an activityIntent object using the packageManager. getPackageName() method returns the name of this application’s package name.
  • 3rd line gets a pendingIntent needed to restart our application.
  • 4th line gets the alarmManager
  • 5th line sets the pendingIntent to start 100 milliseconds from now.
  • 6th line quits the application

The post Qt on Android: How to restart your application appeared first on KDAB.

]]>
https://www.kdab.com/qt-on-android-how-to-restart-your-application/feed/ 4 11458
Qt on Android: How to use Android Toast https://www.kdab.com/qt-android-use-android-toast/ https://www.kdab.com/qt-android-use-android-toast/#comments Tue, 24 May 2016 15:20:59 +0000 http://www.kdab.com/?p=10011 To get you started on this, we’ll be using KDAB’s Android utils with the humble Toast. Toasts are small popups which are used to show the user some feedback. Check Google’s API guide for more info about toasts. The easiest way to show a toast is to use Toast.makeText(Context context, CharSequence text, int duration) static […]

The post Qt on Android: How to use Android Toast appeared first on KDAB.

]]>
To get you started on this, we’ll be using KDAB’s Android utils with the humble Toast. Toasts are small popups which are used to show the user some feedback. Check Google’s API guide for more info about toasts.

The easiest way to show a toast is to use Toast.makeText(Context context, CharSequence text, int duration) static method. This method needs 3 params:

  • the context (we’ll use the activity)
  • the text to show
  • and the duration: one of LENGTH_SHORT (0) and LENGTH_LONG (1).

Then we just need to call the show method.

Of course all the calls MUST happen on the Android UI thread.

As usual we’re going to make use (abuse) of QtAndroidExtras and KDAB’s Android utils (check this article to see how to add them to your project).

Let’s create a simple function to show the toast:

enum Duration {
    SHORT = 0,
    LONG = 1
};

void showToast(const QString &message, Duration duration = LONG) {
    // all the magic must happen on Android UI thread
    QtAndroid::runOnAndroidThread([message, duration] {
        QAndroidJniObject javaString = QAndroidJniObject::fromString(message);
        QAndroidJniObject toast = QAndroidJniObject::callStaticObjectMethod("android/widget/Toast", "makeText",
                                                                            "(Landroid/content/Context;Ljava/lang/CharSequence;I)Landroid/widget/Toast;",
                                                                            QtAndroid::androidActivity().object(),
                                                                            javaString.object(),
                                                                            jint(duration));
        toast.callMethod<void>("show");
    });
}

Yep, it’s that simple!

Now let’s take a closer look at the code:

  • We’re using KDAB::Android::runOnAndroidThread to run everything on the Android UI thread.
  • We’re capturing message and duration params as values not as references, because as we’ve seen in a previous article KDAB::Android::runOnAndroidThread is (most of the time) asynchronous.
  • We’re using QAndroidJniObject::fromString to convert a QString to a QAndroidJniObject string object.
  • We’re using QAndroidJniObject::callStaticObjectMethod to call the makeText static method that returns a Toast java object. We need to pass the following parameters:
    • “android/widget/Toast” is the fully qualified class name.
    • “makeText” is the static method name
    • “(Landroid/content/Context;Ljava/lang/CharSequence;I)Landroid/widget/Toast;” – is the method signature. The method returns Landroid/widget/Toast; object and expects 3 arguments:
      • Landroid/content/Context; – a Content object
      • Ljava/lang/CharSequence; – a CharSequence object
      • I – an int
    • QtAndroid::androidActivity().object(), javaString.object(), jint(duration) are the makeText arguments.
  • after we get the toast object, we call show method and we’re done.

Enjoy!

showToast(QLatin1String("Hello from Qt"));

The post Qt on Android: How to use Android Toast appeared first on KDAB.

]]>
https://www.kdab.com/qt-android-use-android-toast/feed/ 5 11439