Skip to content

Clang Tidy, part 2: Integrate qmake and other build systems using Bear Automated refactoring of your source code using powerful open-source tooling

Introduction

This article is part of a blog series about Clang Tidy. In the previous article we learned about the general usage of Clang Tidy to automatically refactor source code for projects using the CMake build system. In this particular episode we’ll discuss using Clang Tooling on projects using different build systems with the help of Bear.

Motivation: So you want to use Clang Tooling on your project — but what if your particular project of interest is using another build system such as qmake, a build system predominantly used in the Qt world? There is a way to leverage Clang Tooling on any build system out there by just using a small helper tool.

Introducing Bear

Bear (Build EAR) is a tool that generates a compilation database for clang tooling. As we’ve learned in the previous article in this blog series, all that Clang Tooling needs to know about your project is a JSON Compilation Database, which is basically a replay of all the compilations (i.e. compiler invocations) happening during a project build. While the CMake build system can generate one out of the box, other build systems usually require additional work to get one.

Bear is a generic way to generate the compilation database during the build process, i.e. when running make for any build system. The concept behind Bear is that it intercepts the exec calls done by the build tool (i.e. make) and uses the command-line arguments passed to the compiler and stores them in the compilation database. Think of a strace-like tool filtering for exec system calls.

Unfortunately, Bear currently only works on Unix-based systems (FreeBSD, GNU/Linux and macOS), since it’s heavily relying on the LD_PRELOAD mechanism on *BSD/Linux and its macOS counterpart to intercept the system calls.

Getting Bear

On Ubuntu (since at least 14.04 LTS) or Debian (since at least jessie):

sudo apt-get install bear

On any other distributions not providing a package:

git clone https://github.com/rizsotto/Bear
cd Bear
cmake .; make; make install

Refer to the official documentation for more information.

Using Bear

This section will show how to use Bear with different build systems. Always remember that Bear is just a helper tool used to intercept the build tool used. The end result of Bear is a compilation database which can be consumed by Clang Tooling later.

Using Bear with qmake

The qmake build system is predominantly used in the Qt world. While this little tutorial is specific to qmake, the same workflow can be applied to any other build system.

Let’s use Bear on an example project called qtremoteobjects, which is a Qt module hosted on Qt Git.

<checkout qtremoteobjects>
cd qtremoteobjects
mkcd build

PS: Tip: mkcd just creates a directory and cd’s into it — source.

At this point we want to set up the build system. Note we cannot just call qmake .. alone, since on Linux this will use the linux-gcc spec by default. The compile flags used in the Makefiles generated using this spec won’t be compatible with clang, thus we need to resort to using the linux-clang mkspec.

qmake -spec linux-clang ..

This generated the root Makefile. Now we have to run make, but since we want to generate the compilation database on the fly as well, we have to “preload” bear.

bear make

After bear make finished, you’ll end up with a file called compile_commands.json in the current working directory. Similarly as if you would have let CMake generate this compilation database for you.

Let’s check what’s inside:

head compile_commands.json
[
{
"command": "c++ -c -pipe -g -std=c++1z -fvisibility=hidden -fvisibility-inlines-hidden -fno-exceptions -Wall -W -D_REENTRANT -fPIC -DQT_BUILD_REMOTEOBJECTS_LIB ... -I/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects ... -o .obj/qconnection_local_backend.o /home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_local_backend.cpp",
"directory": "/home/kfunk/devel/src/qt5.8/qtremoteobjects/build/src/remoteobjects",
"file": "/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_local_backend.cpp"
},
{
"command": "c++ ... /home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qconnection_tcpip_backend.cpp",
...

Looking good.

Note: If you interrupt bear make, a consecutive run of bear make will just override this compile_commands.json file. In other words, you’ll lose the information inside that file from the previous make run, of targest which have been compiled already. Thus in order to get a complete compile_commands.json, clear the build directory first, then invoke bear ... as before again.

Using Bear with any other build tool (for completeness sake)

This is what you need:

bear <build tool>

The build tool could be anything — it can even be a totally different build system which does not leverage traditional build tools such as make/ninja to invoke the compiler.

Running clang-tidy as usual

Next, as soon as we have a valid compile_commands.json file, we can run run clang-tidy as we learned in part 1 of this blog series, using the run-clang-tidy script:

run-clang-tidy-3.9.py -clang-tidy-binary clang-tidy-3.9 -clang-apply-replacements-binary clang-apply-replacements-3.9 -header-filter='.*' -checks='-*,modernize-use-auto' -fix
...
/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectnode.cpp:419:5: warning: use auto when initializing with new to avoid duplicating the type name [modernize-use-auto]
QConnectedReplicaPrivate *rp = new QConnectedReplicaPrivate(name, meta, q);
^
auto
/home/kfunk/devel/src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectnode.cpp:440:9: warning: use auto when initializing with new to avoid duplicating the type name [modernize-use-auto]
QInProcessReplicaPrivate *rp = new QInProcessReplicaPrivate(name, meta, q);
^
auto
...

We’ll end up with a couple of changes in the source directory which we can commit afterwards.

Troubleshooting

Unfortunately there are a couple of issues when running Bear which one needs to take into account when using it.

Problem with PCHs from a different compiler

If we had used qmake -spec linux-gcc ... before, the clang-tidy invocation would have failed. You’d run into issues such as this one here:

error: no suitable precompiled header file found in directory '.pch/Qt5RemoteObjects.gch' [clang-diagnostic-error]

In other words: Clang tries to read a precompiled header (PCH) generated by GCC which it isn’t capable of => compilation fails.

Solution: Use qmake -spec linux-clang ... in order to use the correct compiler from the start.

Problem with PCH from a different compiler version

If we had used qmake ... without CONFIG-=precompile_header and clang++ is of a different version than the clang-tidy executable, we would have run into this issue:

error: PCH file built from a different branch ((tags/RELEASE_400/rc1)) than the compiler ((tags/RELEASE_391/rc2)) [clang-diagnostic-error]

Solution: Either make sure the clang-tidy version and the clang version match, or just disable precompiled headers in qmake by passing CONFIG-=precompile_header to it.

clang-apply-replacements executable segfaulting

Problem: When running clang-apply-replacements on a set of suggested fixes (stored as .yml files in a tmp directory) — which is done as part of the run-clang-tidy script — one may encounter invalid file references. When encountering the same invalid file references a second time, any currently released version of clang-apply-replacements will crash.

Example invocation:

% clang-apply-replacements /tmp/tmpIqtp7m
...
Described file '.moc/../../../../../../src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectabstractitemmodelreplica_p.h' doesn't exist. Ignoring...
Described file '.moc/../../../../../../src/qt5.8/qtremoteobjects/src/remoteobjects/qremoteobjectpendingcall.h' doesn't exist. Ignoring...
zsh: segmentation fault  clang-apply-replacements /tmp/tmpIqtp7m

The clang-apply-replacements executable crashes. Note I haven’t investigated why the invalid file references happen in the first place, but I’ve fixed the crash in clang-apply-replacements upstream for good.

Solution: Apply this patch when you have a source checkout of the Clang Tooling.

Unfortunately that means you’ll have to compile the clang-apply-replacements project yourself in order to take advantage of the patch for now or wait for a new Clang release.

Conclusion

With Bear one can use Clang Tooling on projects based on other build systems such as CMake. It’s a simple strace-like tool which intercepts the build tool and logs the way the compiler is invoked in a compilation database file, which can later be consumed by clang-tidy and other Clang Tooling helpers.

Any thoughts, questions?

Need help?

KDAB employs several engineers who are working with Clang Tooling on a daily-basis. We’re happy to assist you in case you have troubles using Clang Tooling in your project or want to get the most out of it: by letting us implement project-specific code checks or run automatic refactoring tools on your code base. We’ve helped clients modernizing their very large code bases successfully using tooling – something which would not have been feasible for them cost-wise doing it manually.

Contact us

FacebookTwitterLinkedInEmail
Senior Software Engineer

12 thoughts on “Clang Tidy, part 2: Integrate qmake and other build systems using Bear”

  1. There are few gotcha’s w/ bear. First off, afaik, it does not work in windows at all. I think there was a mention that it should work but atleast i never got it to that stage – your mileage may vary.

    Second, on linux it uses LD_PRELOAD mechanism to inject itself to build chain. While this is ok, when your build tools consists of 32 & 64bit binaries (which is common atleast in certain cross-compiler envs), things get hairy..

    Same author has a new project “scan-build”[1]. Its a python re-implementation of clang’s own scan-build. Way easier to get it working in mixed environments then any tooling to get proper compile commands json out of your build.

    * https://github.com/rizsotto/scan-build

    1. Right, Bear only works on Unix systems (FreeBSD/Linux/macOS). There’s no solution for Windows yet, you have to resorts to hacks for getting a compile_commands.json there (which is actually a topic of one of my upcoming blog posts already…).

      I didn’t have a look at Rizsotto’s scan-build myself yet, but it indeed sounds interesting. Using compiler wrappers to intercept compiler calls during the build process as it suggests is what I’d imagine to be the most portable approach to the problem. I’ll definitely check it out, thanks for the hint!

  2. Hi Kevin,
    Great post, thanks for share your experience with us!
    I’m using Qt-Creator on daily basis (invoking make with arguments as build step), so I’m trying to add a target in my makefile to check tidy errors and integrate this into my IDE.
    It works but I would like to visualize the warns/errors in the issue output of Qt IDE in order to allow me double click to go into the source line, as already happens when gcc raises an warn/error.

    My makefile has the following target:
    clang-tidy: clean
    bear make -j4
    ./run-clang-tidy.py -p . -header-filter=’.*’ -checks=’*’

    I only see the clang-tidy result in the Compile Output, but not in the Issue window.
    I don’t know if I’m doing something wrong or the output pattern of clang-tidy to notify the errors is different from expected.

    Have you tried to do something similar?
    Thanks!

  3. I am finding that Bear outputs the “arguments” form of compile_commands.json, not the “command” form you’ve shown… In other words there is a json list of arguments instead of a single string representing the command. This is relevant to me because the “cppcheck” static analyzer requires the “command” form. How did you generate the compile_commands.json you show above? Was it from CMake? Thanks.

  4. hi, i’m trying to integrate clang-tidy as a name-checker in a standard makefile environment (no access to cmake and friends). when running clang-tidy on a file i’d expect it to just parse that file, not any includes etc. i don’t want it to look for missing headerfiles etc.. however, i still always get the clang-diagnostic-error for ‘file not found’ even though the clang-diagnostic-* rules are not included. any feedback appreciated 🙂

  5. Is there a way to tell Bear and/or qmake not to include the moc files in the compilation data base?

  6. Great guide; it is the only answer I found online for the “how to do I get a compile_commands.json with qmake?” question 🙂

    I would use:
    bear make -jn

    because building e.g. qtbase with one CPU core is really bad for your health 🙂

    1. True. Though I didn’t want to make the example above more complicate than needed. I think people who are used to invoking make from the command-line should know this heart. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *