Last week I attended the MeetingC++ conference with several colleagues. KDAB was a gold sponsor of the conference in Düsseldorf, and I delivered a talk about ‘Modern CMake‘ with Qt and Boost. My slides are available here and the sequence of them is reflected in this post.
As there will be no video recordings posted of the talk, the next best thing is turning my presentation notes into a blog post.
I’ve worked for a while on CMake, and over the last few years and releases I have completely transformed how it works with regard to dependencies over the last few years and releases. The features I’m showing in here have mostly arrived in the last two CMake releases, but some of the features will arrive only in the next release or another future release.
My contribution to Boost so far has only been related to some minor cleanups and dead code removal. I’ve also been working together with Daniel Pfeifer on porting the Boost libraries themselves to CMake. This is an ongoing initiative in Boost, as part of a general modernization of the tools used to create Boost.
I met several people last year at this conference, who didn’t see any need for buildsystems at all, beyond a hand-written Makefile. I want to just mention some of the reasons that people write tools like CMake.
One reason buildsystem tools exist is for finding dependencies. You might have several locations where dependency headers and libraries need to be found. Even if you can hardcode all of the locations into your own Makefile, the result won’t be distributable. It also won’t be portable because Makefile buildsystems are not common on Windows for example. The dependencies you use might also have specific needs for required compiler flags, and compiler flags to use might vary with the compiler, and even the compiler version. This is the case for Qt for example, and the requirement that it has to use position independent code.
Generic CMake features
CMake is a generic solution for all of those problems. It is a cross-platform system, with powerful APIs for finding dependencies of various or specific versions, and with many abstractions for platforms, compilers, buildsystems and dependencies.
The modern way to find the Qt5Widgets library with CMake looks like this:
find_package(Qt5Widgets 5.2 REQUIRED)
The find_package command has a lot of knowledge built-in for where to look to find Qt. I tell CMake that I want to create an executable called myapp, and that myapp requires the Qt5::Widgets library. That’s all there is to it for finding and using Qt libraries. Much of the rest of this article is about implementation details of how the above works in the CMake code and how the same principles may be applied to any library usable with CMake. My previous article contains more information for new users of CMake with Qt.
Successful linking to the libQt5Widgets.so binary requires first compiling myapp with the required compilation flags.
Successful compilation requires specifying the correct include directories for finding the Qt headers. Qt headers are installed into an include/ directory, with the actual files divided into directories corresponding to the Qt module which provides them. Users of Qt (and dependencies of myapp) may use include directives such as <QtModule/QClass>, or simply <QClass>, so that means that both the include/ directory and the include/QtModule directory.
Successful compilation also requires specifying the correct definitions on the command line. Qt expects that compilations which use the Qt5Gui library use the -DQT_GUI_LIB define, and the other Qt libraries have similar expectations. Not adding these definitions in a consistent and minimal way can lead to problems such as the QTestlib problem I described in my Qt Developer Days talk last year. To be brief, the headers of the Qt unit test library behave differently depending on whether QT_WIDGETS_LIB, QT_GUI_LIB or QT_CORE_LIB is defined, resulting in tests using either a QApplication, QGuiApplication, or QCoreApplication respectively. Of course, if defined incorrectly, this can mean that tests which should only link to Qt5Core can be created to use a QApplication, and therefore require linking to the Qt5Widgets library. The buildsystem is responsible for getting these things correct.
One of the nice (and recent) features of CMake (in master branch, to become CMake 3.0.0) is that it gives diagnostics if I try to use a dependency without first finding it. CMake now recognises a pattern of double-colons ‘::’ in the name of a dependency as denoting a special meaning that it is a IMPORTED target which encodes a lot of information about how to use it.
All other dependencies can work the same way, if the project providing the dependency also provides CMake files for depender-use. These features are not specific to Qt. Such IMPORTED targets are provided for Qt 4 and Qt 5 already, and will be provided by Boost in the future. CMake 2.8.12 also ships IMPORTED targets for Gtk2, contributed by Daniele E. Domenichelli of KDE fame.
CMake is aware that using the Qt5::Widgets library involves a compilation step and a linking step. It knows that because Qt5::Widgets is a target defined in files shipped by Qt in the lib/cmake directory. These files tell CMake that Qt5::Widgets is a SHARED library, and that it has been IMPORTED for use from upstream. The files tell CMake where to find the library binary. Anything using Qt5::Widgets will link to that.
The files also tell CMake where the header files it requires are defined. Anything using Qt5::Widgets will automatically use those include directories to find the headers (since CMake 2.8.11 and Qt 5.1).
The code to do that for the Qt5Widgets library looks something like this:
and for the Qt5Core library looks something like this:
The Qt5::Core target specifies the ‘top level include’, and the Qt5::Widgets target depends transitively on the Qt5::Core target. That means that this essential information does not have to be repeated, but will be automatically consumed by CMake through the total dependency tree.
The INTERFACE_INCLUDE_DIRECTORIES property is a special target property built into CMake. The INTERFACE_ prefix is a convention used to specify information which is consumed by users of the target. In this case, the Qt5::Core target tells users of it the specific INCLUDE_DIRECTORIES required for successful compilation.
The IMPORTED targets provided by Qt also tell CMake what defines should be defined when compiling. Anything
using Qt5::Widgets will automatically use those command line defines when
compiling (since CMake 2.8.11 and Qt 5.1).
There is a IMPORTED target for all of the libraries in Qt. Some of them take advantage of the condition system of defining the interface, like Qt5::Core. The information encoded here says that if the user is doing a Debug build, then define QT_DEBUG on the command line when compiling (since CMake 2.8.11). Similar logic defines QT_NO_DEBUG in the opposite case.
This feature is not limited to just includes and defines, but can apply to many other ‘usage requirements’. Some individual compiler features are also abstracted by CMake. For example, Qt requires that any user of Qt enables position-independent-code. It reports an error if the relevant flag is not used.
That usage-requirement is built into the Qt5::Core target. It would be possible to populate the INTERFACE_COMPILE_OPTIONS of the target to attempt to encode the required compiler flag for each compiler. However, the required compiler flags vary a lot between different compilers and even vary depending on whether an executable or a library is being created. CMake provides a feature specifically for specifying the ‘position independent code’ usage-requirement. CMake ensures that any user of Qt5::Core will automatically use the -fPIC or equivalent flag for compilers that need it (MSVC does not, for example), since CMake 2.8.11.
Qt-aware CMake features
All of the above are generic features of CMake which Qt interfaces with, just as any other library can. However, CMake also has some awareness of Qt built-in. All of these special features are available when using both Qt 4 and Qt 5.
Anyone who has used Qt for more than an afternoon will know that the moc tool is needed for code generation when using Qt. Many features of Qt are built on that code generation, and the user of Qt is required to run moc. There are also code generators for user interface description files, and for virtual resource description files.
CMake is aware of these features file types and code generators, and can enable special handling of them. If you enable CMAKE_AUTOMOC CMake will scan compiled files for the Q_OBJECT macro and automatically run the moc tool as needed (since CMake 2.8.6).
In CMake master branch I have extended this feature to cover the rcc and uic tools. If you set CMAKE_AUTORCC to on, you can list Qt resource files in the sources of an target, and CMake will automatically run the rcc generator tool when needed. If you enable CMAKE_AUTOUIC, CMake will scan source files for ‘ui includes’, and automatically run the uic tool to generate them as needed.
Interface library targets
CMake master branch now supports a new type of library called an INTERFACE library. This type of library is designed to provide only INTERFACE_ properties – there is no binary to build or to link to. A consequence of that design intention is that INTERFACE libraries are suitable for header-only libraries, such as those typically provided by Boost, or Eigen, which already uses CMake.
When I showed similar code for Qt5::Widgets before, that was a shared library, so the LOCATION of the library file needed to be encoded. This INTERFACE library does not relate to any binary file, but it only describes an interface which CMake consumes when the library is used. That interface can refer to include directories, compile definitions, or any other compile-related usage requirement.
All of these features are ‘transitive’, which means that the information about usage requirements can be carried through an arbitrary dependency graph. I implemented these features with the dependency graph of Boost as a specifically supported usecase which must be performant. I can confirm that I had http://xkcd.com/276/ in mind when writing the commit message :).
There are other features which are specifically inspired by boost use-cases. For example, at KDAB we have an embedded domain-specific library for composing SQL queries in compiled, templated C++ code. It works in a simliar to other libraries, but with the difference that SQLate creates results in the form of QSqlResult, instead of a query string. It uses the boost::mpl, and it requires a large number to be defined as the BOOST_MPL_LIMIT_VECTOR_SIZE.
If we imagine that we have a second library which also uses the MPL, and which also has a requirement for the BOOST_MPL_LIMIT_VECTOR_SIZE, but specifically a different limit to the first library.
If I use both libraries together, I require the larger of the numbers to be used during actual compilations. CMake has a way to specify that the maximum number should be used (in CMake master), and it calculates what value needs to be passed on the command line. The code for enabling CMake to calculate the correct number involves populating the COMPATIBLE_INTERFACE_NUMBER_MAX property.
CMake also has a way to specify that the minimum value of a number should be calculated. For example, Qt has a way to specify the level of use of deprecated APIs. If one dependency requires interface(header) usage of Qt 5.1 deprecated features, and another one requires interface usage of Qt 5.2 deprecated features then the user needs to compile specifying the Qt 5.1 version. CMake calculates that internally and automatically. The code for enabling CMake to calculate the correct number involves populating the COMPATIBLE_INTERFACE_NUMBER_MIN property.
In the above cases, we specified that a compatible maximum or minimum number must be calculated from the interface of the used targets. The same principle is used to issue a diagnostic in the case of an attempt to link to both Qt 4 and Qt 5 in the same executable. In this case, the COMPATIBLE_INTERFACE_STRING property is populated with QT_MAJOR_VERSION. That causes the INTERFACE_QT_MAJOR_VERSION to be evaluated from the Qt5::Core and Qt4::QtCore IMPORTED targets by transitively following the dependencies. The string values must then contain the same value to be compatible, or an error diagnostic is issued.
Compile feature specification
People often ask whether CMake ‘supports’ C++11. That is the wrong question to ask. What people are thinking is ‘Can it automatically add the -std=c++11 flag for me?’
Hmm, or should I use -std=c++0x for this compiler?
Or wait, is this a C++14 feature? Maybe I need -std=c++1y?
Oh, wait I’m using MSVC, no flag is needed at all.
The right questions to ask are ‘Does the compiler have the feature I need?’ and ‘Is any flag required to enable that feature?’. The version of the C++ standard that specifies the feature is then not relevant to the user and can be encoded in the implementation of CMake. Aiming for ‘C++11 support in CMake’ would not be future-proof or even past-proof.
Because the standard version which introduced the feature is irrelevant, the user does not need to care whether -std=c++11 or -std=c++98 is needed. By not requiring the user to enable the flags manually, a cross-platform trap can be avoided.
Instead, a future CMake version will allow the user to specify features needed from the compiler. CMake will know which compiler has what feature, so it can issue a good diagnostic if, for example, I specify that I need member templates and then someone tries to use MSVC6 to compile my code. CMake will tell them why it can’t work because MSVC6 does not support that feature. This shows how the CMake API is past-proof and future proof. Thinking in terms of features, not standards, is the right way to go as it is future-proof for C++14 features like generic lambdas, and is also extendible on the axis of compiler extensions.
Because CMake has all the information about what features are supported which which compiler, it can also generate a header file for optional use of features when building your code, and when compiling against its headers.
For example, it will generate a define for each of the features and whether the feature is supported by the users compiler. This is essentially the same kind of thing that the Boost.Config library and qcompilerdetection.h are doing.
We can also define aliases in such a header. Before MSVC supported the final keyword, it supported the sealed keyword in the same position as final. Clang even treats sealed as an alias for final in its MSVC mode. Because we’re generating a header, we can also create portable macros wrapping static_assert etc.
CMake is growing increasingly useful to users of Qt as a buildsystem tool. It is adopting features such as these ‘usage requirements’, which are already familiar to users of Boost.Jam and qmake. The CMake files distributed with Qt and maintained by KDAB are pushing and pioneering the way CMake is used in modern, portable real-world complexity projects.