Sign up for the KDAB Newsletter
Stay on top of the latest news, publications, events and more.
Go to Sign-up
Python is a handy all-purpose language. It can make you very productive within a short time period and has powerful expressiveness for data manipulation and processing. Yet, it's not a great fit for lots of tasks. C++ is far better at achieving anything that needs bare metal performance, deterministic timing, or low-level access. Thankfully, some great tools are available that make it relatively easy to create Python bindings that let Python functions call into C++ code.
In this blog, we’re going to explain the process of creating Python bindings for your Qt library using one of our own open source Qt libraries as an example, KDDockWidgets. You can use the same process to create Python bindings for plain C++ (non-Qt) libraries, too. Once you see how straightforward the process is, you’ll want to add Python accessibility for all of your existing libraries. Consequently, a whole new community of programmers will be able to use them. And if you want a working example to download, you can get KDDockWidgets off GitHub.
KDDockWidgets is our open source Qt-based library for adding a sophisticated docking system to Qt applications. (It’s got a number of advanced features that are missing from QDockWidgets.) Making this module accessible from Python requires a number of steps:
While this looks like a lot of work, each step is pretty simple, and the tools will do most of the work. Let's take a more detailed look at each step.
You can do this pretty easily with the Python package installer; however, you must replace [Qt-Version] with the version of Qt you’re using.
~$ python3 -m pip install --index-url=http://download.qt.io/snapshots/ci/pyside/[Qt-Version]/latest/ shiboken2-generator pyside2 --trusted-host download.qt.io
The “gotcha” here is that an incompatible version of PySide2 and Qt can cause crashes. For example, if you’re building on Qt 5.15, you need to use this:
~$ python3 -m pip install --index-url=http://download.qt.io/snapshots/ci/pyside/5.15/latest/ shiboken2-generator pyside2 --trusted-host download.qt.io
You have to install the Pyside2 package directly from the repository. That’s necessary since the standard pip package doesn’t contain the Shiboken application that generates bindings, while the repository version does.
(If you’re planning on binding a plain C++ library rather than a Qt-specific C++ library, you can eliminate the pyside2 in the above step.)
Note that although Shiboken was created to handle generic C++, it has been tested primarily against Qt libraries. That means that if your library uses language features normally absent from Qt code – such as complex templates or smart pointers – you could possibly run into some issues in the generated bindings. (If you do find any bugs, you may wish to report them.)
The next thing you need to do is use a bit of CMake configuration magic to ensure your build process can find PySide2 and Shiboken when required. We use two CMake scripts to do this – FindPySide2.cmake and FindShiboken.cmake – that run some inline Python code to discover where these modules are in your Python installation and set a handful of CMake variables that point to them for later use.
Thankfully, we’ve done all of that work already – you can grab those two CMake files directly from our GitHub repo and use them unchanged.
You could add the requisite CMake variables and configuration to your build process every time you wanted to create Python bindings. However, it’s more convenient to do this with a helper CMake file that allows you to create a Python module with a single macro call. This macro will automatically fill in all the paths, flags, libraries, and variables as needed, making it much easier for any new Python bindings you might need.
We created a CMake macro called CREATE_PYTHON_BINDINGS that sets things up. You’ll need PySide2ModuleBuild.cmake for this – just grab it from our repo and add it to your project.
The fourth step requires a bit more work than the previous steps, where the work is mostly done for you. This time, you must describe to the Shiboken generator the API that your C++ code will make available to Python. More specifically, you need to outline which C++ classes, methods, and types will be accessible, and what they will look like to a Python caller.
This API definition is called a typesystem specification, and it’s defined in an XML file. There is a great deal of power available; you can rename classes and methods, change or add parameters, modify access levels, even add new functions.
The typesystem definition for KDDockWidgets is:
<?xml version="1.0"?>
<!-- The package name -->
<typesystem package="KDDockWidgets">
<!-- Pre-defined typesystem that contains types used by our class.
PySide has one typesystem for each module, so here we use only the
Widgets typesystem because it already includes gui and core
typesystems -->
<load-typesystem name="typesystem_widgets.xml" generate="no"/>
<!-- Our classes are declared in a namespace, so we need to define one -->
<namespace-type name="KDDockWidgets">
<!-- This is used in a public virtual pure function.
We need to declare it otherwise shiboken will ignore the
function and will fail to create a wrapper -->
<primitive-type name="DropAreaWithCentralFrame"/>
<!-- Export our public enums and flags -->
<enum-type name="Location"/>
<enum-type name="MainWindowOption" flags="MainWindowOptions"/>
<enum-type name="AddingOption"/>
<enum-type name="RestoreOption" flags="RestoreOptions"/>
<enum-type name="DefaultSizeMode"/>
<enum-type name="FrameOption" flags="FrameOptions"/>
<!-- Export our classes
For classes we can use two types:
object-type: class that does not have a copy-constructor
and cannot be passed as value to functions;
value-type: class that can be passed as value for functions
Here we only use 'object-type' since all our classes are
derived from QWidget.
-->
<object-type name="MainWindowBase" />
<object-type name="MainWindow" />
<!-- DockWidgetBase contains an internal enum, so we declare it
inside of the object-type -->
<object-type name="DockWidgetBase" >
<enum-type name="Option" flags="Options" />
</object-type>
<object-type name="DockWidget" />
</namespace-type>
</typesystem>
Code sample 1 - Typesystem_kddockwidgets.xml
Three main blocks are in this definition: your package (for setup and loading other typesystem files), your public types (the enum-types), and your classes (the object-types).
Because we’re providing a straight-through API, our needs are pretty simple. We just need to list the public classes and enums, and then let Shiboken determine the types and method signatures for everything. If you need to provide finer control over how your Python interface looks, you’ll want to consult the reference docs for the XML you’ll need.
When you call the CREATE_PYTHON_BINDINGS macro within PySide2ModuleBuild.cmake, you’ll need to pass it a few arguments. One is the location of a header file that includes all the classes that you might need during export. The header itself is simple; you just need to include other headers that Shiboken will need to access all type definitions. In the following example, we’ve included all the headers with the class interfaces that we’re adding to our Python package.
#pragma once
// Make "signals:", "slots:" visible as access specifiers
#define QT_ANNOTATE_ACCESS_SPECIFIER(a) __attribute__((annotate(#a)))
#include <MainWindowBase.h>
#include <MainWindow.h>
#include <DockWidgetBase.h>
#include <DockWidget.h>
Code sample 2 - kddockwidgets_global.h
The special QT_ANNOTATE_ACCESS_SPECIFIER line ensures that signals and slots are visible to the generator. (Since this header file is only used by the generator, it’s not harmful to leave it in even if you don’t need it.)
Here, you set up CMake to invoke Shiboken and add any necessary dependencies. You'll need to customize this file for your application. Replace all the instances of KDDockWidgets directories, source, and header files with those from your application.
# Auto-Generate files - every class will have a cpp and h file
set(PyKDDockWidgets_SRC
# individual classes
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_dockwidgetbase_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_dockwidgetbase_wrapper.h
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_dockwidget_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_dockwidget_wrapper.h
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_mainwindowbase_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_mainwindowbase_wrapper.h
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_mainwindow_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_mainwindow_wrapper.h
# namespace wrapper
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_wrapper.h
# global module wrapper
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_module_wrapper.cpp
${CMAKE_CURRENT_BINARY_DIR}/KDDockWidgets/kddockwidgets_python.h
)
# The includes are needed to parse and build classes specified in our typesystem
set(PyKDDockWidgets_include_paths
$<JOIN:$<TARGET_PROPERTY:KDAB::kddockwidgets,INTERFACE_INCLUDE_DIRECTORIES>,${PATH_SEP}>
)
# Set list of paths where shiboken should look for typesystem
set(PyKDDockWidgets_typesystem_paths
# PySide path, this variable was exposed by FindPySide2.cmake
${PYSIDE_TYPESYSTEMS}
)
# Include flags/path that will be set in 'target_include_directories'
set(PyKDDockWidgets_target_include_directories
${CMAKE_SOURCE_DIR}
)
# Libraries necessary to link the target for the command 'target_link_libraries'
set(PyKDDockWidgets_target_link_libraries
KDAB::kddockwidgets
Qt5::Core
Qt5::Gui
Qt5::Widgets
)
# Dependencies - changes on these files should trigger new bindings
set(PyKDDockWidgets_DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/kddockwidgets_global.h
${CMAKE_SOURCE_DIR}/src/DockWidgetBase.h
${CMAKE_SOURCE_DIR}/src/DockWidget.h
${CMAKE_SOURCE_DIR}/src/MainWindowBase.h
${CMAKE_SOURCE_DIR}/src/MainWindow.h
)
CREATE_PYTHON_BINDINGS(
"KDDockWidgets"
"${PyKDDockWidgets_typesystem_paths}"
"${PyKDDockWidgets_include_paths}"
"${PyKDDockWidgets_SRC}"
"${PyKDDockWidgets_target_include_directories}"
"${PyKDDockWidgets_target_link_libraries}"
${CMAKE_CURRENT_SOURCE_DIR}/kddockwidgets_global.h
${CMAKE_CURRENT_SOURCE_DIR}/typesystem_kddockwidgets.xml
"${PyKDDockWidgets_DEPENDS}"
${CMAKE_CURRENT_BINARY_DIR}
)
# Make moduled import from build dir works
configure_file(${CMAKE_CURRENT_SOURCE_DIR}/__init__.py ${CMAKE_CURRENT_BINARY_DIR}/__init__.py)
# install
install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/__init__.py DESTINATION ${Python3_SITELIB}/PyKDDockWidgets)
Code sample 3 - CMakeLists.txt
If you ran everything right now, you’d have created code that allows Python to call into C++. However, you’d still need to add one small file to the package directory to turn your code into an importable Python package.
__all__ = ['KDDockWidgets']
# Preload PySide2 libraries to avoid missing libraries while loading our module
try:
from PySide2 import QtCore
except Exception:
print("Failed to load PySide")
raise
# Avoid duplicate namespace - the package name would normally look like this:
# PyKDDockWidgets.KDDockWidgets.KDDockWidgets.MainWindow
# (There is a bug PYSIDE-1325 to get this namespace duplication fixed.)
# To avoid this, we use this workaround:
from .KDDockWidgets import KDDockWidgets as _priv
KDDockWidgets = _priv
Code sample 4 - __init__.py
You’ll note we have a small workaround in this example for namespace duplication. If PYSIDE-1325 gets fixed, you won’t need the last several lines of code. In fact, you wouldn’t need any code at all since an empty __init__.py is all that Python requires to turn your code into a package.
With everything in place, you can now let the tools do all the work! Invoke CMake, and it will generate the necessary make file. Then, running make will generate and compile your binding files, adding the new Python module to your Python installation.
Assuming your generation and compilation was successful, you’ve now got a Python package that calls your C++ library. You can import it and start using it, just like a regular Python class.
from PyKDDockWidgets import KDDockWidgets
options = KDDockWidgets.MainWindowOption_None
dock = KDDockWidgets.DockWidget("new dock 1")
dock.resize(600, 600)
dock.show()
Code sample 5 - example.py
Using PySide and Shiboken to create Python bindings does take a bit of setup, but it’s not anywhere as difficult as doing all the Python/C++ interface work by hand! Steps 1 through 3 of the process need only be done once. After that, for each new C++ library you’ll need a new typesystem XML file, a few customizations to your CMake file, and a new header to include the binding-specific classes – all tasks that can be accomplished in a few minutes.
You’ll be Python-enabled before you know it!
About KDAB
The KDAB Group is a globally recognized provider for software consulting, development and training, specializing in embedded devices and complex cross-platform desktop applications. In addition to being leading experts in Qt, C++ and 3D technologies for over two decades, KDAB provides deep expertise across the stack, including Linux, Rust and modern UI frameworks. With 100+ employees from 20 countries and offices in Sweden, Germany, USA, France and UK, we serve clients around the world.
Stay on top of the latest news, publications, events and more.
Go to Sign-up
Learn Modern C++
Our hands-on Modern C++ training courses are designed to quickly familiarize newcomers with the language. They also update professional C++ developers on the latest changes in the language and standard library introduced in recent C++ editions.
Learn more
2 Comments
25 - Sept - 2024
Joel
Does this work with PySide6/Qt6?
26 - Sept - 2024
Renato Araujo
That will require some small updates on CMake, but the main idea still the same.