Copying files over the network in a Qt application
Last week I visited a new customer who is making medical and industrial devices which have one thing in common: image and video capturing, and letting the user save these files onto a USB key. These devices run embedded Linux and the application is done in Qt (and gstreamer for the video capture). The new requirement, and the reason I was called in, is to be able to save these files over the network, to a samba or a webdav share. An abstraction framework for copying files over the network to multiple protocols is something I know very well: the KDE framework called KIO implements exactly that, and I happen to be its maintainer. And after the last 4 years of effort in making this much more modular, it is now possible to use KIO with very few dependencies, as we found out. In this two days on-site visit, I first explained many things about KIO (the API, the framework implementation, the available protocols, etc.), and then we started integrating it into their custom embedded linux distribution.
The KIO framework consists of 3 libraries (KIOCore, KIOWidgets, KIOFileWidgets) and a number of helper programs (daemons). However for this use case, only KIOCore is needed. Some user interface is needed: error and progress notification, the handling of “file already exists, overwrite/rename/skip?”, a dialog for entering user/password for the remote share, a way to display SSL certificate details and approval/refusal… But of course for all this, the KDE user interface (implemented by KIOWidgets) isn’t desired, rather another implementation that integrates better with the existing embedded GUI. The good thing is that the proper core/gui separation in KIO allows for exactly this: using only KIOCore and implementing some C++ interfaces to provide a customized user interface.
After adding an option to the KIO buildsystem for building only KIOCore, we were able to reduce greatly the number of required dependencies. Only the following are needed: extra-cmake-module (buildsystem), KCoreAddons (Job API), KConfig (configuration framework), KDBusAddons (daemon modules), KI18n (translation framework) and KArchive (for compressed HTTP transfers).
A simple patch commenting out some unneeded protocols for this particular use case allowed us to avoid some additional dependencies (e.g. Solid, for the “trash” implementation).
The dependencies listed above are all “tier 1” frameworks, i.e. they only depend on Qt. So we were able to quickly integrate them into the hand-made linux distribution, and we have KIOCore ready to use. After copying over the unittests and setting up a few paths, we had all the tests running without errors.
The final hurdle was the process that handles SSL certificate policies, which was implemented as a module for KDED, a generic process that lives in another framework, bringing in more dependencies. I instead made it standalone, which solved the immediate problem. I’m now working on a better solution – a KIO daemon with multiple modules, for the various service that the KIO framework needs in a separate process (cookie handling, etc.)
On the topic of multiple processes, the customer appreciated greatly the fact that KIO protocol implementations (called “kioslaves”) are running in a separate process, because this allows them to look into running them as a different user, to give them limited permissions for security reasons. That user can have a read only access to the saved files (and to nothing else), since the use case is only upload operations.
All this in order to reduce the number of dependencies that KIO has, making it easier than ever to use KIO in any Qt application, so as to benefit from easy to use, asynchronous, well-tested network transparency with more than 90 different protocols available. I’m sure there are other applications with the same need 🙂