With the recent rust in Linux events in the last couple of days, It’s a good time to write up Rust in illumos. Both to spread the word a bit and also to set expectations for both sides (Rust and illumos/OpenIndiana devs) what is currently possible and what work would need to be invested to make things smooth. And also to let the rust community know about what illumos people were talking about.
What most of the talk currently is about, are the technical details. But we must not leave the social aspects out of it. Software distributions are not made by lone walkers but by groups of people. Bringing in a new language means facilitating change. And that means there are more topics to discuss than just API design. We are talking about impacts on the whole software lifecycle.
Linux DRM API design
Looking at the things people like Asahi Lina want to address inside the Linux Kernel with the Rust bindings and how she describes the issues with Locking I get the feeling something with DRM is not consistent. Looking into our code and our docs for this topic I can already see that things are more complex in the whole kernel than just “do X” We have some general recommendations but it’s a case by case issue when looking at it over the whole kernel sources. When looking at the illumos DRM fork I can see that a lot of X11 code seems to have wandered into the Kernel. And not many files are created by the same people. I am not surprised that this has gotten messy. The illumos kernel docs talk about multiple cases where Data can be accessed and different needs for locking with them. I assume the Linux kernel has similar cases, hence at least mentioning what locks the driver needs to do and which the DRM API does needs to be documented.
Rust in the illumos Kernel
The development model of illumos is different from Linux and thus there are no Rust drivers in upstream illumos yet. But that is to be expected for new things. In our model, we take the time to mature new tech in a fork, and for rust, the Oxide fork has taken that role. In there, we have several drivers for the Oxide Networking stack that are in rust. Based on that some experience could be gained. The current state is, that making things in Rust takes more time compared to C for a trained developer. There is Bindgen which has an overhead to learn and use and there is Language training that people need to become productive. It’s one thing to understand the language but becoming productive usually means quite a bit more training on top of that. So far userland tools have proven to be small enough to get to a working result within a reasonable time. OPTE and fast path Networking exist but they still need an integration into the MAC network framework. So more work needs to happen on that front. Smaller drivers are also a possibility to do I am currently unaware of somebody that had an interest though.
The lack of Systems package manager support in rust
Ever since npm started the packaging ecosystem has changed drastically in focus. Where package managers originally installed Software as a System they now shifted to install software on different Systems in the same way. This also changed how responsibilities are handled and how people now develop software. Distributions have become obsolete and are not of interest to people. And that leads to a couple of interesting issues. Because at the end of the day, you need a distribution to start using a Computer. And people that make distributions also need to make some income for that work. But software developers now only need to focus on their software for things to work. Compiling from source has become trivial. But only if you follow the software developer’s workflow and know the tools. For people needing to read themselves into how cargo works, some quirks make sense for software developer workflows but are a hindrance for system development workflows. Keep in mind these are different requirements than the ones the kernel needs. Systems Distributors usually do the following:
- Download the sources and make an archive with all the patches they want to bring
- patch the sources from the patch files
- build the binaries
- pack them into an archive for distribution (depends on the package manager)
Several of cargo’s features are now counterintuitive and Are the ones I find people criticising. Cargo wants to secure and check that the vendor folder has not been modified. There is no central vendor folder. Rust software can easily have upwards of 100 dependencies and micro dependencies. So if people want audit software it becomes a lot of work. I don’t mind. I am long enough in this industry that I have lived through the Java dependency problems on Linux systems. Cargo improves upon that situation. Several other OS’s believe that there needs to be a clear differentiation between system and Third-party software. And I agree with that. The Debian approach of putting everything into one system and only having one version of each dependency is not feasible for a huge international community of people that develop together but never meet. Most of the time the devs do not talk to each other at all. I am personally of the opinion, that most of rust builds and their tier system works. And I am happy to rely on that and not just my tools. As a side note. None of this topic requires Harsh words. Systems packagers and Software developers (especially the folks in Debian and traditional Linux distros) have very different ways of thinking and cargo is not a tool for system packagers. It would be nice if it grew some support for that but that does not require harsh words. System source repos like illumos-gate can easily vendor and produce binaries with a small list of dependencies to deliver the binaries via a Package manager. The tools work.
Missing support for shared libraries
A feature not wanted by Software developers, developing for multiple platforms. For systems packagers, this is a required feature. Shared libraries delimit the boundary between two responsibilities. And if those people coordinate then that works well. It becomes a system. There are limits to where this can happen. So I don’t know what the perfect solution is. If shared libraries are needed at all or if that feature can fade out. But it is worth a try to build systems and to give people the possibility to do so. So I would wish Rust and Cargo gained shared library support so that we can build such componentized systems easily.
An invite
With all this said I would love to have some more rust folks in the illumos community. And I know this has been expressed by others as well. Userland tools are easy to make in Rust and I for one would love to have people help me with the new Installer and with the package Forge We have gained rust crates for our unique API’s such as libcontract Our new image builder for ISO’s is in rust image-builder and we are always looking for driver developers. Check out the illumos for all repos Including a config manager. Want to write complete Wifi kernel parts? A small serial Adapter that you have lying around? Want to integrate with an existing Kernel without rewriting it? If anything of that makes you want to head over to https://illumos.org/books/wdd/preface.html#preface and https://illumos.org/books/dev/ or Simply just interests you. Then we would love to have your contribution.
Hope to talk to some folks on Socials and email
– Toasty