Hacker News Reader

Comment by Tomte | Hacker News Reader

> rust folks should fix the rust binding when C changes breaks the binding

I have never understood how that could work long-time. How do you release a kernel, where some parts are broken? Either you wait for Rust people to fix their side or you drop the C changes. Or your users suddenly find their driver doesn‘t work anymore after a kernel update.

As a preliminary measure when there isn‘t a substantial amount of Rust code, yet, sure. But the fears of some maintainers that the policy will change to "you either learn Rust and fix things or your code can be held up until someone else helps you out" are well-founded, IMO.

View on HN

Are you familiar with Linux kernel development process? Features can be merged only in two weeks long merge window. After the merge window closes, only fixes are merged for eight weeks. Rust binding can be fixed in that time. I don't see any problems.

That's a gross simplificaftion of the development process. Yes, new features are mostly merged in that two-weeks window -- but you're now talking about the Linux release management process more than its development.

Before features are merged to Linus' release branch, pretty much all changes are published and merged to linux-next first. It is exactly here that build issues and conflicts are first detected and worked out, giving maintainers early visibility into changes that are happening outside their subsystem. Problems with the rust bindings will probably show up here, and the Rust developers will have ample time to fix/realign their code before the merge window even starts. And it's not uncommon for larger features (e.g. when they require coordination across subsystems) to remain in linux-next for more than one cycle.

And if no Rust developer has time or interest in those eight weeks? I don‘t claim that it can never work (or it cannot work in the common case), but as a hard rule it seems untenable.

> And if no Rust developer has time or interest in those eight weeks?

What if Linus decided to go on a two month long vacation in the middle of the merge window?

> I don‘t claim that it can never work (or it cannot work in the common case), but as a hard rule it seems untenable.

There are quite a few rust developers already involved, if they cannot coordinate that at least some are available during a release critical two month period then none of them should be part of any professional project.

I'm not familiar with kernel development, but what's the difference anyway with C code? If you change the interface of some part, any users of it will be broken Rust or not. It will require coordination anyway.

It is customary for maintainers to fix _all_ usage of their code themselves? That doesn't seem scalable.

Yes, that is the custom and is a key advantage of getting drivers in tree. I believe often the changes are applied automatically with a tool like coccinelle,

Keep in mind that actual breaking changes are by design incredibly rare in a project like the linux kernel. If you have a decade's-worth of device drivers depending on your kernel subsystem's API, you don't get to break them, you have to introduce a new version instead.

I think it's more a degree of how much effort it is to adjust to the new interface. If it's just 'added a new parameter to a function and there's an obvious default for existing code', then it'll (potentially mechanically) be applied to all the users. If it's 'completely changed around the abstraction and you need to think carefully about how to port your driver to the new interface', then that's something where there needs to be at least some longer-term migration plan, if only because there's not likely one person who can actually understand all the driver code and make the change.

(I do have experience with this causing regressions: someone updates a set of drivers to a new API, and because of the differences and lack of a good way to test, breaks some detail of the driver)

This isn't true; internal API's change all the time (e.g. adding extra arguments) Try running out of tree drivers on bleeding edge kernels to see for yourself.

Of course, for trivial mechanical changes like adding an argument the Rust binding changes are also trivial. If you've just spent half an hour walking through driver code for hardware you've never heard of changing stuff like

quaff(something, 5, Q_DOOP) ... into ... quaff(something, 5, 0, Q_DEFAULT | (Q_DOOP << 4))

Then it's not beyond the wits of a C programmer to realise that the Rust binding

quaff(var1, n, maybe_doop) ... can be ... quaff(var1, n, 0, Q_DEFAULT | (maybe_doop << 4))

Probably the Rust maintainer will be horrified and emit a patch to do something more idiomatic for binding your new API but there's an excellent chance that meanwhile your minimal patch builds and works since now it has the right number and type of arguments.

> If you've just spent half an hour walking through driver code for hardware you've never heard of changing stuff [...].

Isn’t the point of Coccinelle that you don’t have to spend time walking through (C) driver code you’ve never heard of?

I have never used Coccinelle but yes, sort of. However, you're on the hook for the patch you submit, Coccinelle isn't a person so if you blindly send out a patch Coccinelle generated, without even eyeballing it, you should expect some risk of thrown tomatoes if (unknown to you) this utterly broke some clever code using your previous API in a way you hadn't anticipated in a driver you don't run.

If so kernel is released with broken Rust. That is the policy, and I am flabbergasted why everyone is going "that policy must not be literal".

Because if in a few years I have a device whose driver is written in Rust, a new kernel version might have simply dropped or broken my device driver, and I cannot use my device anymore. But sure, if R4L wants to stay a second-class citizen forever, it can still be acceptable.

this isn't policy forever. it's policy for now. if r4l succeeds, the policy will change.

> Because if in a few years I have a device whose driver is written in Rust, a new kernel version might have simply dropped or broken my device driver, and I cannot use my device anymore.

At least for Debian, all you need to do if you hit such a case is to simply go and choose the old kernel in the Grub screen. You don't even need to deal with installing an older package and dealing with version conflicts or other pains of downgrading.

I hope you're not seriously suggesting this as a reasonable workflow.

For my server or laptop at home, sure. Why not. For servers in commercial fleets you should have staged rollouts as a policy anyway so if you do it right you shouldn't get hit.

It is only a problem if you compile the kernel directly from the source tree instead of using the packages provided by your Linux distribution.

Distros should be your firewall against that sort of thing. Just don't use a distro with a non-existent kernel upgrade process.

I think the way you do this is set things up so that no bits that are written in Rust are built by default, and make sure that the build system is set up such that Rust bindings for C code are only built when there's Rust code that's enabled that requires them.

Then sure, some people who download a kernel release might enable a Rust driver, and that makes the build fail. But until Rust is considered a first-class, fully-supported language in the kernel, that's fine.

In practice, though, I would expect that the Rust maintainers would fix those sorts of things up before an actual release is cut, after the 2-week merge window, during the period when only fixes are accepted. Maybe not every single time, but most of the time; if no one is available to fix a particular bit of breakage, then it's broken for that release. And that's fine too, even if it might be annoying to some users.

> I think the way you do this is set things up so that no bits that are written in Rust are built by default, and make sure that the build system is set up such that Rust bindings for C code are only built when there's Rust code that's enabled that requires them.

Which is currently the only way possible and it will stay that way for a long time because remember that clang support less targets than gcc and gcc cannot compile Rust.

Once gcc can /reliably/ compile Rust, then and only then Rust could be "upgraded" to a first class citizen in Linux. The "C-maintainers don't want to learn Rust" issue, will still be here of course, but there will already be many years of having a mixed code base..

I agree with all you say, but with longterm I really mean when we've arrived here

> But until Rust is considered a first-class, fully-supported language in the kernel, that's fine

A first-class language whose kernel parts may always break does seem unreasonable. I still think policy will have to change by that point.

Because nothing is forcing a distro to adopt a kernel that has items that are broken. Not a lot of folks out there are manually compiling and deploying standalone kernels to production systems.

C can break rust, and Debian/Ubuntu/Redhat/suse/etc can wait for it to be fixed before pushing a new kernel to end users.

You can merge it into your branch as e.g. the DMA maintainer, then the rust folk can pull your changes and fix the bindings. Maybe you as maintainer could give them a heads up and a quick consideration of the error.

Yes, Rust as something optional doesn't really make sense long term. Either it will continue to only be used in nieche drivers (in which case why bother?) or eventually you need to build Rust code to have a usable kernel for common hardware. Any promises to the contrary need to be backed up with more than "trus me bro".

Reply on news.ycombinator.com