Bookworm – the new version of Raspberry Pi OS | Hacker News Reader

Bookworm – the new version of Raspberry Pi OS

raspberrypi.com

182 pointssohkamyung64 comments

I'm always confused by Raspberry Pi OS. It feels like Debian with a pre-set of packages, and a few additional software to control the board and bootloader.

Why not maintain these additional pieces of software packaged in debian, and provide a prebuilt image for the Raspberry Pi? Why maintain an entire different apt repository? Why publish new version months after the upstream debian is already released?

Hard agree and I too am curious as to why it is not like this.

The blog post offers some clues: e.g. they're using Wayland on the Pi 4 and 5 but still X11 on lower devices. It's a tight coupling of hardware and software that means they can best tailor the experience to specific devices. Not everyone needs it but it's nice for newbies (and the Pi is intended to be very approachable) to get sensible defaults out of the box.

You can do this with pre-built images. They can just install different packages on the rbpi3.img and the rbpi4.img, one with wayland one with x11.

If you don't want to use pre-built images you can also do this with metapackages.

I believe one of the biggest reasons is that because the boot process is very different from the way debian does things, it ends up needing significant changes from the starting tasks to do things. Along with that I think the original 32bit RPIs didn't support some version of the compiled binaries in the debian arm repos which meant that they weren't necessarily compatible.

These days on the 64bit version, I'm not sure why it needs that much either. I'd think that having a few new task-* metapackages that setup the boot and raspberry pi specific boot management stuff should be doable and then they could use the main debian packages I would think. There's probably something that's not obvious that makes things just incompatible enough that it isn't simpler to manage that way.

> I believe one of the biggest reasons is that because the boot process is very different from the way debian does things, it ends up needing significant changes from the starting tasks to do things. Along with that I think the original 32bit RPIs didn't support some version of the compiled binaries in the debian arm repos which meant that they weren't necessarily compatible.

No longer correct. Debian on new rPi "just installs" and boots. I think that also worked fine on rPi3

https://wiki.debian.org/RaspberryPi

The problem is that many things have not been upstreamed into kernel so some stuff doesn't work or work badly. Like trying to get video just made the video device return random block of memory instead of an image last time I tried

> These days on the 64bit version, I'm not sure why it needs that much either. I'd think that having a few new task-* metapackages that setup the boot and raspberry pi specific boot management stuff should be doable and then they could use the main debian packages I would think. There's probably something that's not obvious that makes things just incompatible enough that it isn't simpler to manage that way.

I'd imagine the answer is "iteration time". Far quicker to apply a patch fixing some rPi-specific issue if you have your own repo with everything.

Pushing it to upstream of Debian might take few days and might come back with "well that code isn't great, fix it".

Even more if Debian maintainer says to take it up with original project, which is the right way to do it, just slower one.

I think they have bundled non-free software with the OS in the past. Debian isn't very cool with that.

They've been getting far more pragmatic in recent years. e.g non-free wifi drivers are now bundled on the installation images.

Before this it was necessary to do a silly dance, manually downloading drivers and either modifying the installation media or using multiple usb sticks. They understood that this was more than a mere inconvenience, some users are just going to be blocked and move on to a Debian derivative.

You can even use OpenZFS on Debian now, although it forces you to compile it for license compatibility, but it's all automated through apt to ensure it affects the user minimally.

> silly dance, manually downloading drivers and either modifying the installation media or using multiple usb sticks

Or just use USB tethering with your smartphone as the initial network connection

Debian does package Raspberry Pi software [1] (firmware, kernel) and maintains images for each Pi [2] although it is an unofficial effort.

[1]: https://packages.debian.org/bookworm/raspi-firmware

[2]: https://raspi.debian.net/tested-images/

I've being running Debian on my Pi's since buster, and it's being without issues, the only downside would be the lack of support for devicetree overlays.

raspi-firmware is semi-officially maintained. But Raspberry Pi OS also has scripts to set flags on the eeprom like "vcgencmd" and "rpi-eeprom-update"

What I was saying these could be maintained in Debian.

It has less value now with arm64 as a common arch, but for the old armv6, a lot a packages are optimized and compiled for this old platform

Like so many distributions, Raspberry Pi OS seeks to use a lot of Debian's stuff while offering more and sheltering users from the less pleasant parts of the Debian user experience. It was not even close to being the first distro to take this approach.

(as someone else alluded to, Debian abandoned the specific ARM chip architecture the original Raspberry Pi used shortly before the board came out, since it was not popular enough to maintain. I have no idea whether they brought it back.)

The original Raspberry Pi SoC (BCM2835) is ARMv6 with VFP2 Hard Float support.

Debian's "arm" architecture is ARMv7 with VFP3. It doesn't support BCM2835.

Debian's "armel" architecture is ARMv4. It doesn't use BCM2835 to its full potential.

So the BCM2835 is awkwardly positioned in-between Debian's two stock ARM 32-bit architectures, which motivated the decision to recompile all packages for a BCM2835-specific "armhf" distribution.

In a sense, it's a historic artifact.

Interesting.

For my "canned mainframe" (https://github.com/rbanffy/vm370) ARM images, I'm using Debian as a base, since it has the armv6 architecture listed and I didn't notice any adverse effects.

I wonder if I should have used an RPi-specific base image.

OTOH, that'd render the container image incompatible with other 32-bit ARM boards.

I've always said the same thing about Ubuntu.

At the end of the day, it's just how they decided to do it.

in addition to the existing answers, they waited to upstream all of the drivers for the new video hardware until after the announcement. So having their own distribution allows them secrecy while developing the hardware

32-bit raspberry pi os is based on raspian instead of debian for the architecture reasons explained by a sibling comment. But 64-bit raspberry pi OS is based on actual Debian, with extra repos added for rpi-specific software and patched versions of some Debian software

Maybe quality control?

If it relied on upstream entirely and Debian breaks something, people would think Rpi is an unstable hardware.

Wayland, PipeWire, and NetworkManager. Three huge changes in one release. Congrats, Pi software team!

Will be interesting if configuring a Bluetooth sink using PipeWire is less of a hassle than Alsa, which was really unpleasant. Having all the codecs on a Pi Zero is nice!

I've had a much more pleasant experience with Bluetooth under Pipewire, though fixing issues such as incorrect Bluetooth profiles requires understanding still-pretty-young projects like WirePlumber.

Those are upstream Debian changes. Well, not setting Wayland default or QAing it for rpi but still, most it's a Debian's team job

If you read the article, they developed their own(?) lxpanel clone for Wayland apparently, so there's some custom work there. Of course I wonder why that wasn't done upstream (or maybe it was, and it's just not spelled out).

> As you probably know, Debian releases are named after characters in Disney/Pixar’s Toy Story franchise…

Huh, TIL this convention.

https://www.debian.org/doc/manuals/debian-faq/ftparchives.en...

Thanks for the link, plays into a thought about how far Disney's effect on copyright extends to open source.

Just an en-passe mention (under "Documentation") to the new Bookworm way of handling Python modules which already pissed me off^W^W^W surprised me in Ubuntu 23.04. Basically you cannot install modules with `pip3 install module` (or `python3 -m pip install module`) without using a virtualenv or using the .deb packaged version in the repos. I've got used to it but it was a headache for a few days after the first upgrade I did.

Can you use ensurepip to install pip, and then use pip from there? I recently had to deal with missing pip on the last ubuntu LTS version, and that's what I had to do.

Makes some amount of sense to stop novices messing up their python install right off the bat, much like I have on most of my machines.

I think `pip install --break-system-packages` will still work if you insist on mixing pip and deb.

> you cannot install modules with `pip3 install module` (or `python3 -m pip install module`) without using a virtualenv or using the .deb packaged version in the repos.

Wasn't that always a really bad idea to do, though, since it was so easy to break your system Python that way?

I'm not saying it was the best way but if you just ran some small 3rd party script or wrote some small Python script with non packaged deps, well, it was easy, fast and usually safe. But forcing this new behavior is surely a good decision.

I'm fairly ignorant of the python ecosystem beyond extremely casual use for a few scripts, how does it break your system to do this?

There's a few ways Python packages can be installed.

They can be installed at the system level, making the package available to any Python program on the system. This can get problematic if not all of your Python usage can function with the same versions of all system-installed packages.

They can be installed at the user level, sitting in your homedir instead of somewhere under /usr. This gives some isolation from the system defaults, but may just move the problem down a level - you still have the potential for version conflicts.

They can also be installed at the virtualenv level, and now the package install is specific to Python programs launched from that virtualenv. Upgrading a package can only break stuff in that virtualenv, it can't spread beyond it.

If you mostly use stuff from the standard library and stable packages that don't change their APIs much, you can go a long way without venvs before it actually bites you.

The thing that _really_ irritates me about debian (almost the only thing, actually) is how flipping hard it is to use it as a development environment/system.

It's _great_ as a system, but for the love of $DIETY, why isn't there some sort of way to say: `sudo apt install $ALL_THE_COMPILERS@[latest||version]` ...semi-similarly to the "exceptions" made for frequently updated packages like `firefox`, `chromium`, etc...

I know of `asdf`, but it seems like a big miss that it's sometimes so hard to use debian "properly" (ie: via `apt ...`) to actually develop.

Or similarly, wouldn't it be great if similar to `LSP-language-server-protocol`, there was something like `PSP-package-server-protocol` where `npm`, `go.mod`, `pip`, etc... were all like: "yeah, the debian system packages for python, perl, etc. are great for debian-on-debian", but we should also support pulling an arbitrary project at an arbitrary version and make a best-effort attempt at being able to isolate, share, manage, upgrade packages between them.

You're right, it requires a lot of research and reading to it the "right" way. Also, there are multiple correct option, ie. you mention `asdf`, however I'd recommend `volta.sh` instead.

From my experience, I'd recommend sticking with debian packages for everything if possible (they may be outdated, but might still work).

Altenatively, you can also cheat a bit, use docker to separate your dev environment from your system.

Isn’t Volta only for JS?

I always recommend rtx.

(belated), but thanks for the `rtx` recommendation, I checked it out and it seems like "the right thing". Also keep an eye on the `pixi` tool which got some visibility here recently: https://github.com/prefix-dev/pixi

> Or similarly, wouldn't it be great if similar to `LSP-language-server-protocol`, there was something like `PSP-package-server-protocol

If system package managers could support the same use cases as packages managers for your favorite languages then the latter wouldn't need to exist.

Distribution maintainers have a very different idea of how dependencies should be managed than most developers using those package managers do.

/usr/bin/debian-configure-thingy.py => import foo_foo_foo

sudo pip3 install bar_bar_bar

...depends foo_foo_foo@v4.5.6

...but debian-configure-thingy.py expects foo_foo_foo@v1.2.3

(or import path conflicts or whatever)

Yeah it's insanely easy to break stuff.

That's a very good change. You should never have a reason to vomit your app's deps into system, far too easy to cause problems on upgrade

Do you know if `pip3 install --user module` still works?

Suse Tumbleweed made the same change and it doesn't work there (I use it through WSL if it changes anything)

You can by deleting the EXTERNALLY-MANAGED file [1]

I know some Python maintainers think users are too dumb to work with system Python, but there are also some packages that are really hard to deal with on a system level if you try installing them in a venv. Plus venv's are dumb (I use Docker if I need more than one environment for specific apps or dev work), since it's another layer of abstraction I'd need to learn just to use Python.

I'm not forced to use virtual environments for PHP, Ruby, Java, etc... why for Python?

[1] https://www.jeffgeerling.com/blog/2023/how-solve-error-exter...

I think that what bothers you with virtual environments is simply the name. A venv is literally just a directory with files in it. Most of the languages you mentioned actually use a very similar approach to installing dependencies, they just don't call it "virtual", they call it GOPATH, node_modules or something similar.

As a developer working mostly with Python I am so glad to have way to install a dependency in a directory that won't break my OS and without having to use docker. I still use docker for lots of things, but not as a way to resolve issues that can be resolved by creating a new directory.

>I know some Python maintainers think users are too dumb to work with system Python,

No, this isn't python maintainers thinking users are too dumb.

This is distro maintainers fed up with false bug reports and broken systems caused by python's inability of versioning the packages fed to programs without venv.

Using --break-system-packages is a far clearer way to do the same thing.

This is fundamentally a python problem.

python with pip has no ability to version modules correctly in the main package store and feed them to applications by version.

If one system installed package needs numpy 1.26.0 but another system installed package that'll brick your system needs 1.13.5. The result is, "WTF THESE BUG REPORTS IN OUR DEBIAN ISSUE TRACKER".

So virtualenv is now enforced, for everyone because that's python's only mechanism to feed versioned packages that avoid breaking packaged system software

But hey, at least they added the arg `--break-system-packages`

to pip.

LOL

You can also edit/create a pip.conf and add

[global]

break-system-packages = true

This actually improves the user experience for people who acknowledge what they are doing.

Or for people who just don't care, ie. if you're just trying something in a docker image.

The way I run my NAS I have a different user for different purposes so just install modules user level with --break-system-packages. Is there a better way?

What's the current status of 64-Bit on Raspberry Pi? My knowledge is pretty outdated, I remembered that initially, even the Raspberry Pi 4 was still running 32-Bit Raspbian and that 64-Bit was a hack/experimental. Is that all resolved and done and dusted now, and we have a full 64-Bit system? Or will that require Bookworm?

The official 64-bit Raspberry Pi OS was released in February 2022:

https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/

Since then it was up to the user to run the 32 or 64-bit OS. Still seems to be the case with this new version.

Thanks!

Rpi-imager and the website don’t really make it clear but these days the 64-bit OS is the preferred one on Pis that support it

Interesting, I’m curious how the OS is assembled together, since they discourage users from in-place upgrade.

Debian 11 was just a regular Debian with their repos added on top. It feels like since the times of Raspbian they are trying to thin out their changes to Debian (which is a good thing). So to see the recommendation to re-image is quite unexpected.

Sorry, about wayland, I thought it still has some disadvantages and also still not widely supported such as VNC related things? Could anyone enlighten me?

I've been using X to view graphical applications running on the Pi from my desktop - will this still be possible?

Do you mean X forwarding of a window over SSH? You've got a couple of options:

Use waypipe (https://gitlab.freedesktop.org/mstoeckl/waypipe). I haven't tried it myself, but I gather it's not as "easy" as X forwarding.

Use VNC instead (remote the whole desktop instead of just one app)

It just struck me that using Disney character names (given Disney's record on copyright) for Debian suddenly feels jarring? (saying that as a 20+ year linux verteran).

The naming convention was established well before Disney acquired Pixar. Arguably those aren’t Disney character names. (Except for characters new in Toy Story 3 I guess.)

how does it look like for people on older models? also does the compatibility issues should not matter much if you use it in headless mode AIUI.

No mention of the kernel using 16K page sizes?

I don't know what sort of mention you are looking for but there is a blog comment that mentions it.

Did they fix the USB 3 issues with SSD’s yet?

Reply on news.ycombinator.com