I’ve always wanted to use NetBSD for an application for an embedded system / IoT device but never had the pleasure (yet!).
I’ve always wanted to use NetBSD for an application for an embedded system / IoT device but never had the pleasure (yet!).
I'm asking because i have not touched any BSD for over 2 decades...and I'm getting the itch to try some out...and was wondering if for server-type use cases (like you noted) whether OpenBSD is preferred over FreeBSD or the reverse, and why? Thanks in advance for any feedback you might provide!
NetBSD is, however, the gold standard for an OS that runs on just about anything. Their (maybe unofficial) slogan has been “Of course it runs NetBSD!”. Their logo has a flag in it because they “plant their flag” on so many platforms.
And, I mentioned NetBSD for embedded stuff...but really, i *think* its that NetBsd is simply installed on tons of different hardware....so not only embedded....i kinda remembered that about NetBSD.
But, its the other BSDs - in particular FreeBSD vs OpenBSD - that i always forget the differences...but got it now. Thanks!
openbsd = security
netbsd = portability
freebsd: performance, features, drivers, software compat - closest to linux in utility & usability though unlike linux in execution
openbsd: safety for exposed services
netbsd: portable across many cpu & hardware platforms - big-endian powerpc sun, hitachi sh3 jornada, etc, easiest to port to a new arch
The way that the BSDs differentiate cannot be reduced in this way, not least because there is a lot of what Justin C. Sherrill (of the DragonFly Digest) calls 'cross-pollination' amongst the BSDs.
A case in point:
Superficially, and erroneously, one might observe that OpenBSD, NetBSD, and FreeBSD have nvi, and only DragonFlyBSD has nvi2. In fact there was a three-way fork of actual Bostic nvi, all of them making revisions and leaving the original behind, and then things got really complex with nvi2 taking from OpenBSD's nvi, and FreeBSD's nvi taking from nvi2; not even getting into the existence of nvi-m17n along the way and how there are nvis in base and nvis in ports. (https://news.ycombinator.com/item?id=48132452) One cannot divide the BSDs up into those that have nvi2 versus those that have nvi.
The split is complex in other areas, too.
You know even though I said the execution is unlike linux, in fact, there are many many details that are just like linux! What a freaking ignorant liar eh? There's like 100 things like that you could say. No wait, no way it's exactly 100. There's obviously some other number like 105 or 612 things like that. So superficial and wrong!
But it's not a matter of surface area that makes openbsd solid, it's the priorities while writing that affects how every little thing has been written over time.
You can write 10 different versions of a function that all work and are all nominally perfectly free of security gaps.
Yet they will all still be 10 different levels of robust. Some versions will fail as soon as some assumption is violated, and some make fewer assumptions and remain safe even when varying amounts and forms of "that can't happen" happens.
It's not just cosmic ray bit flips either, or a hacker trying to do power glitch attacks or rowhammer etc, stuff that makes the hardware violate it's promises. But stuff like a different developer updating something 15 years later who is not the original and does not realize every single facet of how it works and just how the current implimentation covers all possible edge cases, and so doesn't realize how their change opened up an edge case that was covered before. With fragile code, the new code simply has the new security gap until someone discovers it the hard way. With robust code, it's more likely to still be safe. The edge case maybe makes it fail to function, but not in a way that anyone can use productively.
Not that freebsd is exactly swiss cheese. These are all relative. I would and do rely on freebsd any day.
Very salient comment there! And, while not the only reason for me, but what you noted is sort of one reason that's triggering the itch in me to go back to playing with the BSDs. Don't get me wrong, I still do love fiddling around with some areas of linux once in a while....but then, there are other uses/areas where i just want a server to do its thing, and for my maintenance to be a little less (at least less than some linux distros require). So maybe i'm not the only one? :-)
As a teen I had infinite time to compile Linux and debug stuff. Now I just want to spend time with family/outdoors and not be stuck in a windowless room negotiating with a black box. ;-P
And, wow, do i miss the old X-window workstations...well, i should clarify that i LOVED those (I think they were Sparc?) workstations that ran Solaris or SunOS back in the day! Man, that takes me back some years...but i really loved those machines! :-)
No 32-bit sparc anymore (only UltraSPARC, aka sparc64).
No SunOS compatibility (despite Theo de Raadt inventing it for NetBSD, before being copied by other BSDs).
https://marc.info/?l=openbsd-tech&m=161435521906992&w=2
> Technically there's a niche flavour of 68k that still is supported because of a very dedicated man in Japan
luna88k, while related, is not 68k.
>luna88k, while related, is not 68k
I misremembered it as being similar to the relationship between the 6502 and the 65C816
https://www.youtube.com/watch?v=btwiiZw3B2s
Kenji Aoyama truly is aligned with the best of the hacker spirit. As for getting your hands on a luna88k, I have no clue. The only thing I managed to find was a broken one that sold for ~USD 750 at an online auction.
http://miod.online.fr/software/openbsd/stories/m88k.html
One interesting bit of trivial is Luna-88k workstations were heavily used to implement CMU Mach (which would eventually be used by Apple).
Another part of my nostalgia with those old workstations (besides the core OS) was the desktop environment, i think CDE or motif or something like that. Something about the look and feel of that DE i always thought was cool!
OpenBSD on the other hand is perfectly happy to leave oodles of performance on the table for security. They were the first OS to completely drop Hyperthreading support for example, years before spectre/meltdown.
So with these things in mind, FreeBSD is a lot more performant.
NetBSD is small and simple. It's a lot like an old-school UNIX. It makes a decent platform for small services. I run bind and dhcpd on a NetBSD machine. The source code is very pleasant to read. It uses the pkgsrc software repository. It's my preferred platform for writing POSIX code.
OpenBSD still carries much of the general feel of NetBSD and can fill a similar niche on a network, but the security focus stands out in their documentation, subprojects (OpenSSH, LibreSSL, OpenNTPD, etc.), APIs (see pledge(8)), and policies. It makes for a great firewall. I'd say it also requires the most know-how.
All of them have excellent documentation (especially compared to Linux distros) and the base system is developed alongside the kernel, giving you a very consistent experience compared to Linux distros where everything is developed in isolation. If you write C, it's worth keeping a BSD system around just for the manpages and to make sure you're not letting Linuxisms creep into your codebase.
Just to clarify. It's not emulation in the sense it's slower or something. They call it compatibility layer, which is better, but also nobody knows what it means.
This is simplifying a bit, but it's essentially "Linux is just a kernel" so the interface is just Linux syscalls, so the FreeBSD kernel when executing a Linux binary simply answers like Linux (so it has those system calls). How this is used in practice is that on your file system you have Ubuntu/RedHat/... "installed" (so the files and the file hierarchy are lying there) and you either directly or in a FreeBSD jail execute things in there or the binary you have.
I don't know how well it works in the present but in the past that means you could simply download the Unreal Tournament 2004 multiplayer demo or Enemy Territory or other games and just play them as if you were running Linux, 3D acceleration and all, without VM without real emulating, just the kernel providing what a Linux kernel would provide.
Also "heavy" is very very relative and subjective. You can totally have a tiny FreeBSD and a huge OpenBSD and one could argue OpenBSD is "heavy" because it comes with three window managers, an HTTP server, a full blown SMTPD server, ACME client and a ton of stuff that eg a server install of Debian or Ubuntu doesn't come with. But also if you run eg. ZFS things are heavy of course. FreeBSD has however had a time when it tried to strip a lot of stuff from the default install and make stuff either optional or make things available through ports/packages only.
And also there are surprises to be had with such overviews: Eg. your Lenovo laptop likely will give you a more "out of the box" experience on OpenBSD compared to FreeBSD with things like simple wifi setup, sound often doing the right thing (work, come out the right place, etc.) compared to FreeBSD. Also with stuff like HTTPD with ACME being available in a simple way after install I'd say OpenBSD is easier than FreeBSD.
FreeBSD to me feels a bit more like "it can be everything you want it to be". Ports and packages can be complicated if you just start out, compared to OpenBSDs "just use packages" stance. On OpenBSD things in my experience are more of a "it works or doesn't" and when it works often out of the box and/or with docs, while on FreeBSD it's more like it throws some tools into your direction you can build stuff (poudriere, jails, a build system with many options). So it's really cool if you want flexibility but a bit more like you have to figure out if it's possible and how. But that might simply be because of the use cases I used it for.
That said all of them are real general purpose systems, unlike eg. some Linux distributions. So it's not like "OpenBSD is for routers" even though it often seems like it. There are time when the GPU support is better on OpenBSD than FreeBSD's. But also FreeBSD has official NVIDIA drivers, so it's all not that clear cut.
I will defend my "heaviness" argument, though. Sure, you can run OpenBSD on large hardware, but it's not going to be able to take advantage of it like FreeBSD can. Which makes sense if you think about it - FreeBSD optimizes for heavy workloads. Conversely, if you set up minimal installs, OpenBSD will be smaller. Again, that makes sense, since OpenBSD focuses on security over features (plus the only truly secure code is the code that doesn't exist). There's a lot of overlap in the middle, of course.
I wouldn't use OpenBSD for a NAS, and I wouldn't use FreeBSD for a diskless firewall. Not because they can't do those things - they just each have their strengths and weaknesses.
We've run into instability issues with the newer Linux kernels (starting with 6.x, I think) and have had to stop upgrading.
https://lists.gnu.org/archive/html/bug-hurd/2026-03/msg00100...
In the end Hyperbola BSD will be more free than OpenBSD and the former GNU maintainers themselves...
Now, is LLM code in the hurd a good thing? No, absolutely not. Ignoring the licensing limbo of LLM output that still isn't settled , LLMs make pretty bad code often enough that I wouldn't trust it to work on something as niche and relatively undocumented as the hurd.
Packaging is simple, kernel development and upgrade is simple, etc. Also the kernel code itself is written in a style I like, it's to the point, no useless abstractions, no fuss. I prefer it even amongst other BSDs I tried (netbsd and free*lbsd/dragonfly).
It just feels nice to be able to understand most of your system. It's not as fully featured as Linux, but there is a sense of understanding your system that is refreshing. A bit like if you're on vacation in a small and cute village where life is mundane and calming. At least that's how I feel with it. Mileage may vary.
I ran OpenBSD on my laptop 22 years ago. Back then, a full GUI environment with terminal, web browser, editor: 28MiB of memory for the whole operating system and user environment!
I found a 10+ year old Dell Pentium III laptop in one of the boxes, installed OpenBSD to do some simple connectivity testing, and ended up with a full workstation install and using it for network monitoring and some other random stuff. It stayed in the network/server closet until we moved out of that building just a few years ago.
A while ago I made some blog posts[1] diving into the source code of OpenBSD and FreeBSD (shameless self plug), but haven't had the time recently to write more.
Being able to understand the system, or at least being able to take a quick look when something doesn't work is very refreshing. Not to mention the outstanding man pages. Barely need to google things.
That said, OpenBSD feels unusually coherent (ej. check wifi connection from terminal). The whole system has a level of consistency that's hard to find elsewhere, also between other BSDs.
For pet servers, it usually fits perfect.
That's why I used to run Slackware, and then foud Alpine to be the best - much better than Void or Arch IMO. Works well as a very minimal system, and I know everything very well because of it. It's an ideal approach IMO, the best of both worlds.
You can however have a full-fat desktop environment with xfce4 or gnome and applications like libreoffice, gimp, inkscape, audacity and so on if you wish. I've never tried KDE on top of OpenBSD base but I gather packages are in ports.
I think it is fair to say that the amd64 arch has good support. The i386 platform arch is on a 'best effort' basis these days which is understandable. I've never looked at the others.
I wish I had an OpenBSD development laptop, but I don't have one right now.
I ran it on my personal laptop for several years when I had one, but having a work laptop for these past decades I don't have much use for a personal laptop. I would probably run it again on a nice portable when I retire. It would be nice to focus on being creative on such a machine. Coding and drawing mostly. I will continue to use Linux in my recording studio though.
For mailserver I think it is the best option. And for Gateway, PF is just wonderful.
But even on my laptops I enjoy it. It is rock solid, and I have pretty much no complaints.
And on my laptop, occasionally, to experience it in person.
I used OpenBSD to create the firewalls for our LAN parties when I was at school.
The first shellserver I ran, on an UltraSparc IIi was OpenBSD, gave out accounts to my friends.
And then I used it as a firewall, both professionally and personally, for many years. Until the first Turris Omnia was released, and now I have retired even Turris for pfSense, which is FreeBSD I believe.
But the PF firewall in OpenBSD was superior, definitely to the syntax of IPtables.
To me Linux was a great server OS, and OpenBSD was a great FW/Gateway OS.
WiFi is handled separately by a Ubiquiti UniFi system, but I don't trust Ubiquiti not to exfiltrate data after their underhanded attempt to turn telemetry on a few years ago. OpenBSD WiFI is somewhat mediocre, but it has improved in this release with experimental support for WiFi 6 after years of being stuck at 802.11n.
The closest you will get to the OpenBSD experience on Linux is with Alpine Linux.
This is a big one for me. I've run openBSD and Linux custom boxes as SoHo routers and I just cannot stand Linux firewalls, I've never liked them and IPTables is just terrible. Yes I know there are wrappers around it now but it's still the default everywhere and still used by lots of other software like Docker. I'm using OPNSense now which is FreeBSD based instead of completely rolling my own but I love that it is still BSD under the hood.
One differing opinion I will offer is that I find NixOS to be the Linux distro most in the openBSD spirit despite it being very different from a UX and config management perspective. Alpine is interesting, but it has its own security and compatibility issues, especially around MUSL libc which I have had cause many strange downstream issues over the years, I just hit one recently in JVM GC caused by its memory allocation implementation. I've stopped using alpine altogether because of them.
Work: I need a simple easy to use system that I can configure to meet third party compliance requirements without jumping through hoops. It really excels when you can mostly use the base system there, maybe couple services. For example it's so nice to just have a couple pledge/unveil lines for example in a Go service.
Also super nice for "set and forget" style stuff. For example "I just need a HTTPS server with acme and SFTP". That's something you get out of the box with no third party packages (so everything vetted, pledge/unveil for everything, maintenance just running syspatch and sysupgrade), which is really nice.
Personal: Private mail server, family website, a quick and dirty "watching streams together" service I set up to watch stuff with people not in the same place as I am. prosody to have XMPP for friends and family.
I would NOT use it for "people throw stuff at you" use cases (Linux and FreeBSD do a far better job there). But I absolutely love it for scenarios where you want very very low maintenance. For example that private email server. I don't have time to do big upgrade plans, or "hardening" systems or reinventing the wheel. I cannot afford to do privately what I do in a day job or consulting (setting up or maintaining really rather complicated infrastructure).
I have done that many years with Debian, but the Linux world sadly is a big complex and complicated mess. That's great, when I get paid to deal with it, but annoying otherwise.
And I don't mean that bashing wise. I use Linux, I like Linux, but somehow there is a huge drive to overengineering and then building hacks and weird workarounds that become normalized until it's a proper job. Without wanting to start a flame war, but the whole Docker, Containers, Kubernetes, Helm, Orchestrators, etc. story is a lot of reinventing the wheel and a static executable like a Go service in a container, so essentially coming with a whole Linux distribution even though one never thinks about it that way is just really absurd. That's what executables, processes, etc. were invented for.
And since I've lived through the story and as mentioned make a limit, I understand how that came to be, but it feels like the industry took a wrong turn because it was cool and exciting and then (nearly) everyone decided to use that hammer for everything one could imagine to be a nail. And then the next layer came and the next and the next. But all of them doing things differently. And suddenly to have a Postgres cluster you need Kubernetes, and Helm, but also need to know both PG config and the orchestrator's config, etc.
It's a mess and the OpenBSD people somehow knew that decades before I did.
It ran for over 8 years without downtime, but I’ve had repeated problems in the last year or so.
I used the default partitioning scheme, which makes /usr tiny, and /var huge, and since it is a router, did not install X11.
At some point, they made x11 mandatory for auto updates. This is dumb, because all the upgrade tool is doing is untarring a list of tarballs. So, I had to perform partition surgery from the upgrade ramdisk to make room for X11.
Now, they made some ASLR relinking scheme mandatory, which makes sense, except the relink directory is 1.5GB (larger than the entire rest of the distribution, and far larger than the parts I voluntarily installed!).
For some reason the relink output files go in /usr, which, by default, won’t hold it at upgrade. It really belongs in /var, because it is not immutable, and also, there’s room there! So, I had to repartition the router from a rescue environment again.
They also removed the ability for ntp to sync on machines without cmos clocks, and the alternate config options don’t seem to work. That’s a bit more niche, granted, but my router hw is reasonably common for openbsd use and has that property. You can make it work by using a second utility to force clock sync at boot.
I like that they keep things simple, but they also recently pulled out any semblance of power loss safety for their file system. I’ve had to serial console in a few times to run fsck, which isn’t really the behavior I want from the home router!
They don’t have any way to setup DDNS in the base install, so you have to use a port or pkg. The port I chose was EOL’ed by upstream (ISC), so I’ll probably need to switch to dnsmasq as a dhcp server / dns server, which is fine, but those services are a significant fraction of the attack surface of my router. DDNS seems like a pretty simple thing to implement, and would be really high value for router use cases. Without it, I’d have to assign static addresses to everything on the LAN, then edit DNS records.
I think all this stuff is fixable, but wish they’d take the niche of “rock solid secure infrastructure” a bit more seriously. This used to be a nice “set and forget” weekend project but now it requires attention every few release cycles.
7.8 barely managed to fit in my duct tape and bailing wire partition layout. I’m probably going to switch to freebsd on a box with faster NICs when I finally get a > 1GBit internet connection (hopefully in the next year or so).
If I upgrade to 7.9, I’ll have to give up on using the openbsd hypervisor, since, with the partition scheme that the installer chose, there will no longer be a partition large enough to hold the download sets and also the vm image.
This is particularly frustrating because the boot drive is under 50% full. I’d just do “one big partition”, but they warn against that for good reason - it complicates manual fs repair at boot.
Anyway, I really like the project. It would be nice if they did a “fix common papercuts” release, since I doubt many users are as patient as I am.
If you are looking to install it, either use fewer partitions, or way over provision storage (I was 10x over provisioned at install, and the stuff I use hasn’t grown more than 10-20%) and also make sure you choose much larger partition sizes than recommended. This will add under $100 to your hardware cost, even with the storage shortages.
My one complaint about OpenBSD would probably be lack of resizable partitions. You can expand them, but only if you have free contiguous space and most of the time one partition starts where the prior one ends. It's rarely a problem in practice, as only /home and /var and maybe /usr/local tend to be subject to any guesswork, but it can bite you from time to time as in your case.
I've already done this twice for this box. Its disk is half empty, and the used space is 75% compounding useless bloat:
- 50% of the used space are package sets I never asked for.
- The stuff I did ask for is somehow 2x larger than it needs to be, since they don't randomize binaries in place.
- If they'd actually follow their own filesystem hierarchy standards, and stop using /usr as a build target (very bad things will happen if a crash happens in the middle of that! Why are we making lots of small separate partitions again?!?) then I could just make /var big. Then I would not have to repartition yet again after they introduce /lib/lolz/3gib or whatever in 2027.
Alternatively, if they had a journalling filesystem or still supported soft updates, then I could have one big partition, which would solve it once and for all.
Anyway, I'd argue "take the lan offline, backup the router, repartition and restore" isn't a planned reasonable maintenance task for a router. The fact that its so obviously easily avoidable is really frustrating.
Alternatively, if they just had a "which sets to install?" config option for auto-update (like they do for the OS installer!) then I wouldn't have to do this.
So far that has worked for me.
Some people would also argue that using an 8 year old device as a critical path in your LAN is a risk in itself. Taking routers down to do upgrades is pretty common in the enterprise IT world.
Similarly, if fsck -y is frequently required, maybe just run that way all the time instead of failing to boot, or fix the root problem. I doubt many sers are taking block level backups for forensic repair in case they need to hand assemble inodes.
Anyway, I wish them well. I want a simple, correct and rock solid OS for this sort of use case. The three pillars of computer security are confidentiality, integrity and availability. Hopefully they’ll focus a bit more on the latter two things than they have recently.
The ER4 has 3 ports: 1 was for the uplink, one exposed the WAN connection to the rack, and then the 3rd port became a client inside of the network. I could shell into it from home (he's on the other side of the country) and operate from the residential network and also the server network simultaneously. Worked well enough for a few weeks to keep access around until we could engineer a better solution.
Configuring OpenBSD was really quite simple and rewarding. No insane linux network stack / netplan / cloud-init / bs ... just a few conf files.
obligatory pic: https://i.imgur.com/Mkf9ckc.jpeg