Hacker News Reader

Comment by baldfat | Hacker News Reader

I don't understand why BTRFS isn't gaining more support in Linux. I prefer BTRFS to ZFS with Fuse.

Probably rhymes with "btrfs balance".

I'v had the misfortune of using btrfs in production with a few hundred machines on Ubuntu 14.04. It's one of the most finicky filesystems I've ever used. It's probably better in newer kernels, but if you have a lot of churn it requires constant care and feeding and tends to cause kernel softlockups fairly commonly.

Also this: http://blog.pgaddict.com/posts/friends-dont-let-friends-use-... and this: https://www.phoronix.com/scan.php?page=news_item&px=CoreOS-B... and this (old but a few of the points still apply): http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-...

That being said I'm sure there are plenty of use cases for which btrfs in its current capacity is more than sufficient.

At CoreOS and we tried really hard to make btrfs happen, but it really came down to how different it operated than other file systems. It was mainly a UX issue and thus fell into my lap.

The major issue is that regular debugging tools that folks have been using forever like `df -h` aren't just non-functional, they actively misrepresent the state of the file system. The most common example is indicating that you have plenty of free space when in fact you're out. We had to write a lot of documentation to teach people how btrfs works and how to debug it: https://coreos.com/os/docs/latest/btrfs-troubleshooting.html

The second major issue is that rebalancing requires free space, which is the problem that most folks are trying to fix with a rebalance operation. Catch-22 in the worst way. Containers vary in size and can restart frequently, churning through the btrfs chunks without filling them up, leaving around a lot of empty space that needs to be rebalanced.

I hit that rebalancing needs space (and therefore can ENOSPC) issue at work when trying to compile ZFS on CoreOS on Digital Ocean before CoreOS switched from btrfs to ext4 and overlayfs. Getting ENOSPC on btrfs rebalance when you are seeing regular writes return ENOSPC is a really annoying problem.

> I'v had the misfortune of using btrfs in production with a few hundred machines on Ubuntu 14.04

You are not alone. btrfs seems to be kind of stable - as in does not corrupt itself anymore - with 4.2 but it's been a nasty ride.

It's an experimental filesystem that is neither complete nor stable yet. I wish this would be better communicated.

It's needless frustrating: If you search for btrfs you come across a few slide decks that tell you: It's fine you can use it... after the first strange problems you'll subscribe the mailing list and every other day there is some post that shines some light into strange behavior and stuff that is not implemented.

If you want checksumming on your single hdd backup disk btrfs is fine. For everything else you are up to some surprises... basically everything volume management and RAID is pretty much experimental and has strange behavior.

Performance is not even a topic. I remember the ML discussion on this OLTP blogpost and the majority of responses was: Don't run databases on btrfs you stupid! I'd rather would read a technical discussion about the problems but from reading the ML it seems like it's too complex and few understand the complexity.

@bcantrill called it a shit-show in some podcast and while it maybe technically not true it sure does looks like that.

If you want peace of mind use mdraid+ext4 (or xfs) - ZFS on Linux has a lot of problems for heavy usage but the community is IMHO more invested in making it a good Linux citizen.

On the other hand: This stuff is complicated and everyone expects miracles. I'm just looking at it from sysadmin perspective and on Linux both suck at the moment. But ZFS won't eat your data and has far better tooling.

If you need something that works for high load on Linux I'd use neither.

This isn't ZFS in Fuse - it is a native kernel module.

BTRFS has yet to reach the stability of ZFS.

I wonder if the licensing of the ZFS kernel module will cause any issues with its inclusion in Ubuntu?

Canonical's lawyers OK'd it which is why they are including it.

Weird. What changed in the last 10+ years?

Nothing. It's the same as nvidia's non-gpl kernel modules. The simple fact is they will never be accepted upstream, but that matters little to distributors of ubuntu's scale.

In the case of Nvidia's modules, Nvidia's proprietary licensing disallows distribution of a prebuilt nvidia.ko (as that implies distributing a modified version). Coincidentally, their license terms for the OpenSolaris driver have no such restriction and the OpenSolaris descendants distribute the prebuilt module without potentially violating Nvidia's license terms.

Amazingly, their Linux licensing used to be worse. They used to claim you were only permitted to install the driver on one computer within an organization

Someone hired better lawyers. All these ridiculous EULA and click through licences and idiotic mandatory registration systems we see, I can't help think many companies would benefit from hiring better lawyers. Get rid of the timid who default to 'no' in order to protect their own arse, hire people who help you get where you want to go.

In this case, I can't even see any real liability issues - even if Canonical did get taken to court there are no damages since the software is free of charge.

> no damages since the software is free of charge

Many people can and do charge a lot of money for OSS products.

Pretty sure Canonical saw an opportunity with container management, plus interest in ZFS, decided to get over the unfortunate licensing issue and support the module.

People decided to listen to lawyers who read the licenses instead of listening to statements by people claiming to know how things work without actually reading either license or asking a lawyer about it. That is quite literally the only change.

Didn't Sun use lawyers to design the CDDL to precisely prevent this situation?

No, I've seen talks by the engineers behind Solaris (I don't recall who at the moment) that strongly indicated the Sun lawyers didn't go out of their way to be incompatible with the GPL -- they just wanted a license that allowed them to split proprietary and open code (as they didn't have the right to open up all of Solaris due to licensing agreements with third parties etc) -- and still being able to distribute both open and traditional closed Solaris. This led to the "per file" license nature of the CDDL -- and unfortunately to the "additional limitation"-bit that makes it incompatible with the GPL.

If it was done again today, they might have gone for the Apache license as I recall -- and avoided some of the unfortunate issues.

I check in on btrfs every 6-12 months or so. To date it has always seemed too unreliable compared to ZFS. Lack of decent RAID 5/6 support is another major difference.

Ubuntu 16.04 will have ZFS support in the kernel, via a kernel module. They are not using FUSE.

I have been using btrfs for a couple of years. The nicest thing I can say about it is that it is both conceptually elegant and has made my backup practises bulletproof.

ZFS on Linux runs natively though. This is not the fuse version.

Granted, it's been a couple of years, but I tried BTRFS when openSUSE made it their default and I had filesystem issues that I've never had on anything else. I'm sure it's progressed a lot since then, but I expect it will be behind ZFS for a long time in terms of stability.

SUSE seems to recommend XFS for production data and btrfs for the rootfs that is a read mostly workload that is unlikely to trigger ENOSPC. It is not what I would consider a great endorsement of btrfs.

ZFS on Linux (i.e. kernel module, not with fuse) is very stable in my experience. I have not heard the same about btrfs. I tried ZFS on fuse once, but the performance was abysmal.

ZFSOnLinux does not use FUSE.

Reply on news.ycombinator.com