Hacker News Reader

Comment by eric_the_read | Hacker News Reader

One thing that's been unclear to me with ZFS: How should I design a ZFS-based storage system for extensibility? From what I've read, it seems like you can't add devices to a pool once it's been created. I'd love to create a small-ish NAS right now to store, say, the top 10% of my blu-ray collection, and then incrementally upgrade it over time as I get funds.

View on HN

The easiest way to do this is to use mirror vdevs, that way you can add disks two at a time. Of course, you miss out on RAIDZ functionality that way, but that's the simplest setup.

I've gone ahead and actually created a new array using btrfs raid10 instead (with an eye towards a RAIDZ2/3 like configuration whenever the btrfs guys get around to fixing it); btrfs allows for online reshaping of the mirroring configuration, so it doesn't have the limitations of ZFS in this instance.

Can you do btrfs with FreeNAS? I didn't think that was available.

No, you have to go with Linux. I'm using the latest Ubuntu server LTS (since ZFS is also included, and because I'm familiar with Ubuntu).

I don't have experience with FreeNAS unfortunately.

If you could, that makes no sense. ZFS is superior.

Unless you want to reshape your array. Then md-raid is superior. Btrfs's raid implementation has had some pretty bad issues recently and has been declared "experimental" as a result.

This is inaccurate. Raid56 have always been labeled experimental, but recently, data corruption bugs were found which make it dangerous to use.

Other raid levels (raid0, raid1, raid10) work fine and are considered stable -- at least as stable as the filesystem itself.

Either way, if you care about the data, you must backup, no matter what fs you use.

Apologies, I only saw the recent mailing list post which basically said "we're removing the ability to even compile support for raid56". I didn't mean to say that btrfs was overly-zealous in their trust of new code (I personally am super excited for the future of btrfs).

I agree that mirror is the best (i.e. simple and easy) way to do it, and to avoid raid-greed (http://constantin.glez.de/blog/2010/01/home-server-raid-gree...). Mirror (vs Raid-Z) gives us better fault-tolerance and flexibility.

Agreed, mirror vdevs are the way to go. Here's a post that explains the tradeoffs between RAIDZ and mirror vdevs in great detail: http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...

I'm still not sure what setup I have. I really should have written down every decision I've made while building my NAS (I think I've now found articles that disagree with every single choice I made)

    $ zpool status
  pool: BoxODisks
 state: ONLINE
      scan: scrub repaired 0 in 9h20m with 0 errors on Sun Jul 31 01:31:31 2016
    config:

        NAME                                            STATE     READ WRITE CKSUM
        BoxODisks                                       ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            gptid/18059e22-b6a4-11e5-9cca-0cc47a6bbf34  ONLINE       0     0     0
            gptid/194649b1-b6a4-11e5-9cca-0cc47a6bbf34  ONLINE       0     0     0
          mirror-1                                      ONLINE       0     0     0
            gptid/1a86b3cc-b6a4-11e5-9cca-0cc47a6bbf34  ONLINE       0     0     0
            gptid/1bcd3ad6-b6a4-11e5-9cca-0cc47a6bbf34  ONLINE       0     0     0
        cache
          diskid/DISK-S24ZNWAG903847Lp1                 ONLINE       0     0     0

I think I have mirrored vdevs.

What does that mean for replacing failed disk/s? What does that mean when I want to upgrade to 8TB drives? The biggest "mistake" I made was buying a very expensive 4 bay enclosure - 4 drives is not enough.

When you want to upgrade, you can offline and replace each disk, then resilver, in turn. If the pool is set to autoexpand, you'll eventually get a total capacity of 16TB. You can do this for all the disks, or for each mirror separately in two groups of two.

https://jsosic.wordpress.com/2013/01/01/expanding-zfs-zpool-...

Yeah, I saw that before, but that isn't quite what I wanted, if I'm reading it right. That tells me how to replace all the drives in my array with new ones; I'd prefer to (for instance) start with 4x 4TB drives, and add new ones as my storage needs build up.

http://docs.oracle.com/cd/E19253-01/819-5461/6n7ht6qvk/index...

Nice! Thanks for that.

I don't know if that works on OpenZFS, though. Probably.

You can expand a pool one RAID array at a time, but growing one drive at a time isn't really possible except by not using RAID; ZFS doesn't have a true equivalent to the BTRFS rebalance operation.

You can add vdev to pool anytime. Removing them is not as easy, so some planning is necessary.

For others, I found this, explaining the process: http://alp-notes.blogspot.com/2011/09/adding-vdev-to-raidz-p...

Downside is, you lose a significant chunk of storage (the author measured > 20%), but you can add new storage to a pool online.

Reply on news.ycombinator.com