Hacker News Reader

Comment by zdw | Hacker News Reader

I have this same hardware. A few notes:

- For best performance with ZFS, you want a lot of RAM, and this unit will take 8GB of ECC RAM. You want ECC for data integrity in memory, as ZFS does nothing to prevent in-memory data corruption (there's an article on this here [pdf]: http://research.cs.wisc.edu/wind/Publications/zfs-corruption... )

- You probably want a few mirrors, not RAID-Z, if performance is an issue.

- You're better off with FreeBSD or Illumos kerneled distros (which have run ZFS for years, and have it in their mainline kernels), rather than Linux (which never will have ZFS in mainline for licensing reasons), for stability alone.

- You can get an IPMI card for this unit if you want remote manageability.

- There's an internal USB port if you want to boot off of that. It's kind of handy.

View on HN

Good points - people do need to remember that ZFS was designed for going big. IT does nothing to prevent memory corruption by design.... it was assumed (required?) that your server would use ECC ram.

You also need to ensure you have adequate ram for various cacahing, and CPU power as well for checksum calculation, among other things. It's not a lightweight FS. (in-memory corruption is an issue with all filesysems - just somewhat more-so with ZFS because it was designed specifically to assume you had reliable ram. You want ram to store the hash cache or whatever too....

One could chuck an SSD in there for cache, if memory is a limit, that should speed things up drastically.

And as with all raid-like systems - you want an appropriate number of hot-spares, cold spares, and a system that monitors it and acts appropriately, especially if you're going with huge drives on slow busses.

You want regularly scheduled scrubs, not too many snapshots, probably disable atime (noatime) to speed up those scrubs, and compression probably off....

dedup I'm still on the fence about - I leave it off, I can only see specific situations where it would be truly useful (dedup+verify)

Unless you're using enterprise class SSD's with SLC flash and ultracaps to guarantee that the writes make it in cases of power loss, you're risking your data if using this as ZIL.

Cheap consumer class SSD's are generally fine for L2ARC.

ZFS has an interesting caveat -- when you're adding disks to an existing pool, you can't simply add an arbitrary amount of new disks and rebalance the data across the existing disks and the new disk(s).

I'm using FreeNAS and 5x2TB raidz1 in the N36L and if I need more space, it's going to require some very careful planning.

That said, the hardware is a STEAL for the price, and FreeNAS + transmissiond + remote-gui rock my socks off. I'm strongly considering just buying a brand new one when I'm out of space on the N36L.

I have it's predecessor (N36L if you want to google for prices) Yes, it's very nice; but it's not that quiet. I have 4 x 2Tb in raid1, but might switch to btrfs sometime; I'm a little leery of both zfs and btrfs at the moment. In Fedora because it's most similar to what I use at work (RedHat), not really any other reason.

ZFS has been in production in Solaris 10 since June 2006 and is quite stable and production tested.

btrfs is still considered experimental, even though it's been in the mainline Linux kernel since 2009.

You might look into the KVM support in SmartOS if you want to run other OS's on top of Solaris/ZFS.

SmartOS doesn't support AMD processes (or at least it didn't when I got my Microserver).

This is something we've been working to fix in the community! The good news is that most of the development and testing so far has been on an N36L-based Microserver, where the experimental AMD KVM support seems to work pretty well. There's a github repo -- https://github.com/jclulow/illumos-kvm -- and hopefully I'll get time soon to respin a current SmartOS ISO with the AMD-enabled bits.

Please be sure to submit to HN when you have a new respin ready. Pretty sure there are a lot of potential new SmartOS users on here who'd like to use KVM with AMD. Got my eye on a N40L-based Microserver myself.

Oh, that is fantastic news. I may have a server rebuild in my near future.

Great to see that SmartOS developers are active on HN.

I'm interested in SmartOS but I've heard some things about it not being production ready. What's your view?

Also, being able to boot from an internal volume (not USB) is also crucial to me, as the alternative is simply not an option. (The "best" case, using a USB drive, is not an option as our server is in a shared rack and you run a risk of someone unplugging it).

Reply on news.ycombinator.com