Hacker News Reader

Comment by rsync | Hacker News Reader

I am genuinely curious how a ZFS "special" device[1] which absorbs all metadata for a pool would help this kind of a workload.

The "special" device is a full-blown vdev that you add to a pool (it is not a cache) - typically a 3 or 4-way mirror of SSDs.

Now all metadata read and write happen at SSD speeds.

We wrote about this in the rsync.net Technical Notes for Q3 of this year[2].

I know what this kind of SSD based vdev does to typical mixed file performance but I'm not sure how metadata-heavy a postgres implementation is ...

[1] Yes, they really are called that.

[2] https://www.rsync.net/resources/notes/2021-q3-rsync.net_tech...

View on HN

Would separate SSD metadata devices help if the pool, as in Heap.io's case, already consists entirely of SSDs? It's obviously a win for a use case like Rsync.net's where the data is less “hot”and therefore uses much more cost-effective HDDs.

Would be interesting to see if Optane or even just some faster SSDs for the metadata would give any noticeable improvement. I imagine latency would be more important for metadata than throughput, so perhaps SSDs would be more or less equivalent, but I'd be really interested in seeing the numbers for Optane

This is actually one of the golden use cases for optane! Back in the pre-optane day, the server company I worked for would back SSD/HDD zpools with a device called a ZeusRAM drive: https://www.ebay.com/itm/234256677232

It's all about the latency.

The benefits would be that metadata type FS queries are OOB from the actual data, theoretically allowing more IOPs on your data disks spent on actual data.

If you just add metadata SSDs you’re also adding IOPS to the pool. The question then becomes whether that improves performance more than if you added those same SSDs without the split (my guess would be not splitting is better, since the IO load will be better balanced across drives).

It could possibly help a little bit if the special device is an extra-fast NVMe SLC, and the bulk are SATA MLCs.

Thanks for posting about this. TIL about special devices.

Reply on news.ycombinator.com