Seagate's new hybrid drive raises some old questions
arstechnica.com
arstechnica.com
- ARC may require manual tuning for size and metadata size to accommodate page cache, in-app caches, file count etc
- L2ARC and SLOGs are very obviously useful for some clear workloads, but have some cliffs you don't expect, such as ram required for L2ARC mapping, unexpected cold/hot/boot behavior
- Reasoning about reliability becomes harder the more tiers you have
Just imagine, to pick a simple example, you're running postgres, thousands of sqlite3 dbs and one huge duckdb (larger-than-ram) on a single such system - it would perhaps manage to adapt to one such workload at a time, but would it balance all three at the same time?
So much as I'd love to see this direction, I can also imagine the pains coming with it.
If they'd get around to merging Special Failsafe [1] and hybrid drives could address the flash space separately like under SAS, it would be a no-brainer.
640GB of total L2ARC, 66% utilized (34% available). 664MB of RAM is used to keep track of this.
The machine only has an uptime of a little over a week right now, but over that period L2ARC had a 93% hit rate.
L2ARC would fill more-completely but the machine is low on free RAM right now; Firefox is being a pig, and that means that some things get ejected from ARC before they can be sent to L2ARC.
It's generally quite responsive despite being a box that exclusively uses spinning rust as primary storage.
In terms of absolute reliability: RAIDZ2 is in-use, which allows two drives to fail without data loss. Adding the L2ARC did not change this: Either or both of L2ARC devices could die in an instant and the machine would keep moving along (at reduced performance).
(I'd planned to upgrade the hard drives to something bigger, which is a simple process with ZFS, and also add more RAM, but the market moved faster than I did and I'll be stuck with this for awhile.)
Would have loved a read-through union filesystem but you pretty much had to do your own.
In the machine room, the moral equivalent was just a rackmount server with large RAM for Linux buffer cache and a RAID card with self-powered writeback caching. This could scale to roughly 8-12 HDDs. If we had need for larger stuff, we'd go with conventional NAS/SAN solutions. Eventually, the SaaS equivalents like EBS and S3 took over, and our group retired its onsite iron.
I also stopped doing LVM Cache as a number of trends converged. SSDs got faster and higher capacity, and changing work environments made the desk-side workstation less attractive. Now I just have a single NVME device and nightly cloud backups.
It's been a while, so I don't remember the LVM Cache intricacies. I think I had a mix of some LVs which were SSD-only, some which were this writeback-cached stack, and some which were HDD-only, allocated from the same 4 physical drives. I don't exactly recall how the caching part appeared in the logical hierarchy. I.e. was it a preallocated pool of cached storage, or just a per-LV assignment of chunks from SSD and HDD PVs..?
RAM is always used first for caching, so it's a given.
Buying a Sony or IBM HSM with 20GB tapes backing a 5TB store (built with 2GB disks) soon looks pretty stupid when drives exceeded 40 and 80GB. (Adjust your calendar or drive scaling to match your expectations.)
Back in the day I worked on a quote for a European weather agency who wanted a CD-ROM archive built with robots. CD-ROM! DVD-R was just coming up in those days. The effective product lifespan would only have been 2 or 3 years until large scale enterprise disk would have superceded it.
Nowadays with multi-TB tapes the minimum viable archivable unit is in the gigabytes. Nobody would be 2million files on a single LTO tape.
HSM is really a product category: do you want write-only archive (think weather data, satellite images, etc). Are some files expirable so you want to actually eventually "delete" files by re-writing tapes with only valid data? What's the allowable time-to-first-byte on retrieval? What kind of redundancy is appropriate? What is the operating cost vs capital cost tradeoff? And, importantly, what database is supporting the metadata and how is it maintained?
Antipatterns:
- grep on a caching filesystem backed with a tape library - monthly retrievals of the same file year after year (even LTO has a limited tape operation lifetime) - management failure/refusal to migrate to newer tech every 4 years - buying 10 years of capacity in advance
You can get a lot done with tar, mt, and bash in a grow-your-own nearline archive with a tape technology of your choice.
https://en.wikipedia.org/wiki/Bcache
https://en.wikipedia.org/wiki/Dm-cache
Sometimes I shoot over 100GB of raw photos and don't finish processing it all in 1 session. It would be nice if that got cached in an SSD rather than having to read the hard drive every time. But it's not a big enough issue for me test one of the caching layers.
It's just pretty niche. And in cases where you want it it's often better to have the hot/caching tier be application data aware, for example for caching web requests you might want to cache different objects for different duration, or selectively purge them when app invalidates some data.
The 8TB version seems like very poor value for money. For comparison, Amazon is selling 24 TB Seagate Exos drives for $799.
Might be a scam, you might get a used or a refurb drive. I wouldn't' trust Amazon for such purchases, if the price feels too good.
This appears to be a logical impossibility. If they had no regrets, why would they have fixed it?
I am not sure if it is a fixed problem now, it has been years since I purchased electronics from Amazon in part due to that exact issue.
When be boycott a product, that company has already proven to be a drag on society. Getting them to change isn't about a Christmas Miracle, although every once in a while someone steps down and the new person is less obviously malignant. Instead it's making sure everyone else sees the consequences. Sometimes that makes it possible for a decent person to get votes past the board of directors, because at least a few fear punishment.
Why would you say this? Guilt and regret are unrelated concepts.
You need a pair because if one HDD starts failing, you need to recover the data. Reading a 20 TB HDD takes an entire day of continuous reads, and a failing HDD can't handle that, so you need RAID 1 or a backup.
A 20 TB disk has a 20% chance of failing within 10 years (at least industrial ones, which tend to be higher quality than consumer ones). But the probability of two disks failing simultaneously within the data recovery period of the first failing disk is less than 1 in 16,400.
The exact numbers aren't the point, but the order of magnitude.
In fact you're so much more likely, you're more likely to buy the 20TB option than if it was the only option you had!
This is well known in marketing but I'm struggling to find its name with Google - it's close to the "decoy effect", which is a form of "menu dependence".
The 1-2TB TLC drives that were the best deal before are more like 4x their lowest price.
NB 'strictly newer' doesn't even have 2.5" at all. Out of ~1000 of laptops at my local, pretty big shop - there are only 6 (six) with 2.5" bay.
I had a 2009-ish 17" Macbook Pro (the Cafeteria Tray), removed the optical drive and replaced it with an SSD, then reformatted the whole thing using a bit of voodoo to turn it into an unsupported Fusion Drive. Gave it to my father when I upgraded a few years later and forgot about it. One day he took it in to the Genius Bar for some kind of service, drew a whole crowd of Apple Store employees who had never seen a Macbook modified under-the-hood before…
Spoiled Americans. That drive is $1393 (14160 SEK) here in Sweden now[1]. A year ago it was 4160 SEK...
Three times as expensive in 12 months seems true for most drives.
I'm just praying no drives in my NAS are going to crap out anytime soon...
[1] example from when I was trying to buy NAS drives a few weeks back: https://www.amazon.co.uk/sp?marketplaceID=A1F83G8C2ARO7P&sel...
The dreaded communism came to the capitalistic states but not the way capitalists expected.
Soooooo... it's gonna be like the old days of Drobo, where a little VXworks box did all the heavy lifting of BeyondRAID and transparently exposed it as a big ol' block device to your OS? Good times.
>Users will have to bring their own SSD
uhhhhhhhh