AWS kills Snowmobile data transfer truck
cnbc.com
cnbc.com
The number of companies with that problem is going to naturally fall over time. Companies that are already using the cloud don't need Snowmobile to move their data.
Illustrative point from the article: "A spokesperson for satellite operator Maxar said the company used Snowmobile once in 2017 to move more than 100 petabytes to AWS from its own servers. Snowmobile was an “ideal solution for that moment in time,” the spokesperson said."
You're not going to get many repeat customers for that service!
I should be able to store my data at the exact same $/TB price as Epic Games, for example. They once mentioned it costs them less than a penny to serve several GB from AWS - why the fuck is it more expensive for me?
At some point of very large usage, you also have to work with AWS teams to give input on your demand forecast.
Unless you have such large usage you get list price, that also gives you the illusion of infinite elasticity.
This is not exclusive to AWS though, all hyperscalers have this policy.
You might not be stupid, but there are people who look _just like you_ who will email support over a six cent billing issue which turns out to be correct anyway - but just burned a couple hours of support people’s time.
Epic probably doesn’t do stuff like that.
I’m sure if you had the same volume as Netflix you would pay less than the retail price to.
If you ever flew on a plane, I can guarantee you that there are probably 20+ different prices being charged for the same seat.
You spend far less money than a corporation, and there are customer costs beyond the actual bandwidth/compute charges.
Why? Different people get different prices for products all the time. It's fundamental to a functioning free market.
0: https://www.theregister.com/AMP/2024/01/11/google_cloud_egre...
It ends up becoming a very different beast than the storage classes w/fast enough retrieval guarantees to need disk, in any case.
The amount of the data being shifted was one of the particular needs, but it was more the _other_ aspects of it being a huge truck that entered in to the equation.
But I also never fully understood why an enterprise would pay so much money to give their data of this size to AWS to manage. It’s so much more expensive. If you’re at the capacity of needing a semi sneakernet, it would be far more cost effective and competitively advantageous to manage it yourself.
For some companies this would mean creating a whole department to manage that.
Now try doing that for multiple multi-thousand person orgs with competing processes, architectures, product cycles, etc.
You'll find very quickly that costs spiral out of control in a very unpredictable and irreversible way.
Self-hosting is something a company has much tighter control of the costs for. Most of those costs are known up front as capex purchases and predictable opex expenditures. They tend to seem expensive compared to the PoC S3 demo you just did, but often aren't compared to the actual realized expenses after migrating.
Costs being well known for self hosting is just wrong. It completely ignores cost of ownership and tail risk for your infra
For my argument not "all large" companies need to be dumb, as in the strawman above. It's enough to make more sense for the average company to do so. And there are tons of companies, large, SME and others, with big data to store, that have no ops or IT department as such, or just a couple of people.
They're not going to build datacenters of their own, distribute them globally, research into reliability and failover solutions, add some custom or adapt some third party stack, and so on. The salary for just an extra person could tower their AWS costs.
- most companies will do better to not try to replicate AWS (or the share of AWS functionality that they need)
- most companies aren't tech companies (not talking about the Netflixes and co of the world here) and would botch it, and at best get a costly + high maintenance bad copy
You don't need to build datacenters, or rack servers yourself, or even own servers, or even ever having seen your servers in order to have racks of dedicated bare metal colo racks of servers. Plenty of service-providers will rent you those services by the hour, and plenty of server providers will rent or lease-to-own hardware for you, so you get monthly bills.
I started working in this space in 1995, and while we've often had servers actually on prem, I've also managed racks worth of servers I've never seen in person, in data centres I've never seen, with the help of "remote hands" provided by colo provider employees I've never met.
Some w/rented servers. Some leased. Some owned. Some with staff on retainer just for us. Some just w/support contracts and remote hands. In some cases we rented office space for ops people in the colo facility, because again: They're real estate plays and the big ones have office suites available.
By data volume, I would guess that only 1% of Netflix's traffic (measured in gigabits per second) actually hits their cloud, if even that.
I can tell you just from digging into the publicly available videos let alone some of the not publicly available information that you are greatly underestimating the complexity and costs of Netflix’s architecture.
You can even spend some time watching some of the interviews about Netflix’s transition from their own data centers to AWS and their reasons.
And before you accuse me of being a fanboy or shill. I got Amazoned last year. I have no love loss for Amazon as a former employee
Why do they need all that complexity to host a bunch of (not even that many) videos and user profiling?
Netflix users reportedly stream about 200 million hours per day, and at 1 GB/hour, that's an egress of 200 Petabytes/hour, or about 5 exabytes per day. There is no reason for Netflix to pay AWS to handle that, and they don't. That would be $100 million per day from CloudFront. You can read about their data plane online:
https://openconnect.netflix.com/en/
If you count how much data goes to their cloud vs their CDN appliances, I would be surprised if the cloud gets very much of their actual scale at all.
That's if they use for infrastructure e.g. for some web service etc. And even then, it takes much less to use it, than to even begin designing such a solution.
Put another way: I was around when WorldCom went bankrupt and their datacentre in half the building we were also in was locked down by administrators. Had a server fail? Tough. You wouldn't get in. They were not Amazon of today sized, but they had $20bn/year revenue.
That's one of many failure modes that can take you out if you rely on one provider. I'm uneasy about relying on one provider for far lower durability projects - I usually ensure we at least have backups somewhere else.
But if you care enough that you pick AWS due to a need for 11 nines, and you don't include the long list of failure scenarios that include anything from a bankruptcy (yes, it's unlikely), to them kicking you off intentionally or by accident, or you failing to pay a bill, or any number of other scenarios, they might have 11 nines durability, but you don't have a guarantee you'll get access to it.
I'm not arguing against using AWS. It's expensive, but it has its uses. I use it at my current employer because our customers want it, and it's not a const-sensitive product. Picking it as part of a strategy to manage risks is entirely valid.
But it's certainly not going to save you from complexity unless you cut corners because they get you to ignore risks.
And every cloud provider has their own set of proprietary services that you are going to need to both design around and have staff trained for and constantly be testing for.
No “running K8s and open source software” is not the answer. You still need to integrate with thier IAM solution, how they manage their VMs other supporting infrastructure and often if you are large enough, you will have redundant physical connections to their data centers.
Also “using Terraform” is not the answer. A TF configuration is not portable at all across vendors.
Maybe. Worldcom was eventually bought out too. That does not mean you can assume that there will be no disruption to services, or no loss. And that addressed one of many potential risks. Unless you can guarantee that there are no such risks whatsoever, the point I made stands.
That point was simply that if you rely on a third party's durability level, then whatever that level is, the durability of the data from your point of view is lower, because there will always be a risk that your access to the data is removed either permanently or long enough for the eventual recovery to be irrelevant.
And so if you actually need and care about that durability level, you'll need multiple providers. Which you probably should at far lower reliability levels anyway.
I did not suggest it'd be simpler to use another cloud provider, so I have no idea why you're bringing that up as it's entirely irrelevant to the argument I made.
- CIO of a bank that I was working with
For storage in particular this gets dicey. Is there an off site backup? What data loss is acceptable for that recovery? What availability requirements do you have?
Tales of faulty drives destroying companies are everywhere.
If you are already managing that much data, you already have a whole department to manage that.
And in the end I haven't seen a significant decrease of people to manage stuff in the cloud. Sure you don't have to manage the hardware infra but you still manage tons of smaller virtual infrastructures on a myriad of AWS accounts. Sysadmins just get relabeled cloudops, devops and/or devsecops , end up being paid more, and thus more expensive.
That is at least my observation. I get paid more than I used to but the management hasn't been simplified. The only thing that has changed dramatically is you aren't dependent anymore on delivery time for storage, servers and network equipment.
2. a datacenter doesn't require many employees, most of them not being engineers themselves. To get back to my remark on point 1, while managing your own datacenters just to host storage would be expensive, it is usually mutualized with the rest of the infrastructure you need to access and use the data.
3. colo datacenters
This is a really important point. Virtually every customer that I work with who runs in the cloud has a platform group focused on cloud operation. They don't rack gear any more. Instead they do things like setting up IAM consistently that are beyond mortal ken. (Well mine, anyway.)
https://snowgoer.com/latest-news/yamaha-will-exit-the-snowmo...
“Tell your data to get Trucked”
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway. –Andrew Tanenbaum, 1981
I've always loved that quote.
It looks like over the past 8 or so years you're looking at a maybe a tripling in capacity. Cut the size of the truck down by a third, accounting for the fact the truck couldn't have been stuffed with every cubic inch being hard drive due to the need to walk in it, weight limits, cooling, wiring, etc., and I think it's not hard to see that even if it did make sense at the time, it's not hard to run the numbers and build some 4-6 U storage units and ship those instead. Even if you ship 4 of those it's cheaper than a truck by quite a bit.
Also, if you're willing to spend the dough, flash density can be pushed even higher, I think. You might need some custom hardware to manage power to the drives, so only the drives the customer is filling right now are powered (heat management is going to be a big deal with flash), but that's not impossibly difficult.
You can even drain an entire data center by cycling the same 3 or 4ish units around. Fill one up, send it in, Amazon loads it up, send the same unit back. Even if it takes 10 loads you don't necessarily need 10 hunks of hardware.
Personally I have the sneaking suspicion that it was always a marketing stunt and never actually made monetary sense. It was just a grand gesture to reassure people that they could indeed move to the cloud, which was a large concern for a lot of large customers. In that sense it probably paid for itself. And once you do have it in hand, maybe the marginal costs are still attractive and you go ahead and use it sometimes. But I'd personally bet the "round robin of 4-6U storage racks" would always have been more economical regardless of hard drive densities.
[https://news.ycombinator.com/item?id=1437634#:~:text=Conserv....] says 60PB, which for 20 TB drives would be approx 3000.
A petabyte is still a lot of data!
In my experience the bigger issue is that much large data sets are rarely truly static, and inventorying/tracking/validating is a massive undertaking all on it’s own.
Most large physical hard drives max out around 300-400MB/s transfer rate, so it typically takes several days to fill a single drive. And you can’t ship anything until all the media is written.
So either you have a ton of online, parallel drives (which dramatically increases capex, and decreases available storage due to physical space, power, and cooling issues) or you go really slow.
Even having 300 parallel full speed drive interfaces for instance, it would take approximately a year to fill those 3000 drives with data, and approx. another year to read all the data off. Which dramatically changes all the calculations.