Hacker News Reader

Comment by exabrial | Hacker News Reader

I wish the tape hardware itself wasn't so pricey! would love to have local backups our our colo servers.. alas we pipe everything to "cloud" storage (remembering that the cloud is just someone else's server in Virgina)

View on HN

I am so tired of the "someone else's server" trope as if there is no value add.

S3 is a fantastic piece of architecture and service and the fact that the common dev has such tools and service available instantly from their personal laptop or even phone is mind boggling.

We are very fortunate that AWS led more services being made available to common folk than what had been done in the past.

Its not all about cost or hardware ownership--the value these near infinitely scalable services is easy to ignore but astounding at the grass roots level.

As someone who constantly scrapes and indexes every bit in every public S3 bucket I can find, for purposes of selling query access to my indexes, I couldn't agree more - FANTASTIC value add.

Of course, I never have, don't now, and never would run a business on someone else's computer though... all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s

Do you feel like that practice is unethical at all?

Probing for open buckets and selling the results for people likely looking to do nasty things with the is unethical IMO, but it's still more ethical to do it and then bring it up to warn others of it, than to just do it. I mean, how do you know if the people talking about this fantastic value add at the fingertips of anyone are not doing it?

When it comes to doing basic napkin math and publish the results, that I find highly ethical, actually.

> it's still more ethical to do it and then bring it up to warn others of it, than to just do it.

ala, i was gonna murder you, but at least you were warned!

It took me 3 secs to find this: https://buckets.grayhatwarfare.com/

So I think the practical difference is mostly between praising S3 (as synonym for "the cloud" when Amazon wasn't even mentioned, no less) and warning at least to use it diligently, which when it comes to costs might often be not using it at all.

The context of the conversation is on tape drives which is about mass storage.

The comment I replied to said:

> someone else's server in Virgina

While everyone may operate there now it was exceedingly common in mu experience to be referring to AWS when referencing Dulles / Ashburn / NoVA or just Virginia since AWS started there first.

So I figure S3 is the correct comparison to not having a 50 TB tape drive under your desk when "the cloud" is being discussed.

I feel like companies collecting sensitive information and slapping it on someone else's computer without the slightest clue how to protect it, or running a cloud hyperscaler and leaving storage open and indexed by default is far more unethical than anything I'm doing. I'm not the unethical collector, I'm not the one who decided to host that collected info on the cloud, and I'm not the CSP who gave those unethical collectors shitty security defaults. I'm just the messenger notifying others that this is happening.

US case law is abundantly clear that web scraping is 100% legal. Further, I have never signed any agreement with AWS (I don't even have an AWS account) regarding AUP or anything like that, so I have no contractual restrictions on this either.

> all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s

The benefit is that you don't have to buy the entire server in order to use a portion of the resources on it. If you're going to use the entire server, buy the entire server — if you're only going to use a portion of a server, why buy an entire server?

> if you're only going to use a portion of a server

only if your data usage decreases over time.

There's a break even point, and that break even point is closer than you think. But by then, you're locked in. It is also the same reason why they are able to charge you more money than a self-setup storage system, since once you're locked in, the cost of switching is almost always just slightly higher than staying.

> if you're only going to use a portion of a server, why buy an entire server?

Unless your compute needs fall below those of an Rpi Zero, it is almost always cheaper to buy physical hardware once that doesn't have a monthly bill attached.

Sure, scalability is a real value proposition from the hyperscalers that isn't the same with physical hardware, but is scalability worth paying 10x+ in OpEx for what could've been 1x in CapEx? For most businesses, I have a hard time making financial sense of that one.

Do you sign long term lease agreements for colo space, power and bandwidth for the hardware or are you still paying that annually/monthly?

There are no free lunches. I do not disagree buying hardware is cheaper but operational overhead and opportunity cost are real.

Eh, buying hardware isn't necessarily cheaper if you factor in operational costs like power, cooling, and especially internet (which gets very expensive very quickly for servers). Datacenters get to spread this cost out over many servers, small installations don't.

As a business owner, I don't need a whole data center worth of electricity and internet.

No, but as an example, if you want more than 30Mbps upload, it can cost you hundreds of dollars per month depending on location. (Depending on location.)

have you considered that maybe the hyperscalers and other massive cloud storage solutions (apple, comcast, AT&T) leverage tapes in this manner? I suspect if you're using something like AWS Glacier, you're using "someone else's robotic tape library"

Reply on news.ycombinator.com