We handle 80TB and 5M page views a month for under $400
blog.polyhaven.com
blog.polyhaven.com
It cost less than $2K/Month.
The cloud is crazy expensive. Private servers are beasts, and they are cheap.
Of course, for this price, you don't have redundancy and horizontal scaling.
You also don't have to maintain and debug a system with redundancy and horizontal scaling.
> It cost less than $2K/Month.
The solution in this article is serving on the order of 100TB/month for $400/month including a high speed global CDN, their API and database servers being hosted reliably, and redundancy and backup being handled by someone else.
Your solution is hosting on the order of 1000s of TBs of month (ignoring the database and other aspects of this website), but the price is an order of magnitude higher. You’ve also given up all of the automatic redundancy and hands-off management, and you don’t have the benefit of a high speed global CDN.
But more importantly, you have significantly higher engineering and on-call overhead, which you’re valuing at $0.
If anything, that only makes Polyhaven’s solution sound more impressive.
> Of course, for this price, you don't have redundancy and horizontal scaling.
Which is a huge caveat. The global CDN makes a big difference in how the site loads in global locations. Maybe not a big concern if you’re serving of static files with a lot of buffering, but they have a dynamic website and a global audience and they said fast load times are important.
> You also don't have to maintain and debug a system with redundancy and horizontal scaling.
But you have to do literally everything else manually and maintain it yourself, which is far from free.
All of these alternative proposals that value engineering time at $0/hour and assume your engineers are happy to be on-call 24/7 to maintain these servers are missing the point. You pay for turnkey solutions so you don’t have to deal with as much. Engineers don’t actually love to respond to on call events. If you can offload many of your problems to a cloud provider and free up your engineers for a nominal cost, do it.
The entire team is composed of half of one dev.
I'm 100% sure it's way cheaper than anybody that has AWS on resume.
Of course, some things have to give, like the global CND, and some data guarantees.
Everything is a compromise. It's all depend of what is important for your project.
EDIT: also, my comment was not meant to oppose the article, but rather confirm the view that you should calibrate your setup to your project. Doing so will lead to great savings in hosting, and project complexity. A lot of projects don't need the cloud.
Who is on-call 24/7/365, never takes vacation ever, and is always available to fix the website?
It’s weird how much HN hates jobs with on-call requirements, but every time cloud services come up the solutions always involve forcing someone to be permanently on call to save a few hundred dollars per month in hosting costs.
If the system has a problem during the night, the users will wait until the morning.
The world doesn't stop because a streaming service is down for a few hours. It's not a medical service, or something thousands of businesses rely on.
It just looses a bit of money and users are grumpy for a day because they had to wait until they could access new content.
It's ok.
This one dev is literally on call 365 days a year and can never be away from a computer on vacation. If he leaves, the project has no one.
How is that not a problem? Surely you can see that this isn’t reasonable for anyone who wants to run a business, or any employee who doesn’t want the website to be their life.
> If the system has a problem during the night, the users will wait until the morning.
> It just looses a bit of money and users are grumpy for a day
If you’re running a website where extended outages are no big deal and you don’t care about lost revenue, then it’s not really a valid comparison to the typical business website.
Your situation is unique, not a model by which other companies should follow.
The company I work for has a lot of stuff in the cloud. We seem to have quite a few position's worth of people permanently on call.
The percentage of "data center"-type on-call situations has perhaps gone down somewhat since moving into the cloud, but it has not gone to zero, and it was never the majority of problems anyhow.
It seems like you're sneaking the idea in that if they would just pay a lot more money, the person on call wouldn't have to be on call. I'd like to know what cloud you're using, because it doesn't seem to be any of the ones I know of. If your service is critical to your business or project, you've got someone on some sort of call (maybe not overnight call, which is the really rough bit), period, or you've got a business that can disappear at any second.
Like.. what are you expecting to happen? You'll just gaslight them into thinking they dont know how their business works?
Labour isnt expensive if you're operating towards a minimum needed to function, and your systems are sufficiently operationally stable.
Engineering labor is definitely expensive.
The entire $400/month bill for this linked website will only get you 2-3 hours of consulting time. They’re getting an enormous value by just offloading the work to someone else and not having to worry about it.
I'd argue it actually need a more expensive team.
Barring in-depth research (which I'd love to read if someone has any links), it's not clear on a 1:1 basis what's cheaper. Paying for someone's time to research hardware and talk to vendors, run POs for them, figure out where/how to install them (Equinix is expensive), and RMA hard drives as that comes up; versus not paying for that and instead paying a cloud vendor for that privilege. Throw on top a changing hiring landscape (how much 'sysadmins' cost vs 'devops') and it really depends on the size of this hypothetical fleet that we're trying to manage, and how complicated the backend of the site is. If there's no real backend to speak of, Cloudflare's CDN for static assets is going to be way cheaper, and available now, vs anything you could possibly build from scratch, that would maybe be ready in a couple months.
-But google/facebook/amazon...
-But uptime needs to be 99.999
-But everyone uses cloud
Most businesses are not a trading-market, have less then 100 peoples (aka you are probably not another amazon), and no bonus using a cloud/kubernetes etc.
But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
I worked at a company once that, from higher up, said that they had to have five nines of uptime. We had some really good cloud engineers there (one guy set up a server / internet container for the military in Afghanistan; in hindsight he said they should've just sent a container of porn dvd's), and they really went to town. For that five nines uptime, you're already pretty much required to set up your infrastructure to use multiple availability zones, everything redundant multiple times, etc.
Of course, the actual software we wrote was just a bunch of CRUD services written in nodejs (later scala because IDK), on top of a pile of shit java that abstracted away decades of legacy mainframes.
Isn't AWS down like every two months for a few hours? That's far off the 99.999% mark. No one can guarantee 100% uptime and sometimes it's even better to have that under your control (eg. have a dedicated server and a backup one from different providers).
My point is that, if you want the highest possible uptime, you shouldn't rely on a single (cloud) provider.
All means, no end.
If they didn't but cared about their business (they know I hope) and make hard requirements to IT, that would help. IT should then come up with solutions to those real-world problems. We're not talking just about hobbyists, do we?
That's what Eric Evans talks about in DDD as I got it.
Oh absolutely true, you don't want to look completely clueless in front of people who take your money to setup an infrastructure ;)
>and make hard requirements to IT, that would help
That would make stuff so much more easy. Often it even lacks even a inventory of used applications/hardware....network.
Just one example:
Made a plan for new hardware and network (new cabling etc), then i walked around the workshop and there was that dusty machine running...i asked what it is...a dos-machine with...wait...TOKENRING. That machine was an integral part of the whole workshop ;) However we made a virtual FreeDOS machine and buy'd a software converter for the machine protocol, the 25yo cnc-machine needed a new card (ethernet instead of token-ring, very lucky we found that thing) so there was that one little DOS-machine no one though about who could stop the whole "modernization".
But no matter how logically convincing your arguments were, most of the time upper manglement just went on buying Oracle, right...?
Some stuff makes sense to put on iaas, dns often does for example.
If it’s 2k/month for a non resilient solution, it’s in the order of 4k for a resilient solution (you need to relocate assets both ways but it’s in that order of magnitude)
I have some clients who use AWS and others who prefer colo and/or dedicated servers from traditional datacenters. The latter group can afford to over-provision everything by 3-4x, even across different DC's if necessary. DC's aren't yesterday's dinosaurs anymore. The large ones have a bunch of hardware on standby that you can order at 3 a.m. and start running deployment scripts in minutes.
What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months, or to make minor quick tweaks and run some updates. Let's be generous, and say you need 10 h/month, which is about 1/16 of a person-month. And if things go horribly wrong, everybody drops what they are doing to fix things, anyway, no matter if you're on AWS, dedicated/colo or run your own data center.
When you significantly change your architecture/deployment, then you need to put in more time again, but if you build your code with need to scale and such things in mind from the get-go, then that won't come up much or at all.
You don’t even need a single employee to manage a single server…
In the real world, once most hosting platforms are up and running, the maintenance overhead is pretty low.
Where? Costs vary hugely across the world
I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
$20/month for Cloudflare Pro is a steal for what you get. Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
You’re talking past the point here. It doesn’t matter how cheap if you’re fundamentally opposed to enabling cloud flare to reach its meat hooks further into the Internet.
This is no different from arguments about embedding google analytics or “just paying for windows” instead of using Linux.
You should take issue with all the other companies that have failed to deliver something as compelling.
What point? Nobody said anything about "cloud flare to reach its meat hooks" in the article or the above thread except you?
Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference? most of that group could probably do a lot of other optimizations that probably count for more.
The common mistake is to pick a server geographically close to yourself, only access it from low-latency connections, and then assume that everyone in the world is seeing the same thing.
Or to only visit your own site with everything already in the browser cache. If you're not seeing cold start loads, you're not seeing what every new visitor to your website is seeing.
Consider the Photopea.com website. The author explained in a comment below that he spends $60/month to host the site without a CDN. Several of us loaded the site and it took 2.5 - 5.0 seconds to load. He could sign up for a cheap Cloudflare account, reduce the size of his server (due to caching), and the load times for everyone would drop by a significant amount.
If you're hosting simple, static content like a blog for an audience that doesn't care about load times, then of course nothing matters. But for modern, content-rich websites (photos especially) it can actually be a substantial improvement to add a CDN even if you have a single fast server. You may not see it, but visitors from distant locations definitely will see a difference.
[1]: https://stackoverflow.com/questions/536300/what-is-the-short...
If not free then very cheap, as I understood TFA: Wasn't that why they have two separate domains and serve static assets from one of them, to be able to use the cheapest Cloudflare tier for that domain = those assets?
Once cloudflare captures the web market we'll all pay back with interest. They are not a charity.
I tend not to realise when my site goes viral, as I'm based in Australia whereas my largest audience is in b the US (and I'm a bit of a Luddite!)
hm
This occurs on many company blogs as well operating under a subdomain like blog.whatever.com
To be clear, this is a very tangential and irrelevant nitpick and I understand it does not contribute to the content of the website itself.
Or so goes the theory anyway.
Even if 5 of them would go down at the same time, the site would still work as intended (thought probably couldn't handle peak load with less than 3 or 4). If one of or two are down, nothing happens.
Also completely reinstalling a server takes around an hour.
oh the irony
> Google Firebase is nice and convenient, but it is quite expensive. We could investigate some other managed database options in future.
I've only seen people get annoyed with Firestore over time, and migrating out of it. People do end up worrying. At first, They seem to choose Firestore because it's strongly marketed and seems suitable for a new project. And then data modeling, high prices or scalability becomes a problem.
So choosing something which makes it "one-click" to set up but total madness to manage is a really short-term optimization, only worth it for a pure prototype which you will throw away no matter how successful.
If you know you need those things to reach success, then it is better to make the up-front investment to get good tools for those.
If you still want to go with a cloud provider, AWS Amplify has some interesting tooling. I've build products both against Amplify and Firestore (and Firebase). Yes, Firebase is a few days to a week faster to set up (integrated user management, as you say), but AWS gives more sophisticated control and is built around scripted deployments.
You pay for it, of course, and I'm not arguing AWS vs running your own server. I am saying if the choice is AWS or Firebase, that a few days researching the choice would give you knowledge you could use for launching the next 10 prototypes you have in mind.
I think quite a lot of other people have mentioned in the thread that they are getting a lot of other "benefits" from using multiple services, but I don't see how these help solve the problem of data delivery besides taking advantage of the Cloudflare + Backblaze alliance which is $31 if their main website is a static one.
Let's say a month has 30 days -- 5 million views a months is 166,666k per day, 6944 per hour, 116 per minute, and ~1.92 per second.
1.92 qps.
Of course it's not expensive, it's a tiny amount of traffic!
So they're saturating 25% of a gigabit uplink every second of the month.
Also note that they say "page views" which can translate in many more requests per each page opened.
A raspberry PI could handle the compute.
I don't think they are overpaying for what they are getting. $400 is not a lot for a proper site that serves a lot of bytes. I just don't think it's that impressive either -- it's just yet another website serving static assets through a CDN, with low QPS too, so I don't see why it's noteworthy.
Have you considered using Firestore in Datastore mode[0]? It might make all of your reads free[1], though migrating could be a project.
[0] https://cloud.google.com/datastore/docs/firestore-or-datasto...
I'd also encourage you load your fonts late via JS. Your main JS package competes right now with WOFF files from Google Fonts for priority and there's no need for that.
[0] https://www.webpagetest.org/result/220106_BiDc42_428a3caec56...
can't tell if joke, so:
there are already enough sites that display content for a split second and then some script runs (or fails?) and there is either nothing on screen or an error message.
this is ridiculous - please stop!
What you’re describing has been thankfully avoidable for many years.
Now, I upgraded to $60 a month. I never used any CDN.
> Now, I upgraded to $60 a month. I never used any CDN.
Why not go back to the $40/month plan and spend $20/month on Cloudflare Pro?
I was able to load photopea.com in about 5000ms, uncached. That's not terrible, but it was slow enough that I wondered for a few seconds if the site was broken. A CDN would cut that load time massively, and it wouldn't even be a net cost increase because you could downsize your server.
I mean really cold cache from the above user is at 5000ms that's pretty horrible. When I tried I got 2800ms. Take a look around it seems to be mostly from fetching static content which is a classic problem CDNs solve.
To answer the actual question - its the obvious answer. They make a product that works well and is relatively cheap. Is it without drawbacks? obviously not, nothing is. However, for a significant segment of the market the value proposition makes sense.
So what? Unless your website is offering some superficial junk that can easily be found elsewhere, you’re not going to lose a user.
For something like photopea, there aren’t sub-second alternatives out there.
> For something like photopea, there aren’t sub-second alternatives out there.
if there is't one now, there will be. Besides, with a little CDN and tweaking you improve your users enjoyment. Why not?
If I update a single file, how long it takes until nobody in the world can access the old version anymore? Does it take seconds / minutes / hours? Also, some files should not be cached at all (like PHP requests).
I am afraid it would take me days or weeks to learn everything and to cofigure the CDN properly, and I am risking being offline for a part of the world during that time. Also, if there is a problem at the CDN, my website would be broken, too. If someone could help me, you can write me at support@photopea.com
Regards oftentimes forgotten Linux user
FYI I tried to bookmark it with cmd-D, but the app has hijacked that combo!
I use it myself, but just as a normal static file CDN, but they have dedicated tabs in their UI for video stuff. Their bandwidth pricing is also very reasonable ($5/TB with their "bulk" option and $10/TB for standard)
Two thoughts:
1) CloudFlare offers incredible service. What would need the team of netadmins/sysadmins can be handled via their UI easily. I use Argo in one project and yes, it is a very efficient tool. CloudFlare is becoming like AWS, knowing its tools is a skill. Maybe one day well see an official CloudFlare Solution Architect certification?.. :))
2) For such traffic heavy websites I don’t think there’s a better combination than Static Websites hosted over CDN, which is cached aggressively and for the dynamic part serverless as a backend. If you configure it properly (cache optimization, rate limiting to not get a huge paycheck if DDoSed, etc…) this setup is like set-up-and-forget-it, without the need to invest loads of resources into it.
Not sure how most CDNs work, but for Cloudflare, content is not actively pushed to edge nodes, it’s only pulled to an edge when needed and then cached there usually for an hour or two.
The caching is still distributed, and all of the tcp+https roundtrips are done to a local data center which makes things faster.
> what the source article describes is essentially just that it is cheaper to run a static website with a CDN than without - right?
Right. The cost savings comes from caching done by the CDN, and public static content is super easy to cache.
The distributed part makes things faster, but doesn’t necessarily save money.
I believe it's also against Cloudflare's TOS to use it as an asset-hosting platform.
I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about. A guy I know was telling me how they were spending tens of thousands of dollars per month on AWS at his company.
Is it because everyone is writing their stuff on node.js, putting 500kb of JS in every web page they serve, putting Docker everywhere when a chroot would suffice, microservices, Kubernetes, or hell even writing SAPs when we don't need them?
I am genuinely confused
If you're building a basic CRUD app and you have low traffic numbers, you don't need to spend tens of thousands of dollars on AWS.
But it's not as simple as picking a dedicated server, setting it up, and hoping for the best. At minimum you need periodic backups, testing and staging environments, solutions for rate-limiting your API, and so on.
And the "everything is just a CRUD app" meme is just that: a meme. Usually engineers who repeat this have only ever worked on simple CRUD apps, so they don't understand what it's like to work on anything different.
> I'm currently clicking through various European hosting services and they seem to offer great dedicated servers at good prices. I cannot wrap my head around these ridiculous costs I keep hearing about.
If you're working on the types of problems that are a good fit for setting up a single dedicated server on a random hosting provider, you're not working on the same types of problems that necessitate $10K AWS bills.
> I am genuinely confused
I've worked on projects with $10K+ monthly AWS bills. It wasn't node.js or Docker or Kubernetes. It was the sheer volume of connections we had to maintain and data we had to process.
But even if we could reduce our $10K/month AWS bill to $1000/month with a lot of engineering effort and manual management of our own servers, what would we gain? If I had to hire a single additional devops person or engineer to help manage this custom solution, the entire savings would be wiped out. And then some!
Wasting AWS resources isn't smart, but trying to DIY your solution to everything rarely makes financial sense when you look at how much engineering effort it takes and how much engineers cost. If I can spend $10K/month to avoid hiring 1 additional engineer, it's a financial win. I do not care if someone thinks they can do the same thing in $1K/month with endless amounts of custom setup and maintenance. I don't want it.
It also reduces the number of moving pieces that we have to manage manually, which reduces the on-call burden, which keeps people happier.
I really don't understand the anti-cloud hate on HN. It doesn't mirror the real engineering world at all.
My response may come off as rude, but that's not the intention. From the above quote you've vastly over simplified all but the most simplistic environments. Someone could run all of the above for well under 10k month - those are not the reasons why costs are high
> putting 500kb of JS in every web page they serve
that's not much and CDNs serve that without issue or significant cost
> putting Docker everywhere when a chroot would suffice
Docker does not have much overhead over chroot with a large drop in security/isolation
> microservices
there are good and bad ways to do this - does not need to cost much. saying "microservices" is so broad to have no meaning in this context
> Kubernetes
think about why someone might run kube. Again, not that expensive unless you're really small where having master nodes would have a big impact on costs
> or hell even writing SAPs when we don't need them?
Do you mean SAP? SAP is the largest non-American software company by revenue. I don't think anyone likes working with SAP. You think people use it for without reason? There are reasons. Think about what those might be
But those are just details - when building anything of a decent size, nothing is a simple CRUD. There are always exceptions, limitation, biz logic, migration issues, schema issues. I don't care if you use Postgres, Mongo, Kafka or $SOMETHING_COOL
Then there's data retention. It's very very easy to keep PB of data in s3 or other data stores for biz or compliance needs
Then AWS gives IAM, which is very helpful in teams > 5
Docker adds plenty of attack surface that could easily be avoided by using any sandbox or a systemd unit file.
Complexity inflates HN readers CVs and keeps them employed.
Because most web apps aren't simply CRUD apps. It's a meme said by armchair engineers who believe they could build twitter in weekend.
Only people here on hn seem to think that everyone is slinging k8s and 1000 layers of microservices to build the most complex things on earth; the rest is earning their paycheck by building some forms in php.
Sadly many here are overengineering crud apps and then somehow making them out to be something more than crud apps because somewhere in the future they might be (probably not); as in, you can build 'a twitter' in 1 weekend and it will even probably handle good volume of users ( more than Twitter did at startup; it was often down or very slow) etc if you just take postgres with php on a few $/mo server. It is a crud app at that time (the flow for these user authored posts is built into every trivial and complex framework these days). Also it will most likely be bankrupt in a few months as that is what happens to startups; I rather spend money on building the business first and then scaling the tech; which is exactly what Twitter did. On hn it seems a lot of people work backward in that regard but that probably has also something to do with being a techie and VC interest in scalable tech.
Also, you know what you're getting security-wise with AWS, and no one will blame you when your website / service goes down because AWS is down.
I've had this argument too many times to count. Every page view does not need to be dynamic per user request. Creating a sane cache policy reduces origin resources, servers, cost, etc... and surprise gives a better user experience at the same time.
A few years ago when I hosted what felt to me like a popular site, I was serving serving > 150K page views a month from a 1.5Mbps adsl uplink. With that said, back then you could gzip everything and there was no letsencrypt or CloudFlare walls across the board (didn't exist yet).
Good thing bandwidth is easier to come by these days.
> Running a massively popular website and asset resource while being funded primarily by donations has always been a core challenge of Poly Haven.
My brain went: poly.. fill.io or something
Thank you, sorry, am embarrassed.
No, that's low by an order of magnitude. It is 16MB (16,000KB) per page view.
HDRIs, high res textures & 3D models
'Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."'
They are pushing about 245 Mbps out of Cloudflare (averaged over the month). Wholesale IP transit prices are anywhere from 10 cents to a dollar per Mbps depending on volume. Cloudflare dumps 40% of their traffic over peering, putting their price at $14.70/mo. Ignoring fixed capex of servers, Cloudflare is making about $25/mo on this customer.
Given the $11 Backblaze bill, I estimate about 2 TB of data.
A capable dedicated server with 2 TB of disk and 1 Gbps unlimited bandwidth will run about $30 in a major European metro, maybe double that for the US.
With a grand total of $370 vs. $60 worst case, they are spending 516% more to be "serverless."
Edit: Yes Cloudflare has more than one server. Double the price and put one in the US and you still come out ahead.
Edit 2: I'm not saying one way or the other is better. Just that the title is very clickbaity for a "put a credit card into a website" payoff.
Not even close to comparable. I think you're ignoring all of the functions they're paying for. They're not just hosting a few files. Also, your "worst case" is literally the most optimistic best case for a standalone single server, which completely disregards any redundancy and assumes that the $30/month server is truly unlimited/unmetered in every way. That's not a good assumption.
Cloudflare is a global CDN that will be fast for everyone regardless of location and will soak up bursts of traffic with ease. It's also fast in a location-independent way, which won't be true for a single server in a single datacenter somewhere.
Finally, the amount of effort it takes to do this with Cloudflare is trivial. The amount of effort it would take to maintain and optimize a standalone solution is not negligible.
Cloudflare seems like a very good deal to me, unless you value your time at $0 and you have access to these truly unlimited, high-performance $30/month servers.
Back in the day we used Texan Colo data centers with DrFTPD to do just this at massive scale.
Bytes in, bytes out. Not a popular opinion here, but I firmly believe it's not necessary to play the CentralizedFlare game to get a winning outcome.
Cloudflare Pro is $20/month per domain.
Trying to run everything through a single server in a single datacenter just doesn't compare for globally-accessed websites like this, even with a 1Gbps unmetered link.
I've accessed plenty of sites physically located in the USA from various EU countries, not to mention AUS and I didn't notice much difference as long as they had a good interconnect.
Maybe not the best for YouTube, but there's only a few of those.
Don't get me wrong, I see the appeal and fell for CF early on but not a fan anymore since they've grown into such a behemoth.
Absolutely power corrupts absolutely.
If it’s not your core competency and unless you possess the vast breadth and depth of skills to dig yourself out if something goes wrong, it seems like a distraction at best and a fatal mistake at worst, with the end result of saving 1-2 engineer hours worth of money per month.
For some self-hosting is a no-brainer, but I suspect that community is much smaller than many here would expect.
Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Set things up right to begin with, and in my experience you can leave them running for a long time without intervention.
Downvote all you like, but it's the truth.
Is the job getting done for the donors? I bet they don’t care about $300 if it means the website is highly available and the maintainer isn’t burned out from giving away $200/hr of opportunity cost all the time.
> Fact: Things break most often because of humans doing things - changes, like deployments and config modifications.
Totally agree, and sometimes success in your project forces your hand as you are pressed to add functionality or need to scale. If you can guarantee that you’re doing this work once, ignoring hardware failure or scaling, the scales may very well tip toward self-managed.
For a vast majority of projects, I think it just plainly makes sense to go this route, hence the success of these centralized hosts. It is the pragmatic option.
PS. For what it’s worth, I’m sorry you’re being downvoted simply for having an viewpoint that others disagree with.
RedHat usually break setups every 3-4 years which I personally consider way to frequent.
If I configure a server with automatic updates then I would like it to run with very little maintenance for at least a decade and a long time would be two plus decades.
10 cents is far from the lower end for high quality IP transit.
Cloudflare doesn't pay anywhere near $14.70/mo for 80 TB/mo. Even I spend much less than that on 80 TB in a month. On top of that, peering makes it even cheaper for Cloudflare, as you said. Cloudflare is very likely using peering for much more than 40 % of the traffic now, so it's even cheaper for them.
The worst peering region for Cloudflare was North America in 2016 with 40 % peering according to Cloudflare blog posts.
This would be approximately equivalent in spec to what you could build for $2500 purchase cost if buying a 1U machine and colocating itself.
For 30 bucks a month you'll get something very old and weak.
Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
The bulk of the data transferring is static files because they're hosting large assets, but the rest of the article is about their API, database, etc.
A single server in a single datacenter isn't comparable to a global CDN. They made it clear in the article that they value global latency, and they're willing to pay more for it.
> Here is a 3.3 GHz Xeon, 24 GB of RAM, with the required disk and bandwidth for $33: https://oneprovider.com/order/item/dediconf/59
That has mechanical spinning hard disks and a CPU that was released literally a decade ago.
You're not going to replace Cloudflare, Firebase, their API server, and a global CDN with a single 10-year old machine serving files from a mechanical spinning hard disk, hosted by a company that almost nobody has heard of.
You could probably do it with Cloudflare for a similar price, but that's not what this article is about. The article says that their Cloudflare (not Argo) bill is only $40/month, and that's only because they have two domains. It's $20/month per domain.
The $400/month figure isn't just for static file hosting. It includes their API, database, and global CDN. They make a point to say that low-latency, global CDN is a priority for them, which is why they're paying extra for Cloudflare Argo.
I'm not sure why everyone is comparing what they're doing to a single fileserver hosted somewhere, serving up static files without regard to global latency. It's apples and oranges.
But fine, if you don't believe it I will happily extend an invitation to come see in person my racks of spinning rust and 10 year old servers in Los Angeles or Fremont that are running services that are used by millions of end users. My email is in my profile.
Which they ditched as soon as physically possible. The internet has changed a lot since then. That much should be obvious.
You can't reduce this article to "millions of end users" and then equate it with every other service. There's more to this article/service than you're suggesting, and there's more to a website than just users/month.
If you're really strict on this it's trivial to preload these files in RAM after compiling your assets so you don't have to wait for a request.
If you have 2.75TB of files to serve and have a bunch of simultaneous requests for different files:
- everything doesn't fit into the RAM page cache, unless you have 3TB of RAM in your server
- for files that aren't cached, you have to read from the drive
- when you have a bunch of different requests for different files, you're going to have a lot of disk seeks. Seeks are expensive on rotating media.
Serving a large collection of static files at scale can be quite challenging. At my previous e-commerce company, back in the early 2000's, we did our own image serving with RAID1 multiply-mirrored drives (before SSDs!) so we could multiply the seek capacity by the number of drives.
I’m a happy customer of theirs. The UI for managing things is a disaster, but once you get ssh access, it doesn’t really matter.
This reads like a paid marketing post for Cloudflare.
It's astonishing how many times the author conflates browser cache-able assets with cached assets on a CDN. When a browser downloads a static asset if the web server is configured properly those files will be cached and there won't be a need to re-download them for a very long time.
There is of course the issue with modern web frameworks like React generating a single massive js/css file that bundles everything all over again in a unique file busting all previous cached versions across all users' browsers just because a comma was added to a sentence.
Keep your js/css files small, serve them from your own web server and set a reasonable expire header, no need to pay Cloudflare $40/mo to continue gatekeeping the internet.
If anything, I’m fine with a new competitor to BigCloud that’s been unchecked and increasingly hostile (cost-wise).
Asset storage: Backblaze B2 – $11 (replace with Cloudflare R2)
Web hosting: Vercel – $20 (replace with Cloudflare Pages, $0 cost)
Database: Firestore – $100 (replace with Cloudflare)
API: Vultr – $5
Image hosting & optimization: Bunny.net – $27 (replace with Cloudflare)
Domains: Cloudflare – $4
Email fees: MXroute – $3
R2 is 0.015$ per GB while backblaze B2 is just $0.005.
Does Cloudflare have a database offering? Do you just mean the Worker KVs, or is there a full relational database?
If you were OK with your content only updating periodically you might be able to do away with Firestore, Argo, and Vultr.
You definitely do a great job of mitigating db hits. Consider publishing all the data to a CDN using something like Gatsby.
In this scenario your CI/CD would build a static website from all the assets and publish it to a static server.
You mentioned that only your view counts are the only really dynamic part. You could just estimate and emulate those or design them a different way.
Martin Fowler has an article on this: https://martinfowler.com/bliki/EditingPublishingSeparation.h...
Then at the end he says he uses Bunny as the CDN for images. Why use Bunny instead of Cloudflare? One more moving part.
I presume it is cost related?
I speculate it's because Cloudflare's TOS prohibits you from using their CDN to serve "video or a disproportionate percentage of pictures, audio files, or other non-HTML content" on typical plans. See section 2.8 on https://www.cloudflare.com/terms/ . Since PolyHaven seems to be a purveyor purveyor of 3D assets and they use Bunny to host "all of our images shown on the website (thumbnails, renders, previews, etc.)", I'm guessing a lot of their assets exceed Cloudflare's TOS.
PolyHaven mentions using Bunny's image optimization service, so that'd factor into the decision too.
- Storj has the lowest cost per TB, but charges for egress - Wasabi doesn't charge for egress, but has a fair use policy where if you egress more than you store they can kick you off their platform[2] - Wasabi is also a better fit only if you plan to keep your files around for 90+ days (they have a minimum object retention period) - The bandwidth alliance is only available for HTML related content unless you're on a specific paid plan[3]
[0] https://www.storj.io/ [1] https://wasabi.com/ [2] https://wasabi.com/paygo-pricing-faq/#free-egress-policy [3] https://www.cloudflare.com/terms/#28-limitation-on-serving-n...
*opinion is mine, not my employer's
You can sign in with
Username: hn@hn.com
Password: hackernews
-
The site is currently in demo mode, and the db will be wiped before launch - feel free to sign in and poke around. Also it's hosted in Australia currently, so site may be a little slow for those in the US.
Source is here: https://github.com/jjcm/soci
I want to make a live video streaming website a la Twitch.tv. How much would CloudFlare charge me to stream 8 Mbps to ~80,000 viewers for 4 hours?
19,200,000 minutes at $1 per 1,000 minutes. https://support.cloudflare.com/hc/en-us/articles/36001645087...
Though, mux.com is decent too, from what I've heard; for certain workloads, LiveStream and Vimeo might be cheaper.
There's Peer5, PeerTube, and BitTorrent in the P2P space (among many such solutions).
First time hearing of them (due to my own ignorance probably). Any details on them?
I thought about doing this a while ago, after I left my job with an ISP. I would just have bought transit from them (and maybe made them regret their 10Gbps for $1000/month plan ;)
I am asking this because YouTube seems to have a massive monopoly both on technology as well as content/audience/network-effects. Even if you can get all the people of Youtube to shift over, you still need to solve the problem of bandwidth and the costs associated with that.
That's massively overpriced, with current prices you should fit into a 5 dollar budget.
They have sweet deals with the cloud providers?
Also see their post about it https://blog.cloudflare.com/aws-egregious-egress/
The more relevant number for this comparison would be our gross margin — much we have to spend on things like bandwidth and servers to service our customers divided by the revenue those customers generate — which in Q3 2021 (our last reported quarter) was 78%. Which is… pretty good for a services business like ours.
I don't know the specifics of this customer, but I don't see anything that leads me to believe our margins would be out of the ordinary with them. There are a lot of scale economics in our business. In other words, we can definitely do things for less money because we service a lot of customers than any one customer could hope to do it on their own.
My two cents from a regular guy to a billionaire, :).
Noting they are operating at a slight loss, during a period where they want to drive growth, isn't "naysaying". The question is whether the current pricing is tied at all to the "operating at a slight loss". He replied to say it mostly wasn't.
Remember, for example, when Uber rides were really cheap?
Foul play?