Region-specific Machines pricing
community.fly.io
community.fly.io
Fly’s pitch is not just on demand scalable compute, but also that compute being close to the edge, and with “ops-less” deployments. All of those are factors you pay for.
You could pay a little less with a big cloud provider’s serverless platform, a bit less again with regular cloud VMs, a lot less with colo hardware, and a ton less running your own DC (at scale). Obviously not all of these work for all businesses, but saying Hetzner is cheaper than Fly misses the point of those two services on the spectrum of service levels.
This has me so confused. Should the cloud abstractions not clearly make the service more optimizable at scale and thus cheaper to offer? Sandboxing things, limiting options, not having to deal with users wanting to provision VMs and do all kinds of unexpected stuff? The marginal cost of the beautiful software to make all of that happen smoothly, and all the dx it affords, that will just go towards 0, no?
I understand that people ask for what they can get away with. But does the market fail or am I missing a piece of the cost/price puzzle?
Only very specific serverless products where the provider can bin-pack you with other tenants allow them to possibly offer lower prices.
At one side, the big cloud providers have their very fair profit margins. But I've always assumed that comes mostly from that they can, and do, charge extra for the beautiful software (not seldomly through ridiculous egress bandwidth pricing).
I'm skeptical of how much flexibility and optimisation the big cloud providers really get compared to a very large VPS provider?
All the compute products (different form of container hosting, serverless) are essentially still priced at CPU+RAM + a premium. Some additional abstraction for nouveau databases ("capacity" or "billing" units). Many of the highly abstracted offerings also have the scale-to-zero concept, while the provider still have to have physical capacity ready (to some extent).
Sure, you can probably crank out some percentage more utilisation, but that doesn't really matter for the 100+% price-add on the market bears.
But I of course have no data to back up my claims, just speculation.
- Things change and break. You are paying for people to fix that for you at 2 AM, and to keep up with industry developments and best practices and make those decisions for you.
- Legal risk and compliance requirements change. You are paying for them to keep up with those for you.
- Overhead imposed by multi-tenancy. Security boundaries, control planes, etc. These are fixed operational costs caused by the abstraction.
You're not paying for just an abstraction, you're paying for the things the abstraction enables. It's also not always what you need. But it often makes good business sense.
And then you’re at the mercy of whatever support agent decides to look at your ticket and that can take days.
Use literally anyone but Hetzner is my advice.
Also support is competent and quick. We usually get an answer in few hours most of the time.
Took my data and never returned.
https://www.trustpilot.com/review/hetzner.com <-- There are a large number of both 5-star and 1-star reviews with almost nothing in the middle.
Please give us an update here when you are done building your hetzner beowulf cluster and let us know how it fares. I think you are grossly underestimating the system you describe.
E.g. a load balancer in front of some relatively stateless http microservices is really not going to tie you to Fly.io because of complexity. A globally distributed edge compute environment with database, RPC, varying scaling patterns, and environment certifications is probably not going to have the value prop impacted by this kind of minor pricing change though.
None of this is impossible, but when you are a small startup it makes sense to offload this to someone else and not waste time reinventing the wheel.
You won't be a small team by the time you're at a scale to need horizontal scaling. Only the biggest consumer facing/social media sites get value from that.
In your specific situation: it really just is the case that Brazil soaks us on import fees. Getting machines in racks in Brazil is just crazy expensive. We're going to keep doing it! Brazil has too many good sandwiches. But the idea behind building a new public cloud on our own hardware is for us to be around for the long haul, and flat low global pricing is not a "long haul" decision.
(It wasn't a long haul decision last year, either, but building a billing system capable of expressing this stuff is a nightmare from which we have not yet fully awoken. It turned out to be a shockingly hard problem.)
There are going to be cases where places like Hetzner make a lot sense compared to us. We hope you get your thing launched and find success wherever you end up. Some of our stuff, like LiteFS, you can take with you! :)
I was expecting a larger grace period from announcement to price change though. You got it right when you started charging for stopped volumes - you overcommunicated and gave users estimates for a few months before you started charging. Smoothing the increase over the next 4 months was nice.
A couple of nits as well: - AFAIK prices increased in all but two regions. The announcement could be clearer or more direct regarding this. - The pricing page could have a clearer from/to. I have to click on the example toggle to know how much machines cost before the change.
There is an interesting dynamic of server prices and customer spend in different parts of the world.
Usually where the server prices are high, customers spend less.
At some point everything will saturate and probably energy production costs will dominate (largely for cooling), but I think we're still a bit far from that.
Looking at AWS site it explains that PIS/COFINS and ISS are charged, excerpt below:
"AWS SBL can issue an NFS-e to a Bra zilian customer only after receiving a valid Tax Registration Number ("TRN") (i.e., CPF/CNPJ). The relevant taxes applicable to AWS cloud services offered locally are the following: PIS and COFINS (9.25%, combined rate) and ISS (2.9%, charged by São Paulo municipality)."
https://aws.amazon.com/tax-help/Brazil/
This is below 15% which many countries do tax for cloud services.
I'm not saying that Brazilian taxes are really high, they are for hardware imports, but the taxes into the service itself does seem "fair".
It's ridiculously expensive.
Very little hardware is produced domestically, and the little that is, you don't want to be running your switches on what is in essence a no-name "intelbras" brand.
1. AINAL, so I might say legal matters wrong for not understanding completely how the tax law works in Brazil, (Tho I do like to search about it), but I might find someone who could help you guys with it.
2. Currently, there is a law already approved (now being regulated with complementary acts) about how taxes will change in Brazil to the VAT model, this could increase/decrease taxes for a service like yours.
Besides these 2 points, we could talk via email (xunjin.coder@gmail.com) to might find a way like AWS did, creating a Brazilian subsidiary, to be able to generate NF-e in accordance for each valid Brazilian Tax Registration Number (CPF/CNPJ) and issue the invoices in BRL.
This page https://aws.amazon.com/legal/aws-sbl/ could have more info to elucidate how AWS does.
PS: Life is sometimes a funny coincidence. My brother, who is a State attorney where I live, called me to share private matters 20 minutes ago, and already asked him for some help/directions about it. This weekend, I will speak to him in our spare time about it.
It's too bad the closest region to Dubai is Romania, and Africa/SE Asia are a bit uncovered, but that's ok for now. I'm really enjoying using LiteFS and globally distributed instances for each customer, making it really easy, fast, and scalable. As the sole dev on the team right now, this is fantastic.
I wish Upstash Kafka was a bit more baked, but I ended up just signing up with them directly. I had trouble using the built-in-to-Fly version, but that's fine.
Can't wait to give them $1000s a month.
The reason to like fly is you get a 20-ish line yaml file that replaces (and I'm not exaggerating) thousands of lines of cloudformation. It automagically sets up the equivalent of a load balancer, autoscaling group, VPCs, db, vpn, etc. And it's trivial to replicate that environment per employee.
Plus the autoscaling using their firecracker stuff is fast.
I'd recommend most companies who aren't using tons of aws services take a hard look at fly.
It would be nice to have stable pricing and not have a fear of different features costing more out of the blue.
Other than this, fly has been a great service, I love the system, the CLI and litefs.
It scares me that internal bandwidth charges will start being added, the application I’m using has a lot of internal transfer which is currently free.
My apps are all on Fly but in the regions without big hikes thankfully, so I’m sticking around for now, but it sucks some people are being hit by this…
However with fly it’s just one command to add a new region and my data is auto replicated there.
That’s super nice compared to manually managing this.
As long as bandwidth costs do not go up drastically I think it’ll be okay.
"Any traffic exiting a Fly.io Machine is considered outbound data transfer, including:
- traffic destined for the internet - traffic destined for other Machines or apps in your network"
Edit: maybe you are right actually. I have to check on this closer. App is just about to launch so I hadn’t paid so much attention
FWIW, I haven't had too many reliability concerns but I have had some issues with builder VMs (can always build locally so no big deal) or non-prod environments with just one instance.
I hope they don't cancel this plan, currently on it :D
EDIT: Ohh, it's only visible when I'm logged in, weird...
EDIT2: ...indeed
> The paid Hobby plan and the Legacy Hobby plan are not available for new sign-ups.
https://fly.io/docs/about/pricing/
damn :(
> If you were on the free Hobby plan at the time that the paid Hobby plan became the default for new organizations, your plan is now called the Legacy Hobby plan. Your costs stay the same as they were, with no monthly subscription fee, and no included usage beyond the free resource allowances.
so i get to use the legacy hobby plan because i'm grandfathered in, nice. only use it for a hobby project with one instance anyways(see bio), if they really decide to start charging me i'll gladly leave.
> The first improvement we're excited to announce is that the $5 Hobby plan is no more. We're replacing it with a very simple Pay As You Go plan. On this plan there's no more upfront $5 charge and no minimum monthly commitment. You only pay for what you use. If you don't use anything for a given month you pay $0. You still need a credit card on file to prevent abuse. But your card is only charged if you use the service.
[0]: https://community.fly.io/t/fresh-produce-pay-as-you-go-plan/...
they need a table that's a human readable name and an average/range for the cost modifier as compared to the baseline
I wrote the community forum announcement and was one of the people that worked on implementing our PAYG plan. I just wanted to clarify that we didn’t deprecate silently, so I quickly created an account to do so.
Apologies for any confusion this caused!
Render has been quite solid and the support has been on point when we have found issues or run into unexpected edge cases. I have been impressed that despite a big raise and subsequent scaling up, they continue to ship a solid product and the platform improvements have been useful as well.
I really like their Docker support and infra as code, makes it very easy to spin up a whole thing while not being too far conceptually from Kubernetes for example.
Both have some reliability issues and rough edges. My new stuff is mostly on Fly, but I'd use Render where I had a junior team who had zero infra chops.
Fly.io and similar give you a higher abstraction layer where you throw a Docker container and the stuff below and around it is managed for you. Basic suff like OS patches, but also autoscaling, load balancing, request routing, etc.
Does the 4x difference make it up? It will depend entirely on you and how good you are with the things fly.io abstract for you, and how much you value your time spent on it.
I forget when exactly they raised their prices, but it was several years ago.
I’ve heard the same thing about their core product.
Oh wait, they even acknowledged it: https://community.fly.io/t/reliability-its-not-great/11253
(We fumbled the ball hard at DigitalOcean ngl, fingers crossed Ben can build what I dreamed of building one day!)
I host my little mobile braai simulator on it (braaisim.com) because it's an art project and easy to add new domains / certificates. Convenient to be able to scale up easily if it went viral in ZA. Nice that they support Johannesburg region (jnb), though, which is close to me.
There’s also a hard container size limit I’ve run into multiple times. If you add a dependency and go over the size limit, your app won’t deploy unless you switch to a GPU instance, which is substantially more expensive.
What are you trying to do and what's the size of your container? My instinctive response here is that you're not holding it the way we expect you to: that some of what you're sticking in your container, we'd expect you to keep in a volume.
That’s about every early stage company, though.
my production app has had 1 outage where stripe webhooks weren’t delivering for 45 minutes which is a huge issue for sure.
I mitigated by checking in addition to the webhook event and also it hasn’t had a problem since. Their explanation was a 3rd party networking issue at or near one of their data centers
I love the idea and will still use it for many less-business-critical apps but after multiple late-night weird un-debuggable database outages we had to switch.
Basically I’m just using the API to spin up machines, which do some work and shut down. There’s some extra machines per build job, like database containers or a headless browser for testing. Pretty smooth in my experience.
I think the only occasional issue I hit is the internal DNS being a few seconds behind reality.
Detailed description of every incident (a superset of "outages"), including stuff not visible enough to make the status page, going back to April.
I found it impossible to distinguish between user error and platform outage. Too often it was a problem on fly's end yet the status page gave nothing (perplexed, I'd rerun the deploy a few hours later and it would work).
Can't stress it enough: if fly's services aren't working for any reason, big or small, put it on the status page. Devs need to know when something's not in their control so they can inform their team and customers and go to bed or take a coffee break. They shouldn't be up till 5am thinking they borked something when the problem is 100% on fly's end.
Every week, we go through the incident management system, and all our incidents, whether or not they had user impact, get written up here.
One thing you're probably running in to, which is not on you but rather on us, that we are aware of, and that we are grinding steadily away at: as the platform has stabilized over the last year, an increasing share of problems users run into aren't "platform" issues (in the sense of: things going wrong in our API or on our cloud servers) but rather bugs in `flyctl`, our CLI.
We have two big reliability projects happening:
* We've finally landed our "white whale" of being able to move workloads that have attached volumes between different physicals. I'm writing a big long blog post about how that works now. We've been doing it for about 9 months, but we're at the point where we can do it automatically and at scale. Really hard to express how much not having this capability increased our degree of difficulty.
* We're taming `flyctl`; in the immediacy, just doing a better job of reporting errors (it's a Go program, we love Go, but showing "context deadline exceeded" to a user is, obviously, a bug, albeit one we didn't recognize as such until recently).
I don't have a way of addressing these kinds of concerns, which are valid, without just being clear about what we're up to and how we're thinking. I wrote a sort of "once-and-for-all" thing about this mentality here:
Apps get scaled down to 0 despite setting minimum running applications.
Our logshipper just stopped working for a week.
Our apps occasionally just stop working.
The worst part is that I could cope with the above if they communicated the issues.
They don't. They make major changes - "improvements", break stuff regularly, acknowledge that they broke stuff in the forums, but don't communicate to those affected.
I am seriously considering going elsewhere - and I really don;t want to. The intention is good, but I am not convinced that they are stable enough for people to gamble their businesses on it just yet...