Design Thinking: B2 APIs and the Hidden Costs of S3 Compatibility
backblaze.com
backblaze.com
E.g:
=> POST https://upload.backblaze.com/bucket/file
<= 307 redirect to https://pod-000-1007-13.backblaze.com/b2api/v1/b2_upload_file/...
=> POST https://pod-000-1007-13.backblaze.com/b2api/v1/b2_upload_file/...This is standard HTTP/1.1, and it's used by the S3 API for PUT requests.
A redirect on a POST never results into another POST with the same content.
- API typically don't follow redirects (without special flags). Too much risk to cause damages by doing repeated calls.
- Browsers follow the redirect by a GET with no data. It's typical for a completed form to redirect to another page, do not re submit the form there.
There are some settings and flags to alter the behavior into what you describe but it's not a good idea to go there. It will definitely not work out of the box with much of anything.
https://tools.ietf.org/html/rfc7231#section-6.4.4
https://tools.ietf.org/html/rfc7231#section-6.4.7
303 means "GET this new URL" while 307 means "resend your request at this URL without changing the verb or body".
You will have to work individually on every client you plan to support, both browsers and libraries. It can be done but it's not necessarily a good idea.
To illustrate this, here's a recent S3 bill:
$0.005 per 1,000 PUT, COPY, POST, or LIST requests 289,727,754 Requests $1,448.64
$0.004 per 10,000 GET and all other requests 62,305 Requests $0.02
$0.023 per GB - first 50 TB / month of storage used 18,990.009 GB-Mo $436.77
As you can see, most of our spend on S3 is from the PUT requests, not the storage, or download. Probably there are some things we could do to reduce the number of PUT requests. We don't really care that much, because the total cost is not that large, but there is at least some incentive to reduce the number of PUT calls.
But if it was free? I would never change this system. Does Backblaze really want this sort of traffic profile?
> Does Backblaze really want this sort of traffic profile?
Oh heck yes, we would very much like your business! :-)
Backblaze only charges you $0.005/GByte/Month so your $436.77 bill would go down to $94.95 and we would be happy to have it. That is profitable for us. (Since Backblaze doesn't have any deep pockets or VC funding, we have to stay profitable.)
Although I agree that storage density growth for HDDs has greatly outstripped any I/O growth, I don't agree that the latter has been flat (i.e. no growth).
Are you sure you're not doing something like comparing 7200rpm drives from 10 years ago to 5900rpm (or slower) or variable-speed "green" or even SMR drives from today?
That said, I think what many people forget is that for "cloud" storage, the I/O bottleneck is almost certainly going to be the network and not the backend storage, especially for mainly sequential access.
If each of their "pods" holds 60 drives and each "vault" holds 20 pods (17 of which are non-parity data), that's over a thousand drives per vault. If each drive is 7200rpm non-SMR, it can saturate a 1Gb ethernet with sequential I/O, and random I/O would divide that by 10 or so. That's the equivalent of 100Gb/s, per vault.
That kind of bandwidth is possible and even affordable to provision inside the datacenter without metering and charging for it and is likely to dwarf the size of the connection to the Internet.
This is because they were pushing an average of 112 requests per second, the entire month.
I understand, I was comparing the storage cost.
Backblaze publishes a cost calculator here: https://www.backblaze.com/b2/cloud-storage-pricing.html (scroll down, it is half way down that page). But it might not contain transactions.
Backblaze does charge for transactions to recoup costs from that sort of thing. But it isn't intended as a "profit mechanism", we believe we approximately break even on the bill for transactions.
In this case their bill would have had $115.89 for the 290 million requests.
Glad to see it's not a free lunch, I'm sorry for assuming otherwise.
And thanks for b2, it's a great service :)
We do make heavy use of the S3 versioning API to let us roll back files, lifecycle rules for past revisions and have never been able to find any service which offers the same amount of functionality with reasonable pricing.
One thing we have done which has saved us quite a lot is adding a buffer between our service and S3 which delays file writing by up to an hour in case an updated version of a file is uploaded. This lets us limit our PUT requests but still gives us hourly revisions of files if we need to roll back. Without this system we'd be looking at over a billion writes a month. Certainly not a strategy suitable for everyone though.
> We do make heavy use of the S3 versioning API to let us roll back files, lifecycle rules for past revisions and have never been able to find any service which offers the same amount of functionality with reasonable pricing.
Backblaze B2 has both lifecycle rules ( https://www.backblaze.com/b2/docs/lifecycle_rules.html ) and file versioning ( https://www.backblaze.com/b2/docs/file_versions.html ). I would LOVE for you to evaluate it and tell us what we are missing to win your business!
The Backblaze B2 system is still new and we are actively expanding the APIs all the time, so we're always collecting customer requirements looking for common issues we can tackle next.
Beyond that, S3 is pretty good and it surely makes sense to model the content as files rather than blob.
You can zip very small files into bigger archives. It will save a lot of requests.
Generally speaking, it's very inefficient to deal with numerous small files because of latency and overhead. It's not a specificity of S3.
I want to second this. Backblaze's Personal Backup uses the same storage that B2 uses, and what we see is that tiny 10 byte files transmitted from "far away" (New Zealand to the United States) can be as slow as 15 kbits/sec transmission rate (yes, dial up modem speeds) because throughput is slaughtered by the latency to push each file.
Due to internet philosophies such as "TCP Slow Start" we do not see full throughput rates until after 40 MBytes in a single HTTPS POST if a customer is as far away as New Zealand.
One way we circumvent TCP Slow Start is to launch multiple threads. If TCP Slow Start results in only 1 Mbit/sec of throughput, we launch 20 threads to achieve 20 Mbits/sec of throughput. But you have to ask yourself what is going so horribly wrong on the internet if EVERYBODY has to write multiple threads on the same identical path through the internet in order to take advantage of the bandwidth available. Why not just allow 1 thread to get the full 20 Mbits/sec of throughput?
Also- Wasabi has fantastic pricing and full s3 compatibility.
> Care that explain why it’s substantially more than a few round robin’d smart reverse proxies moving data to the correct storage node?
That is exactly what it is. Plus making sure the network is provisioned with enough bandwidth to handle the extra copies, or even just the extra overhead. That's where Backblaze saves money, not purchasing "a few" servers, and only provisioning the network to handle the smaller amount of bandwidth required if you make a connection DIRECTLY from the client to the final storage Vault the data will stay on forever.
Companies that have a networking, power, and computer hardware and spinning drive bill less than 2% of their total yearly company revenue often have PLENTY of ways of solving these problems for a "small amount of money". Backblaze is quickly approaching an Exabyte of customer data on more than 100,000 spinning disks. We have to purchase millions of dollars worth of hardware a month now. If Backblaze can solve a problem that shaves a few percentages of cost off then it can save quite a bit. Backblaze can then pass that savings through to customers.
They also use Reed-Solomon and split data into multiple pieces to store on multiple servers. So they need all those "load balancing"-like nodes anyway and probably no new hardware or infrastructure is necessary to conform to S3 API.
> They also use Reed-Solomon and split data into multiple pieces to store on multiple servers. So they need all those "load balancing"-like nodes anyway
Yes. We definitely do "load balancing" or more accurately "disk space loading balancing" but we do it all in software. The net outcome is the same, but the cost is lower.
> probably no new hardware or infrastructure is necessary to conform to S3 API
No, it would require additional hardware we do not purchase at all right now. Backblaze's philosophy is to shave off cost at all layers if it doesn't actually contribute to uptime or durability. Put differently, if there is a lower cost way to achieve the same uptime or durability with some intelligent software or possibly an extra network round trip, we do it that way instead of purchasing extra hardware.
So we both agree it is more than zero cost? Backblaze saves that cost passes on the savings to customers. I'm not sure what the exact costs would be because Backblaze did not implement it that way.
> a few cores
By "a few" do you mean 10, 100, 1000 or...? For how much bandwidth will your solution support? For example, can your few cores support 10 Gbits/sec? 100 GBits/sec? 1 TBit/sec?
Backblaze is COMPLETELY FREE of worrying about these questions, because our solution does not require this additional step and this additional hardware, and therefore does not have this choke point.
S3 does a much better job of utilizing available bandwidth. It supports multipart parallel uploads and resumes nicely. In my experience, it will usually fully saturate any available upload capacity.
There are other protocols that may be more efficient for high bandwidth, high latency connections, i.e. WDT from facebook or UDT, but S3 does okay.
Granted it's not hierarchical, so listing a bucket lists everything in every folder, but I don't see why that's too big of a concern. Especially since there are pseudo directories in many of the S3 tools. Many people use S3 and have a folder-like hierarchical naming convention anyways.
With WebDAV, the URL is the key, and the value is the contents of the upload / download.
> The interface to upload data into Amazon S3 is actually a bit simpler than Backblaze B2’s API. But it comes at a literal cost. It requires Amazon to have a massive and expensive choke point in their network: load balancers. When a customer tries to upload to S3, she is given a single upload URL to use. For instance, http://s3.amazonaws.com/<bucketname>. This is great for the customer as she can just start pushing data to the URL. But that requires Amazon to be able to take that data and then, in a second step behind the scenes, find available storage space and then push that data to that available location. The second step creates a choke point as it requires having high bandwidth load balancers. That, in turn, carries a significant customer implication; load balancers cost significant money.
In fact, S3's REST API requires callers to follow HTTP redirects, and the PUT documentation expressly mentions the HTTP "Expect: 100-continue" mechanism precisely so that the S3 endpoint you reach in your initial PUT request does not have to handle the HTTP request body.
https://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.ht... https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPU...
> The Dispatching Server (the API server answering the b2_get_upload_url call) tells the Client “there is space over on “Vault-8329.” This next step is our magic. Armed with the knowledge of the open vault, the Client ends its connection with the Dispatching Server and creates a brand new request DIRECTLY to Vault-8329 (calling b2_upload_file or b2_upload_part). No load balancers involved!
Again, this could be done directly with HTTP. PUT to the first server, receive a redirect, PUT to vault-8329, receive "100 Continue", transmit file. There's no need to have a separate API call to get the "real" upload URL.
> 3) Expensive, time consuming data copy needs (and “eventual consistency”). Amazon S3 requires the copying of massive amounts of data from one part of their network (the upload server) to wherever the data’s ultimate resting place will be. This is at the root of one of the biggest frustrations when dealing with S3: Amazon’s “eventual consistency.”
Wait, I thought they were load balancers? Why does the load balancer need to copy any data once it's done uploading?
As for eventual consistency, there is truth to this complaint -- but much less truth than in the distant past. Every S3 region except us-standard has always had read-after-write consistency for new objects since launch, and as of August 2015, us-standard does too:
https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3...
If your PUT returns 200 OK, a subsequent GET will return the object, assuming you're using unique keys. This prevents the 2015-and-earlier problem where you'd create a new S3 object and enqueue a job to process it, then the job gets 404 Not Found while retrieving the new object.
There are other cases where S3's eventual consistency can be an issue, but none of them have been dealbreakers for my applications. Having said that: S3's consistency model is a weaker than the model B2 provides, so this is not an argument against providing an S3-compatible interface.
> In fact, S3's REST API requires callers to follow HTTP redirects...
Yes. Let's take the example of uploading 3 small files.
With S3, every upload call must start by hitting the same URL that is then redirected. To upload the same 3 small files in B2, it is a little different. For B2, one call is made to the "dispatch server" to ask for where there is spare space, then there is no redirect. The client must disconnect, and contact the final destination three times. So for this example, S3 would need to do 3 redirects (3 total network calls) and Backblaze B2 would do zero redirects but make 4 total network calls to upload three files. I hope that makes sense.
I'm not saying one is better than the other, and I freely admit S3 can be a little more intuitive. But it saves Backblaze money to not have such high loads and high uptime demand on the original URL.
> Again, this could be done directly with HTTP.
Yes I agree it could have been.
> There's no need to have a separate API call to get the "real" upload URL.
The "need" was to save money. See example above. Amazon S3 would require 3 redirects from the original URL, while Backblaze B2 chose a different tradeoff that is zero redirects and 4 total requests where 3 out of 4 requests were NOT redirected. The 3 out of 4 requests go to the final location DIRECTLY.
If you are only uploading one file, the Amazon S3 API is about as efficient as Backblaze and I agree with you. But if you upload 1 million files, the Backblaze B2 architecture is cheaper/simpler. But either way, it is the tradeoff we chose.
> as of August 2015, us-standard... has read-after-write consistency for new objects
TIL. We launched the B2 API before August of 2015, sorry if I propagated old info!
An API call that retrieves the address of another machine to contact is functionally the same thing as an HTTP redirect, just with different syntax, right?
> The client must disconnect, and contact the final destination three times.
This seems like the key idea that wasn't quite explained in the blog post: the client is expected to cache and reuse the same "vault" server for future requests, not just for multiple pieces of a single upload. If the client doesn't conform to that expectation, your approach would end up with exactly the same performance characteristics as S3.
What I find interesting is that unless I'm misunderstanding the API documentation [1], file downloads apparently do go through a load balancer. That is, you make an HTTP request to a single endpoint that's the same for all files in the account, and it fetches the data from whichever "vault" server is actually storing it. So does that mean that Backblaze's rate of incoming uploads is much larger than the rate of downloads? Otherwise it seems like you could just reuse the same load-balancing infrastructure for both.
[1]: https://www.backblaze.com/b2/docs/b2_download_file_by_name.h...
If you are only uploading exactly one file -> yes.
But if you are uploading 1 million files, a redirect system would result in 1 million redirects. The B2 system would result in less load on the dispatch server.
> the client is expected to cache and reuse the same "vault" server for future requests
Yes, exactly! This is the very core part of the B2 architecture. In fact, it is a waste of time and performance to keep asking the dispatch server for a location for every file. Just assume the last vault you got continues to be "valid" until the vault kicks you off with a 503. In B2 world, the 503 is NOT a fatal error, it means "go back to the dispatching server and ask for a new location to upload to".
> file downloads apparently do go through a load balancer
Correct. Because we wanted the ability to serve up static web content such as this picture (this is served out of B2): https://f001.backblazeb2.com/file/bucket9/cute3.jpg and do it in a highly available and highly scalable way if the content goes viral. The way we do that is we have load balanced "download servers" that also act as a caching layer. The FIRST time somebody fetches a file from the vault, the caching layer has to ask the vault to reassemble the file from parts, and then the caching layer caches it on fast SSDs inside the download servers. The second time (in 24 hours) that a customer requests the file, it comes out of the cache and not out of the vault. Otherwise, if a video went viral the vault would get crushed trying to reassemble the video for all 10 million views. :-)
The load balancers we used are described in a different blog post here: https://www.backblaze.com/blog/load-balancing-and-b2-cloud-s...
> does that mean that Backblaze's rate of incoming uploads is much larger than the rate of downloads?
Yes. VERY MUCH yes. Backblaze has more than 10x the incoming bandwidth as outgoing bandwidth. Backblaze started as an online backup company with the "Backblaze Personal Backup" solution. Online Backup is highly inbound heavy.
Though I really don't get what all this jerking off is about. I use B2. It took me like 2 fucking minutes to get working. It's ridiculously simple. My concern is more that they'll be acquired by some shitty company and I'll have to migrate off of them. It's not how long it takes me to setup an API.
> It took me like 2 fucking minutes to get working. It's ridiculously simple.
I'm so glad you said that, thank you! Our confidence is sometimes shaken when a customer says they would prefer it to be S3 compatible. It honestly isn't that hard, I always wonder what the disconnect in communication is.
> My concern is more that they'll be acquired by some shitty company
Backblaze is not for sale. And definitely not for sale to shitty companies. And even for non-shitty companies Backblaze is not for sale at prices any company would probably want to spend (or could justify to their board). I don't know if this is common knowledge but Backblaze is employee owned. We never took any significant VC funding, so the only people with voting rights on the board of directors are the original 5 Backblaze founders (including myself). We like what we are doing and we COMPLETELY control our own destiny and work environment. Literally nobody (except customers) can tell us what to do, how to price our products, or what features to build. Backblaze is a fun company to work at, and we all make a good living. So it would be inordinately expensive to buy us out just to put us out of business and dissolve our tight-knit group here and ruin all our fun. Our plan is to stay independent forever.
More background: the original 5 Backblaze founders have worked together for 19 years and have worked together at three startups. The first two starts were acquired by shitty companies. I do not want that to happen ever again, as long as I live. You know what it is like spending YEARS of your life building something beautiful, then hand it over to a shitty company and stand helplessly by while the beautiful product is first twisted, mangled, then withers and dies? It sucks. It sucks dead bunnies through a bent straw. So Backblaze was born different. Backblaze was born LEAN. The original founders and a few demi-partners going 18 months without salary, and another year on minimum wage, so that we wouldn't lose control of it. Employee-owned. Employee-run.
Not. For. Sale.
Keep up the good work!
Well… hold on. The docs say that S3 _can_ redirect PUTs and clients need to handle it correctly. This means a service could redirect every single incoming PUT and be S3-compatible. For example, this is how Internet Archive's S3ish service works:
https://archive.org/help/abouts3.txt
In practice, S3 doesn't seem to redirect very much at all, and I've never actually seen it redirect small objects. My speculation is that S3's frontend service is responsible for erasure coding and replicating inbound PUTs. Remember that S3's default storage class needs to send bits to multiple machines in multiple datacenters; it makes sense to me that this happens while you're uploading, instead of waiting for the upload to finish and then having the single point of failure replicate back out.
S3's API gives the frontend service the _option_ to redirect each incoming PUT, but it almost always dispatches uploads to the internal storage systems with a single PUT from the client.
> But it saves Backblaze money to not have such high loads and high uptime demand on the original URL.
The endpoint that handles `b2_get_upload_url` still needs to be highly available and serve lots of requests. Its load isn't 1:1 for each uploaded object and its failure wouldn't immediately stop all uploads, but… it still has to work.
> The "need" was to save money. See example above. Amazon S3 would require 3 redirects from the original URL, while Backblaze B2 chose a different tradeoff that is zero redirects and 4 total requests where 3 out of 4 requests were NOT redirected. The 3 out of 4 requests go to the final location DIRECTLY.
How much does this help? A search of GitHub shows that many users are calling `b2_get_upload_url` and then uploading a single file, a use case for which B2 requires twice as many HTTP requests as a typical (non-redirecting) S3 PUT:
https://github.com/UKn0Me/b2-api/blob/master/examples/b2_upl...
https://github.com/volnt/e2e-upload/blob/master/project/uplo...
https://github.com/djoon/webboard/blob/master/src/com/webboa...
https://github.com/jaeming/benjidalton.com-rails/blob/master...
> But either way, it is the tradeoff we chose.
Sure, that's fine, but I think this is disingenuous:
> And it’s a big deal that B2 is free of the “load balancer problem.” It solves for a huge scaling issue. When we roll out new vaults in new data centers in new countries, the clients are contacting those vaults DIRECTLY (over whatever network path is shortest) and so there are fewer choke points in our architecture.
When you connect to s3.amazonaws.com, you're talking to the S3 frontend service in the us-standard region. This is the service that knows how to list your buckets (by talking to the internal index services), knows how to encode and distribute your incoming objects (by talking to the internal storage services), knows how to request/repair/decrypt/assemble your outgoing objects, etc. It's not in the way, it's not a "choke point", it is the _only_ part of S3 that performs these frontend functions.
When S3 adds a new region, that region gets a new endpoint: s3.us-east-2.amazonaws.com, s3.us-west-1.amazonaws.com, s3.eu-west-1.amazonaws.com, etc. all perform the same functions for their respective geographical areas.
And if you request something from the wrong region:
$ curl -s https://s3.us-west-1.amazonaws.com/elevation-tiles-prod | tidy -q -xml
<?xml version="1.0" encoding="utf-8"?>
<Error>
<Code>PermanentRedirect</Code>
<Message>The bucket you are attempting to access must be addressed
using the specified endpoint. Please send all future requests to
this endpoint.</Message>
<Bucket>elevation-tiles-prod</Bucket>
<Endpoint>s3.amazonaws.com</Endpoint>
<RequestId>D18584530CD6A9E1</RequestId>
<HostId>
N7Oa6U6FouXBvz40Wa920r+D2qj5nWUZ8nxP14L+HhqfbEX5jeR+Yo80boODWqVxUFVeCeZVBqw=</HostId>
</Error>
You're right, but in my experience, S3 does not send 307 on each request (which it would have to do, if it were load balancing, right?)
I do care if I have to write my own API client for your storage backend. Or if you have examples to go off of. Backblaze doesn't seem to offer either for non-C++/Swift languages. Complete non-starter.
The, perhaps obvious, win of being S3 compatible is that you open the door to thousands of existing S3 clients already implemented in my different technologies, for free. And you get the developers who use them as customers.
> Backblaze doesn't seem to offer either for non-C++/Swift languages.
On each of the API web pages there are code examples for the following languages: 1) cUrl, 2) Java, 3) Python, 4) Swift, 5) Ruby, 6) C#, and 7) PHP.
For example, go to https://www.backblaze.com/b2/docs/b2_authorize_account.html and scroll ALL THE WAY TO THE BOTTOM of that web page, and you should see "Sample Code" section. Click on the blue buttons to see the different code examples.
If your favorite language is missing, we can add it for you! One of our client engineers wrote most of the code examples in all 7 languages in less than 1 week. With these working code examples, the expectation is you should be able to get B2 working in literally less than 2 days in any application in any language.
Would be nice if there was an official Node library.
While I think of myself as a 'C' programmer, the GIGANTIC amount of work being done in JavaScript nowadays is just amazing, and you can reach so many customers via JavaScript that I think JavaScript is a huge part of the future of computing.
> surprised that AWS, GCP, or Azure haven't acquired Backblaze by now
Sometimes press/customers/people ask us how many customers has Backblaze "converted over from" S3 to B2. To be honest, I think the answer is approximately zero. If a customer already has a Petabyte uploaded into S3, it is simply not practical or cost effective to download that from S3 and import it into B2. The S3 download costs are extremely expensive (9 cents per GByte to download out of S3, vs 1 cent per GByte to download out of B2).
Most of Backblaze's B2 customers fall into one of two camps:
1) New customers starting out that have just started to look at cloud storage and need to decide where to put their data. They look at S3, compare with B2, and make a decision and Backblaze B2 gets some of those customers. Realistically we don't even get half of these new customers either because B2 is missing a feature the potential customer needs, or because the customer has not even heard of Backblaze. So Backblaze is probably not causing Amazon too much lost revenue yet.
2) Multi-cloud customers. Any one "cloud provider" can have outages, including Backblaze and including Amazon. So if you store an entire copy of your data in Amazon S3, and also store an entire copy in Backblaze B2, you will (by definition) have both more durable data and higher availability data than a single copy in either one alone. By definition this doesn't actually cost Amazon S3 any sales, because you STILL need a complete copy in S3!! For multi-cloud, Backblaze B2 doesn't harm Amazon one bit.
> Seems like an obvious move (for Amazon to acquire Backblaze).
Amazon has never offered to buy Backblaze, but also we are not for sale. Or at very least not at the prices Amazon would probably want to spend (or could justify to their board). I don't know if this is common knowledge but Backblaze is employee owned. We never took any significant VC funding, so the only people with voting rights on the board of directors are the original 5 Backblaze founders (including myself). We like what we are doing and we COMPLETELY control our own destiny and work environment. Literally nobody (except customers) can tell us what to do, how to price our products, or what features to build. Backblaze is a fun company to work at, and we all make a good living. So it would be inordinately expensive to buy us out just to put us out of business and dissolve our tight-knit group here and ruin all our fun. Our plan is to stay independent forever.
This as much as anything encourages me to trust and invest in Backblaze. Long may you live. Maybe I'll get around to writing that Rust SDK I've been meaning to write for a while...
If you think harder, the closest service is google drive that already provides tens of GB for free to million of consumers. It's very different to backblaze, yet close enough in some regards. By some metrics, BackBlaze is a joke while google is backing up billions of android phones already and other stuff.
> Backblaze is a low cost service for cheap customers. AWS and GCP are "enterprise" services.
I agree. This is accurate. Backblaze is a reasonable choice for customers who want to pay less. Plus "enterprise" means the vendor sucks up to the customer and makes them feel important, visits the customer in person, stuff like that. Backblaze saves money by skipping the on-site visits, and we pass on the savings to the customer.
Backblaze is more of a self serve product. Customers can just sign up in the middle of the night, pay by credit card, and use the service for 1/4th the cost of an "enterprise" experience.
> Backblaze is a joke while Google is backing up billions of android phones already and other stuff
It is true that Backblaze is not as large as Google or Amazon yet. Customers that sign up with Backblaze must be comfortable that. I'm biased, but I would suggest there are advantages of storing your data in both locations: both Amazon S3 and also in Backblaze B2. One advantage is that if Amazon S3 has an outage, you can still retrieve that data during the S3 outage from Backblaze B2. Also, data stored in both Amazon S3 and also Backblaze B2 is more durable than data stored in either location alone.
In the end, is it Ok for some customers to have an alternative choice to use Backblaze?
Carbonite is also a good choice for some people with certain needs.
And if you aren't using anything else, Backblaze Personal Backup is also a fine choice for some customers! Backblaze has a feature called "Business Groups" that is focused around allowing IT people to pay for all their employee's backups in a central location, and even manage backups in an organization. To be clear: "Business Groups" is a free feature, so whether you are a company or an individual our online backup is the exact same cost. Customers are not forbidden (or discouraged) from using Backblaze Personal Backup in a corporate setting.
Choice is a good thing for customers. Competition is a good thing for customers.