Hacker News Reader

Comment by hajile | Hacker News Reader

Reducing max document size from 4GB down to 8MB seems hyper-restrictive.

For those interested, looks like the guts of CouchDB are going to be swapped out for FoundationDB.

https://blog.couchdb.org/2020/02/26/the-road-to-couchdb-3-0-...

8MB is just the default, you can switch it back to 4GB if you want, but you won't have an easy time switching to 4.0 due to the 8MB limit imposed by FoundationDB.

It seems really odd to me that they are changing to an 8MB limit. "Document storage database" that can only hold 8MB per document. It can't hold full text output for a large documents... What am I missing here?

In CouchDB 4.0 the backend of CouchDB will be switching to FoundationDB, which will have a 8MB limit, so they're preemptively making the change. You can remove the limit now if you'd like.

Yes, but how is a 8mb document size limit a good thing?

FoundationDB needs is a consistent (as per CAP) database, to make that work in a distributed fashion, there are limits on distributed transactions in size and time. CouchDB needs to adopt those in order to move to FoundationDB.

The 3.x series will be supported for a long time for folks who can’t move up. Until then, as others have stated, you can up the default limit.

In CouchDB terms, you’d store data like this in attachments, not as raw JSON.

If you're trying to store single GB documents in couch, you're doing it wrong... Unless those are binaries you can usually fragment data logically across many documents, then write custom views to aggregate however you need to.

Updates on huge docs would be painful!

I agree that 4GB is more than a sane person should probably be using. I don't think that going above 8MB is very hard or uncommon though. If I'm going to spread everything across many different documents and document types and then join them all together, I then have to make a case for why I'm still choosing Couch over a RDBMS.

I agree in large with your point that multi-GB documents is perhaps excessive but this does create a heck of a migration problem for a lot of users that aren't even close to 4GB.

It does, but keep in mind for 3.0 this is a change to the default settings, not a hard cap. The idea is to give people lots of warning and time to do any migrations necessary prior to 4.0.

Couch/Pouch combination is really slow when document +attachment are too large. Based on my experience, Practical limit was more like 20-30Mb, and I'd suggest not even getting close to that. 8Mb simply recognizes reality.

That’s another great point, an exactly why we are doing this. It is a best practice already.

At least since 2.0 (haven't used Couch before), the docs have always recommended to only use small documents in a CouchDB and to use an external storage for large files.

The IBM Cloudant free tier only allows Doxs up to 1 MB.

So this doesn't really come as a surprise or feel hyper restrive to me.

That still exists? I thought apple bought them and shuttered them.

FoundationDB got re-released as an open source project: https://www.foundationdb.org/blog/foundationdb-is-open-sourc...

Reply on news.ycombinator.com