Hacker News Reader

Comment by papersmith | Hacker News Reader

I've heard that each Mnesia table is limited to 4GB. That sounds like a pretty serious limitation now-a-days. Not sure if there are any ways to work around that.

View on HN

That is a limitation of the 32-bit OS ... If you are using a 64-Bit OS that is no longer a problem.

I have a 64-bit Xeon with 8GB of RAM and I have created tables lareger the 4GB as a test ... to make completely sure ...

Does that apply to DETS storage as well?

It says on this page (http://erlang.stacken.kth.se/faq/x1197.html) that "Dets uses 32 bit integers for file offsets" and "In practice your machine will slow to a crawl way before you reach this limit."

As far as I know it does ... the 32bit limitations in Erlang are more about the OS then anything else.

You can add table fragments to overcome this limitation. However, keep in mind that Mnesia isn't designed to store very large datasets. It works best with data sets that fit in RAM.

But RAM can be substantially larger than 4GB these days.

Heh, the same principal restricting the mnesia database to 4 gigs on a 32 bit system is restricting the ram size to 4 gigs on a 32 bit system. (Some OS/Hardware magic can get around this but for the most systems this is true).

True. However, the reason Mnesia is better suited for in-RAM data sets is that its disk-only storage engine, dets, requires a potentially long repair process if the VM crashes and a dets table isn't closed properly. This could keep your table inaccessible for a while after you restart the VM.

Reply on news.ycombinator.com