Hacker News Reader

Comment by londons_explore | Hacker News Reader

It's a shame we're using the top of pointers for other data... It's almost as if we didn't learn from the "3gb memory hole", the "gate a20 memory limit", and countless other times when using special address bits came back to bite system designers as RAM got bigger...

View on HN

Is this really still a problem with virtual memory management? The virtual address space size could be a property of the process (e.g. 48 bits vs the full 64 bits). The vast majority of applications today doesn't even need more than 32-bits of address space and could use the top 32-bits for something else if the memory mapping hardware ignores the unused bits).

2^48 == 256TB.

Anyone needs more than 256 Terabytes of RAM?

It ought to be enough for anyone

I am flirting with the idea of having 1TB of RAM on my servers as a small start-up. Having 256TB in 20 years seems reasonable.

You're gonna need at least that much to run the Electron apps of 20 years from now...

A 128-bit CPU is not something I'd find outlandish in 20 years. OTOH, I was reading my e-mail on a 64-bit workstation (largely by accident, but still).

In any case, I'd be quite surprised to see a 64-bit CPU doing such heavy lifting in 2040.

Note this is for the virtual address space, not only for actual physical RAM. Having 256TB mmap’ed (to another kind of storage) does not seems totally out of reach.

And in fact support for 5-level page tables was recently merged into the Linux kernel because people were coming close to this limit and needed more virtual address space.

True, but also note that a 52b mode (i.e. 12 bit tag) is also supported.

I can't imagine anyone needing more than 4PB in the next 30 years, and the security gains now is (in my opinion) worth the potential refactoring in 30+ years that would be required.

The couple year old power9 E980, can take 64T of ram. So there are people who have problems that need lots of addressable memory today.

Even so, it wouldn't be that hard to actually build a system that could in theory try and map 4PB of storage via mmap today, its about 1/2 a rack of fairly common equipment. Given there are various companies selling 50T ssd (https://www.anandtech.com/show/11639/viking-ships-uhcsilo-ss...) its probably possible to do it in under 10U.

Hmmh? A few years ago there were already a few sites with more than a PiB of storage total. I don't know of any which used some virtual memory or single address space scheme to access those, but I wouldn't want to rule it out.

A Javascript website has entered the chat

It's 256TiB of virtual address space. If you include NVMe drives memory-mapped into the address space, hitting that cap is not entirely unreasonable.

Intel added support for 5-level page tables (bumping virtual memory to 2^57 bytes) a few years ago: https://en.wikipedia.org/wiki/Intel_5-level_paging.

The estimated number of atoms on earth is on the order of 1e50, which is 60 bits. If you take the 4 bits out of the 64bit address for tagging, you can still address individual atoms.

EDIT: My math is bad, see comments below.

I can unsarcastically say that no one will ever need full 64bit addresses on this planet.

2^66 bits - estimated storage space at Google data warehouse as of 2013 [0]

2^71 bits - total hard drive capacity shipped in 2016 [0]

I doubt your calculations.

[0] https://en.wikipedia.org/wiki/Orders_of_magnitude_(data)

Yes, you're totally right, my math is way off.

10^50 is definitely not 60 bits: log_2(10^50) = 50 * log_2(10), and log_2(10) > 3

Reply on news.ycombinator.com