Hacker News Reader

Comment by dragontamer | Hacker News Reader

> Right now we have to fetch everything into the CPU before we can make decisions.

Think about virtual memory. The memory at location #6000 is NOT at where the program thinks is at memory location #6000.

In fact, the program might be reading / writing to memory location 0x08004000 (or close to that, whatever the magic start address was on various OS like Linux / Windows). And then your CPU virtually-translates that address to memory-stick #0 column#4000 or whatever.

Because of virtual memory: all memory operations must be translated by the CPU before you actually go to RAM (and that translation may require a page-table walk in the worst case)

View on HN

Well, all memory operations must be translated by an MMU. So can the MMU be a distributed beast that lives in the CPU, the memory and the IO?

> So can the MMU be a distributed beast that lives in the CPU, the memory and the IO?

That "blah = blah->next" memory operation could be:

* Going to swap thanks to swap / overcommit behavior on Linux/Windows.

* Or going to a file thanks to mmap

* Going to Ethernet and to another computer thanks to RDMA

So... no. I'm pretty sure the CPU is the proper place for that kind of routing.

why is that? we need the CPU to handle page fault interrupts, in order to populate the RAM. But assuming the page is already in RAM, there's no reason any of the memory accesses actually need to go through the CPU. (hardware can already raise interrupts; if the MMU can raise a page fault indicator, then you might be able to bypass CPU entirely until a new page needs to be loaded)

moreover, if we have support for mmap at the MMU level, we can cut the CPU bottleneck for disk access entirely. the disk controller can already handle DMA, but there's simply no way for things that aren't the CPU to trigger it. DirectStorage is an effort for GPUs to trigger it, but what if we could also trigger it by other means?

Pointers are virtual addresses but memory is accessed physically. All of the means of translating virtual to physical are in the CPU. If you are proposing throwing out virtual addressing, I imagine you won't get a lot of support for that idea.

Or just have ubiquitous IOMMUs, which we should do for a million other reasons anyway.

I think the point was more like the following situation.

You have a linked list. Each memory location, since it’s user-space, stores the virtual address to the next location. How do you offload a program to process this “in situ”? You’d need to translate these to physical addresses. Parent is 100% correct about the challenge this poses.

Okay, lets say page-faults are off the table for some reason. Lets think about what can happen even if everything is in RAM still.

* MMAP is still on the table: different processes can share RAM at different addresses. (Process#1 thinks the data is at memory location 0x90000000, Process#2 thinks the data is at 0x70000000, but in both cases, the data is at physical location 0x42).

* Physical location 0x42 is a far-read on a far-away NUMA node. Which means the CPU#0 now needs to send a message to a CPU#1 very far away to get a copy of that RAM. This message traverses Intel Ultrapath Interconnect or AMD Infinity fabric (proprietary details), but its a remote message that happens nonetheless.

* Turns out CPU#1 has modified location 0x42. Now CPU#1 must push the most recent copy out of L1 cache, into L2 cache... then into L3 cache, and then send it back to CPU#0. CPU#0 has to wait until this process is done. If CPU#1 wants to modify the data again (or even read it), it may require messages from CPU#0 (who is now the owner of the data, according to simple MESI models).

Modern computers work very hard to hold the illusion of a singular memory space. Eventually, these details are turned into a consistent memory model and become well-ordered sequential operations. The CPU is a good place for that.

---------------

That's how stuff works _today_. If you wanted to make a new programming model that's incompatible, that's fine. (CUDA does it: GPUs don't have as many virtual-memory features as a CPU. And __shared__ memory has a different model than L1 cache.)

But if you invent a new memory model that does things differently, it means that it won't work for the vast majority of code. Which means you need to bootstrap a new programming environment (much like how CUDA bootstrapped a new community from scratch).

Or the mapping info for your process is shared with that coprocessor (the OS just keeps this around in RAM anyway). Heck, you could have an OS-provided code that needs to be loaded so that it can properly resolve that mapping (which is what the TLB does by the way - it has a well-defined structure and when it’s missing from the cache if I recall correctly it’ll fetch some info directly from RAM until it gets to a point where it has to generate a page fault).

I don’t disagree that the memory model becomes more complex. For one CPU cache invalidation becomes really tricky. So do memory coherence rules.

I’m less clear how mmap matters here. That’s just a mechanism the OS uses to hand out views into the page cache to the process - if you’ve solve the virtual-physical mapping (which you have to do) then mmap is not relevant.

You’re spot on though that the particular design decisions are critical for this to be successful - pick the wrong point on the complexity/cost/perf curve and your solution will definitely be DOA.

Reply on news.ycombinator.com