Hacker News Reader

Comment by userbinator | Hacker News Reader

Apple's GPU Dynamic Caching ensures that cache and memory spaces are dynamically assigned based on the actual needs of different tasks and workloads

Isn't this not too different from what Intel DVMT was doing twenty years ago?

https://en.wikipedia.org/wiki/Dynamic_video_memory_technolog...

Then again, this is the same Apple that took "integrated graphics" / "UMA" as a marketing point.

View on HN

No, it's about replacing the static shader register allocation with more of a cache, allowing for more dynamic occupancy usage of the GPU. Memory is still memory.

The patent mentions MMU? Is Apple mapping the GPU register file into the CPU address space?

See paragraphs 31 & 32. If I'm reading it right, they answer that question, and positively.

My read of the patent is that it discusses a virtual memory system that allows the GPU to page memory into cache according to various scopes, (based on tile or threadgroup or maybe even shader stage?).

But aren't caches already a dynamic copy of main memory? Don't GPUs already prefetch buffers as needed? What is actually new here?

I don't think this is correct. My understanding is that this is about more efficiently utilizing the GPUs available resources (e.g., cache) at the core-ish level. Like imagine if hyperthreads could share register space based on what they're doing at the time, I think.

The OP isn't really about UMA, but about caching.

In any case, in Intel's DVMT you have a dedicated memory space for graphics. It is a dynamic amount, but it is still dedicated for graphics. You need to copy data from system memory over to graphics memory for it to be handled.

In UMA, no data copying is needed. There's 1 memory space

Zero copy shared memory absolutely is available on all modern IGPUs. Do people really think such a low hanging fruit was ignored for the past decades?

https://www.intel.com/content/dam/develop/external/us/en/doc...

https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...

> Do people really think such a low hanging fruit was ignored for the past decades?

Yes, they do, it's mentioned as some major M1 revolutionary technology all the time. It's utterly absurd, but that's the power of Apple's marketing.

Hell, the UMA was considered a downside on the original XBOX because everyone was used to each processor having it's own memory bus, so even though the bus was (for the time) a sprightly 6.4GB/s, it was considered lesser.

I can’t run llama on an intel iGPU, because it OOMs - the memory is unified because it’s in the same space and shared on M1. Not just marketing.

So, how much memory is available to your Intel iGPU? What was the machine?

If it OOMs, then one valid assumption is that it OOMs because it has less memory available than there is for a GPU on M1 but we can't know that if you fail to mention the specs.

You need to copy data from system memory over to graphics memory for it to be handled.

No you don't. You can map any page of physical RAM into the GPU address space.

This has been the case since at least the Intel i810. Here's a relevant part from the i815 documentation:

https://i.imgur.com/p53Dxsn.png

These GPUs reserved a variable amount of your system memory for the GPU. So if you had 4 GB of RAM the GPU would maybe use 500 MB leaving you with 3.5 GB of RAM for your CPU. There was often a option in the BIOS that allowed you to change the amount of memory reserved for the GPU.

That option is not a hard limit on the RAM IGPUs can use.

There is a hard limit with DVMT that is described in the white papers by Intel. Let's say you allow up to 256 MB of RAM to be allocated for graphics. The memory could be partitioned in a way that 128 MB are fixed memory that are always only available for the graphics driver and 128 MB are DVMT memory that can be used by the OS if the graphics driver doesn't need it.

Reply on news.ycombinator.com