Linux: Controlling access to the memory cache
lwn.net
lwn.net
___________
P.S.: I already thought about the backward compatibility considerations being primordial. That would have meant that the technical advancement transparent to the existing usage was a reason in itself, but that doesn't quite hold water. The 16 bit to 32 bit architecture switch opened the possibility of adopting a secondary (but slower) layer of RAM going over 64 KiB and keeping the 16 bit manageable memory range close to CPU, as (at least one of) the cache layers.
A cache speeds up nearly all code written for any system, not just the 0.1% that someone bothered to hand optimize the memory access patterns for.
I recall that when that technology came out, there was a heavy marketing stunt involved. For the boast's sake, performance had to be conjured. And as seamlessly as possible. And as noticeable as possible. That really affected the design development, and not in a good way. The problem (in what I quoted) here is the poor measurement base, the wrong comparison base, and what is counted or discounted as value. So that cache really speeds up nearly all the code written for any system. Such a flat model does not, however, make much distinction between a memory hogging application that might not require performance and some server process that has to be up and running with as little lag as possible. What is counted there in the terms of cache performance really? That 0.1% (that someone bothered to hand optimize the memory access patterns for) might be the only one really needing that cache. That, however, is not how you sell technology to the public, isn't it? You have to show them a general speed-up, even when it doesn't matter. You have to show them that the same application that used to do something in a given period of time does it now in less. That's how you sell. But that's not how you develop, hopefully.
I am going to claim that it isn't. I think you underestimate the amount of speedup the cache gives you. Few applications require performance so little that they will tolerate a 50-100 times slowdown. Even if they could, who's to say the right 0.1% ends up optimized? You add a new function without profiling it, or a user has a slightly different use case than you do, and suddenly the hot path is waiting for main memory in every loop. This was all decided a long time ago, general purpose caches won and scratch memory lost.
> You have to show them that the same application that used to do something in a given period of time does it now in less. That's how you sell.
Yes, that sounds like a swell idea. What is wrong with that?
Applications and users had or still have to wait a lot more than 100 times access slowdown when it gets to trashing (excessive number of page faults), memory swapping, or simply normal waits for some slow actions.
"Even if they could, who's to say the right 0.1% ends up optimized?"
I haven't said it's about only 0.1%, and that it has to be dictated by some policy. For all I care, even if they all would be manually cache managed, I don't expect the result to be worse than the caching mechanism currently employed, which works without really knowing what's going on.
"This was all decided a long time ago, general purpose caches won and scratch memory lost."
A long time ago they had a different world. Their reasoning might have been true back then. We are creatures of reason, however, aren't we?
"Yes, that sounds like a [sell] idea. What is wrong with that?"
Salesmanship and the engineering of future tech have at their cores different objectives. At least in my book.
Compilers (even the high-end modern ones) are very quickly clueless about both memory and execution profiles. Those problems are hard to reason about statically. Is this pointer the same as this other one ? Will this loop be executed 10 times or 10^6 times ? The hardware being a dynamic optimizer has a lot of runtime information (pointer values, cache entry usage, ...) to take advantage of.
Sure, one can argue that we just need better JITs, PGO, and programming languages that are easier to analyze statically. In the current state of affair we are way better off leaving this to the hardware.
By the way you do have "some" control over it using NT moves on x86. Turns out almost nobody uses them except by hand in very specific peak-bandwidth code. Compilers don't dare emitting those since the penalty of getting it wrong outweights the benefits of getting it right.
That is something I'm not so sure about. That might have been true before, when the hardware served well the then simple computational model of a single high frequency processing core. Things got more complicated since the times of first CPU cache offerings and hardware general use-case implementation can only give you so much. The performance boost that once could be relied on without much care now dissipated. This cache thing now become just a leaky abstraction. We can not abstract it away completely if we care for performance, nor we can control it in any meaningful way. It is not a good model anymore. Just recognize it as a failed experiment.
"Unless you want programmers to explicitely manage this fast memory..."
Yes, that's exactly what I have in mind (and implied before). Most applications do not need maximal performance and can disregard the fast memory completely. That will only mean that those processes either will not consume their share and leave it more for the processes that need it, or that the fast memory will (occasionally) be used indirectly, through the optimized bits in the layers that the given process relies on. This memory, however, won't be unproductively overwritten or invalidated on every cold cache or whatnot.
That's how we did it in the good old days on the 6502 (zp address mode) and we liked it like that...
There are chips in the wild today that give you the option to move some or all of your cache to manual control if you want to. This project is about giving you the software interface needed for you to get access to the feature those chips already have.
I feel I have to point this out not necessarily to usefulcat, but because I see a lot of comments arguing against a false-dichotomy strawman of this project.
Or maybe we can add another pseudo-socket interface in the style of rtnetlink, or the ioctl this one fella has proposed. Frankly, they need to take a page from Windows and offer proper programmatic Interfaces to all of this stuff.
cgroup does provide programmatic control over this feature. perhaps you had something more in mind. may you please elaborate ? thanks !