Hacker News Reader

Comment by gpderetta | Hacker News Reader

Considering that on average you have a branch every 5 instructions and the reorder buffer of high performance OoO CPU is in the order of a hundred of instruction, such a CPU is pretty much running under speculation all the time (often of multiple branches at the same time).

Not being able to fetch new cachelines when under speculation would be a huge blow, as exposing memory level parallelism is one of the most important features of OoO CPUs.

Edit: also fetching a cacheline is not really undoable as it is observable from other CPUs via the coherency protocol: i.e. while it might be possible to hide to the local cpu that a cacheline was loaded under a failed speculation, the effect can be still observed by another core by noticing that an exclusive line is now shared (by timing for example the latency of an atomic instruction)

View on HN

"Considering that on average you have a branch every 5 instructions and the reorder buffer of high performance OoO CPU is in the order of a hundred of instruction, such a CPU is pretty much running under speculation all the time (often of multiple branches at the same time)."

Yes, it is. Remember, again, that the vast majority of those instructions can still be speculated, because the vast majority of instructions are not loads or stores.

Now, certainly, the expensive ones are loads and stores, but i'm just pointing out that the hundreds of instructions you are talking about in the buffer are mostly not loads and stores.

It's true that lowering memory level parallelism would be a huge blow, as the vast majority of time in well-tuned cpu bound apps is usually spent in stalls waiting for memory (otherwise, if it's really just arithmetic bound, it may make more sense to run it on a GPU or something), and this would just increase it.

The real question is what percent can you prove are safe to speculate, and at what point can you prove that safety (ie assuming it must be dynamically speculated, can you prove safety with enough cycles left that it matters). If you have 5 instructions, yeah, no, probably not.. But it may also be the case that the execution environment can prove it safe for you as the program executes and tell you.

I expect getting back this performance is going to be done using a variety of methods, some cooperation between jits/compilers and processors, and possibly some weird abstractions around marking memory you want to protect or not (IE not speculate around).

I mean, in the absolute worst case, you could make loads/stores take constant time and speculate as much as you like :)

It's just that this has a much higher performance cost right now than not speculating at all (by a few orders of magnitude)

I doubt that any CPU manufacturer is going to go first in crippling its memory subsystem.

I think realistically the only safe and not completely performance crippling workaround will be, at the very least, to run any untrusted code under a separate address space (assuming that the cpu is immune from meltdown). That doesn't necessarily require a full blown separate process, but something like memory protection keys might work.

The alternative is full static analysis and source level annotations, which realistically is only going to be done for very few programs and will still be error prone.

Yeah, i don't disagree at all. Something like this what i meant by "cooperation" (lfence or equivalent for code that can't be isolated but must be protected) + "weird abstractions" (IE back to segmented land we go)

Forgot about segments! Time to resurrect them on x86-64 cpus?

Reply on news.ycombinator.com