Hacker News Reader

Comment by noobermin | Hacker News Reader

Something very relevant to this discussion was this article[0] from a few weeks back. A bit of a clickbait title, but the points are very relevant: C, based on the PDP-11, has a computation model that no longer really tracks with how a computer actually works today, and in those gaps is where people insert in stuff like speculative execution in the first place to be faster while maintaining the guise to app developers of a "fast PDP-11".

[0] https://queue.acm.org/detail.cfm?id=3212479

Old discussion (skip the first reply for meat!)

https://news.ycombinator.com/item?id=16967675

View on HN

Correct me if I'm wrong but assembly doesn't offer many more entries into this computation model either does it, especially when considering some non-standard extensions to C? So they've added a lot of processor features with no API.

Is there a reason speculative execution had to be added to the processor and not compilers? Would we have been better off if the true memory model was exposed to compilers?

> Is there a reason speculative execution had to be added to the processor and not compilers? Would we have been better off if the true memory model was exposed to compilers?

Binary compatibility is the reason. You can't break the memory model or the instruction set unless you actually have the ability to re-compile everything that your users want to run. This almost never happens in the personal computing world, and emulation is only a partial solution. Intel managed to transition from the 8080 to the 8086 without backwards compatibility by preserving assembly language compatibility, but every architecture shift since then has required either emulation or a backwards-compatible processor mode.

> Binary compatibility is the reason. You can't break the memory model or the instruction set unless you actually have the ability to re-compile everything that your users want to run.

I fail to see how having shared speculation is incompatible with existing binaries. AMD processors for example check security before speculating, instead of just running things in parallel and hoping they don't need to reset registers later.

Can you explain how speculative execution is systemic to x86? I can't see it to be honest.

> Is there a reason speculative execution had to be added to the processor and not compilers?

Wasn't this supposed to be the lesson of Itanium and VLIW?

I have read (without understanding for myself) that VLIW it had two problems: (a) no one can actually write an appropriate compiler, and (b) that's not surprising because the non-literalness of assembly is a useful hinge point allowing CPU designers to evolve their implementations while retaining a stable interface.

But this thread is talking about going even further and exposing new CPU features to programmers, and not just compilers. Does anyone have thoughts about what abstractions they would like to see?

> But this thread is talking about going even further and exposing new CPU features to programmers, and not just compilers.

You could argue they already do, __builtin_prefetch and it's associated assembly instructions offer a way to manipulate the cache, but at the same time they seem to hedge their bets and pretend the cache isn't there. On the other end we've got programmers that know it's there and jump through hoops to try and keep things in contiguous memory, tricking the CPU into doing the correct thing.

That is truly an elephant in the room situation, everyone knows there is a magic black box there but do their best to ignore it because there is no other option.

> Wasn't this supposed to be the lesson of Itanium and VLIW?

It was the lesson, but that doesn't mean it was the correct one. In an alternate universe where there was no x86_64 we might have learned the opposite lesson.

I think I'm with you about caches.

I sometimes wish we wrote modern software for swarms of little 16-bit CPUs with 64kB of RAM and dealt with larger data structures by passing messages to other CPUs and to a DRAM-backed database.

And yes, a lot of software architecture is actually about emulating the above picture on a machine that pretends to have a flat address space, even though it actually only has 64kB of truly random-access memory.

the non-literalness of assembly is a useful hinge point allowing CPU designers to evolve their implementations while retaining a stable interface

Case in point, we have already seen a few times how unanticipated NUMA or thread scheduling weirdness can totally tank performance- and these are finer details!

The ground is littered with the dead bodies of CPU designers who said “we’ll just make a smart compiler”.

The CPU knows what the software is actually doing and has done in the past. The compiler does not.

The CPU can change implementations; the compiler must bake any assumptions in at build time.

Speculative execution is required so long as memory latencies are so large. That trend isn’t reversing itself anytime soon.

>The CPU knows what the software is actually doing and has done in the past. The compiler does not.

profile guided optimizations?

Profiles capture only program brhabiour under test Data and sssumed that that pattern is prevalent. If the actual runtime behaviour is bimodal or worse, then only one mode will be optimized at most. A CPU can keep statiatics at runtime (the branch predictor does) and thus adapt ever so slightly to the actual current program behaviour.

Runtime optimizations have there own problems. Sometimes the rare case will also be the most sensitive, sometimes the statistics will take too long to turn around, etc.

At least profile guided optimization gives you a degree of control.

Not that people making the smart-compiler assumption in the days before JITs were justified... but that sounds like an argument for using JITs.

It seems like the main problem is that the compiler doesn’t know the specific target it will run on. We actually do have our smart-enough compilers. They just come in the form of microcode that we can’t audit or even opt out of because the true instruction set of the machine is a trade secret and we’ve been locked out anyway by code signing.

Well you can target a specific CPU model and its specific cache sizes and functional units, even the microcode revision it uses. Some cryptographic code certainly does that. But it is a major overhead to comprehend all that detail for even a small section of code. There are undocumented micro ops that are present, but they are not really relevant to this, they are not the cause of speculative execution attacks.

The compiler is poor at anticipating the many types of dynamic program behaviour. But the CPU is also bad at dealing with it. The very simple statistical model for branch prediction is used because space on the die is precious. Cycle accurate simulation is done routinely, if anyone had come up with a much better (and equally general) predictor that could fit on die it would already be in use. People have tried many variants of dynamically recompiling machine code. For example, early MIPS processors (R3000) recompiled everything to be slightly closer to the specific CPU, and exposed pipeline stages (like delayed branch, delayed load), software page fault handling etc.

In some ISAs you can even encode the branch prediction hints the compiler makes (from analysis or profiling) into the opcode. But the gains are limited.

Reply on news.ycombinator.com