Or NSA had them do this intentionally, and thus, its not a bug :)
A malicious process allocates a 256 member array. Then it creates a conditional where the speculatively executed part writes a byte at offset array + kernel_memory_value. The speculative branch is executed but then backed out, but the byte in the array was touched so it is in cache now. Then the malicious process reads all of the members of the array and looks for one that returns much faster than expected (is in cache) and they know the value of that byte. Rinse and repeat to read the rest of the kernel memory. It's not going to give you MB/s of throughput, but it's plenty fast to read some key material or process tables or anything like that.
It's a very impressive attack. My hat is off to whomever thought it up.
The crux is:
array[value_of_kernel_memory_byte] = 1;
So it speculatively indexes into the array by the value stored at that memory address and writes the byte. Then to figure out what the value is you just have to see which element in the array is cached.
array[value_of_kernel_memory_byte] = 1;
This assignment gets rolled back like it's supposed to. It's when reading the array after the rollback that the exploit measures that a read to array[value_of_kernel_memory_byte] is faster than the rest because that index is already in the cache.
Or maybe the idea is speculative execution itself was a dream of the NSA that was Inception-planted into the brains of CPU designers in the 90s; who knows what the theory-of-the-hour is regarding 3-letter-agencies and their capabilities.
Ultimately I think what we're really learning is that guarding against things like microarchitectural attacks on contemporary superscalar, OoO CPUs is going to be an uphill battle that we didn't ever think of due to incidental complexity (among other reasons), and will serve as a new class of attacks. Who knows how long this bug class will exist; we've killed some. What's also likely is that, like most security failures in the industry, this is a result of various things like basic lack of forethought/ill considered design, as opposed to plants (3 letter agencies aren't responsible for the vast majority of security failures you see, it's simple mistakes). But peddling conspiracy theories involving them gets you upvotes, so, you know...