Hacker News Reader

Comment by munificent | Hacker News Reader

I really really liked this article, and reading the comments here is blowing my mind. Did we read the same thing?

I think it's a strong insight that insight that chip designers and compiler vendors have spent person-millenia maintaining the illusion that we are targeting a PDP-11-like platform even while the platform has grown less and less like that. And, it turns out, with things like Spectre and the performance cost of cache misses, that abstraction layer is quite leaky in potentially disastrous ways.

But, at the same time, they have done such a good job of maintaining that illusion that we forget it isn't actually reality.

I like the title of the article because many programmers today do still think C is a close mapping to how chips work. If you happen to be one of the enlightening minority who know that hasn't been true for a while, that's great, but I don't think it's good to criticize the title based on that.

View on HN

As someone who does large scale computational work for a living, stuff like this is close to my heart. I often run into serious memory and run time constraints due to poorly written codes that have rather dumb understanding of the real underlying machine implementation that modern processors actually have rather than this imaginary PDP-11 that we've been brought up to believe.

I wonder how much I could save (and how many more sims I could run) if my codes were rewritten in a language that has an abstract system that is much more cleanly and simply translated to what the computer actually does in 2018.

Actually, the author's argument about PDP-11 is interesting because C would have never been considered a low level language back then, for any platform.

Wiki definition, also what I was taught in my first CS class:

"A low-level programming language is a programming language that provides little or no abstraction from a computer's instruction set architecture—commands or functions in the language map closely to processor instructions. Generally this refers to either machine code or assembly language."

The term is evolving to match the time, as shown by the author's interpretation already being higher level than the original intention despite the goal of preventing exactly that.

> Actually, the author's argument about PDP-11 is interesting because C would have never been considered a low level language back then, for any platform.

Sure, agreed, but I don't think it's super interesting that words evolve in meaning over time.

What I find strange about the comments here is that some people think the article's title is bad even though my experience is that many people today do think "C is a low level language" is a reasonable thing to say.

I guess I'm arguing that it is a reasonable thing to say today in the right context. When I talk to web developers who don't understand anything about computer architecture for example it's much easier for me to tell them I'm a low level developer rather than explain that I design digital hardware (FPGA) and write drivers and firmware to interact with it. But I do agree that it's important developers know that C doesn't correspond to the CPU in the way they might mistakenly think it does

Heh. Yeah. Was programming ARM assembly in 1988. C compiler was far too high level. Look how its always saving these registers to memory! Meanwhile I'm dropping into FIQ mode just for the banked R8-R14.

Now, sure, people say C is low level, and compared to Java it sure is. But it isn't low-level.

I also really liked the article and found it thought provoking. This is all way above my pay grade but I like to think that there is a more optimal cpu design and language pairing that we will eventually reach and it's fun to imagine what that might look like.

Obviously, it would be very hard to shift the incumbent model in reality. We just have to look at the lack of prosperity for the Itanium and Cell processors to see how hard it is to achieve success. But imagine if new computer languages had been created just for these processors. Commercially this would make little sense but it might be possible to create languages that fully used these processors yet retained simplicity for developers. Or maybe it isn't possible to beat the clarity of sequential instructions for human developers or maybe Out Of Order processing is the optimal algorithm. There are other changes coming too such as various replacements for DRAM that either integrate more closely with the CPU (such as 3d chips) [1,2] that by reducing the latency of main memory, could actually bring us back closer to the C model of the computer? or just change computing entirely...

[1] https://www.extremetech.com/computing/252007-mit-announces-b... [2] https://news.ycombinator.com/item?id=16894818

It'll be an interesting exercise to ask the Clang folks to relax backward compatibility, this designing a new language, if it makes their compiler go faster.

That could be deployed as a new language, or adding features to existing ones, like value types in Java, or even compiler switches that relax some C rules for faster speed. Imagine -fpointers_cant_be_cast_to_ints or -freorder_struct_fields.

It seems like the article is mostly useful for inspiring research; that is, most of us aren't the target audience.

I'm wondering what will happen as GPU's become more general-purpose. What's next after machine learning?

Would it be possible to make a machine where all code runs on a GPU? How would GPU's have to change to make that possible, and would it result in losing what makes them so useful? What would the OS and programming language look like?

> It seems like the article is mostly useful for inspiring research; that is, most of us aren't the target audience.

As a group of professionals, it is highly beneficial for us to be interested in these things. People who design languages and compilers do it largely on what is perceived as being demanded, and us as programmers are the ones that create the demand for new languages.

To put in other words, if programmers aren't aware of what's going wrong with our current languages, they cannot express their need for new languages. So, there's less incentive for researchers to produce new ways of programming computers. It is much more tempting to "please the masses" in a way that causes this local-maximum problem. It's much more interesting to research problems that translate into mainstream use than academic things that nobody actually uses.

I still think a research project (either academic or in industry) with a hardware component would be the best way to explore radical new processor architectures that are further away from C.

- Hardware designers are conservative. They aren't likely to implement a different hardware architecture because "programmers demand it" (really?), unless there's existing research showing how it can be done and a compelling reason why customers will buy the chips.

- As a hobbyist language designer, I'm still going to target something that exists and is popular: x86, JavaScript, wasm, C, or something like that. A low-level language targeting a platform that doesn't exist isn't all that appealing. But, someone might get some good papers out of doing the research.

I think it is going the opposite direction, where cpus get more powerful graphics cards integrated more and more into them. This allows for matrix math to become a bit more standard in day to day programming.

However, in the opposite direction where a gpu becomes more like a cpu, if streams could do some level of limited branching without slowing the whole thing down, it opens the door to threading frameworks and design patterns where you write a loop in code, every thread gets it's own copy of memory, and on the threading front, it just kind of works for a lot of generic code.

Then if gpus added some sort of piped-like summation-like instruction, in the cases in a loop where variables need to be shared, they can still be added, subbed, mul, div, or mod, easily and quickly, allowing for what looks and acts like normal code today, but is actually threaded. That would kind of bring code back to where it is today.

Who knows? It's kind of fun to speculate about though.

> cpus get more powerful graphics cards integrated more and more into them.

Actually the wheel of reincarnation seems to have stopped at least for the time being. It seems that there is a fundamental, hard to reconcile, disconnect between a latency optimised engine like a CPU and a throughput engine like a GPU. Hybrids CPUs like larrabee or extensions like AVX512 do not seem to be enough.

Short term, probably the best we are going to see is separate CPU and GPU cores in the same die (or more likely jus the same package), but even that is likely suboptimal.

Or, conversely, could the iPhone 11 have 20 cores that are optimised not for latency but for power-efficient execution, such as how much you can do before the device runs out of battery? These would still be ARM, and backward-compatible with mainstream languages like Swift, so will have a low barrier to entry.

Maybe apps will be allowed to run longer in the background, if there are always extra CPU cores available that don't consume much power.

I think the bigger take away for me is that what constitutes low level has evolved over time. I still think C is low level because you have to manage your own memory and can play tricks with pointers and memory that other languages protect you from. Meaning that low level takes off a lot of the training wheels. The compiler still does what it can and optimizes things but you have a far greater ability to shoot yourself in the foot than in some other language. The arguments about what a "real" low level in these comments seem mostly pedantic.

> I think the bigger take away for me is that what constitutes low level has evolved over time.

Yeah, this is a good insight. The height of the overall stack has grown. The lowest low level is lower than it was in the 60s and the highest high level is higher. So we need more terms to cover that wider continuum.

I agree that this article hits pretty hard against a lot of assumptions about how our machines are working.

I also feel like the author is trying to say something about how imperative scalar (meaning 'operates on one datum at a time') languages are causing more trouble than they're worth. Sophie Wilson said something similar in her talk about the future of microprocessors [1]. This implies that declarative and functional semantics would be more amenable to parallelization, as the author mentions in the article, as well as allowing the compiler more freedom to deduce a suitable 'reordering' of operations that would better fit the memory access heuristics the machine is using.

[1] https://youtu.be/_9mzmvhwMqw?t=26m30s

> cost of cache misses

How is the C memory model a leaky abstraction here? What better way do you suggest? Are we not fine coding sequential (in memory) datastructures in C?

C leads you to believe that memory access has uniform cost regardless of address. What is the perf cost of:

    *foo

Depending on what foo points to, and which memory you have previously read, the cost can vary by close to two orders of magnitude on many chips.

C does give you the ability to control those costs, but controlling how you lay out your data in memory and controlling imperatively in which order you access it. But the language doesn't show you those costs in any way.

> But the language doesn't show you those costs in any way.

I think it's pretty clear. Access memory sequentially, and you can expect to hit the cache. Access more memory than the cache size in a random order, and you can expect to pay memory access latencies (100s of CPU cycles).

I doubt you would be willing to manage the cache yourself in every line of code. That would be a lot of code. Some programmers might want to tune cache eviction behaviour by changing it in a few controlled points in time. But not in a way that couldn't be exposed to assembler/C. (I don't know if that's even realistic from a hardware architect's point of view).

> Access memory sequentially

Except memory is virtual. Memory location 0x1000 might be forward, or backwards compared to 0xFFF, depending on the state of the Translation-lookaside buffer (TLB).

Ever notice how (when ASLR is disabled), programs all start at the same location?? (https://stackoverflow.com/questions/14795164/why-do-linux-pr...)

Hint: Virtual address 0x0804800 doesn't belong at physical address 0x0804800. The OS can put that memory anywhere, and the CPU will "translate" the memory during runtime.

This means that in an obscure case, going forward a linked list (ie: node = node->next) may involve FIVE memory lookups on x86-64

* Page Map Lookup

* Page Directory Pointer lookup

* Page Directory lookup

* Page Table lookup

* Finally, the physical location of "node->next".

An even more obscure case (looking at maybe address 0xFFFC, unaligned) may require two lookups, for a total of 10-memory lookups (the page-directory walk for page 0xFFFC, and then the page-directory walk for 0x1000).

There is a LOT of hardware involved in just a simple "node = node->next" in a linked list. Its not even CPU-dependent. Its OS configurable too. x86 supports 4kb pages (typical in Linux / Windows), 2MB Large Pages and 1GB Huge Pages.

> Except memory is virtual. Memory location 0x1000 might be forward, or backwards compared to 0xFFF, depending on the state of the Translation-lookaside buffer (TLB).

Does that matter though? I would assume that the "prefetcher" (or whatever it's called) can make its predictions in terms of virtual memory.

Regarding the linked list, it has been common wisdom for a long time that one should prefer sequential memory instead of linked lists. A nice benefit is that this simplifies the code as well :-). Growing and reallocating sequential buffers might not be possible in very dynamic/real-time and decentralized architectures - like a kernel - though.

> Does that matter though?

Yeah. Meltdown means that the OS (as a security measure) wipes away the TLB whenever you make a system call. So all of those cached TLB entries disappear every syscall due to Kernel Page-Table Isolation.

And then the CPU core has to start from scratch, rebuilding the TLB-cache again.

So last year (when we didn't know about Meltdown), a system-call was fast and efficient. This year, on Intel and ARM systems (vulnerable to Meltdown), system-calls are now forced to wipe TLB. (But not on AMD-systems, which happened to be immune to the problem)

Both AMD and Intel implement x86 instruction set, and now the performance characteristics is different between the two boxes for something as simple as "blah = blah->next".

-------------

The important bit is that C is still quite "high level", and indeed, even Assembly language "lies" to the programmer through virtual memory. The OS (specifically page-tables) can interject and have some magic going on even as assembly-code looks up specific addresses.

The simple pointer-dereference *blah is actually incredibly complicated. There's no real way to know its performance characteristics from a C-level alone. It depends on the machine, the OS, the configuration of the OS (ie: MMap, Swap, Huge Page support, Meltdown...) and more.

And all that's without even getting into things like swap, or memory-mapped files. Or, to really go all-out, mmap() an NFS file - and you could find your mere pointer dereference waiting for the network.

How about a different example entirely? Memory nowadays is either CPU (and the GPU can access it) or GPU (and the CPU has a window into it). It's terribly important to use the right one: if a chunk of memory is mostly used by the CPU, it needs to be CPU memory, and if it's mostly used by the GPU, it needs to be GPU memory. But there's no good way to specify that.

You might argue that a modern computer is more like programming a tightly-bound, nonuniform multi-processor system. And I'd agree. But C doesn't much to help program such a thing.

Still, what you normally do and can assume is that you're using just "CPU" memory with pretty predictable (I think) latencies and a transparent caching layer.

When different things are mapped into the address space, that's an abstraction the programmer (or the user) consciously made. It should be possible to figure out the performance characteristics there.

Of course, many programs work on various machines with their own performance characteristics. You should still be able optimize for any one of them by querying the hardware and selecting an appropriate implementation. If you want to put in the work.

I don't think assembler/C is such a big problem here. But then again, I'm not a low level guy (in this sense) for now.

Reply on news.ycombinator.com