Hacker News Reader

Comment by lmm | Hacker News Reader

LLVM seems to be where the interesting action is these days, I think mostly because it's adopted a superior modular design. GCC's technical design was compromised by RMS' desire to keep it tightly coupled for licensing reasons, and in the past few years the pain of that has really started to be fault. Which is somewhat ironic considering how the free software movement started.

View on HN

Both LLVM and gcc suffer from the problem that they're vast underdocumented code bases with Internal Platform Syndrome and mostly-but-not-quite-finished internal tools which generate undebuggable code. The actual working set of knowledge which you need to do serious work on the compiler, such as porting it to a new backend, mostly exists as tribal knowledge only, and the only way to find this out is to either investigate the source yourself or to try and ask for help on the mailing lists. LLVM's is significantly more helpful than gcc's, because LLVM has more people participating and more momentum behind it, but you're still not guaranteed to get an answer. Actually producing a working backend is an enormous investment of time and effort.

By way of example: I'm interested in the Broadcom Videocore IV processor, also known as the main processor on the Raspberry Pi. I bounced off both gcc and LLVM, and for much the same reason --- after getting the port about 75% complete I ran into blocking issues that I couldn't fix on my own and couldn't get enough assistance on the mailing lists to fix. Which is a shame. (I should point out that prior to that I did do a VC4 backend for the ACK compiler. It generates terrible, terrible but working code.)

I have run into commercial compilers which were easy to port, and generated good code, to boot. Volker Barthelman's vbcc even has a VC4 backend, which produces amazing code, and his compiler is a tiny fraction of the size of either gcc or LLVM! But it's not open source. Maybe one day we'll get a similarly good open source small compiler...

Offtopic: what's a good way to learn about the Videocore processor? I now own a B+ and a Pi 2 so it might be worth my while.

I have absolutely no idea about the processor in the Pi 2, but I got all the specs for the Pi 1 from here:

https://github.com/hermanhermitage/videocoreiv

(It's a beautiful processor. 32 registers, built in DSP, built in vector processing, some really elegant instruction design, dual core, and enough weirdness that writing good code is satisfying without being frustrating. No double precision float, alas, although it's got single precision, and for some bizarre reason there's no adc or sbc instruction which makes 64 bit arithmetic annoying.)

If you're interested in playing with it, you might also like a look at my half-finished pure-VC4-code boot loader here:

https://cowlark.com/piface

Stick it on an SD card, hook up a serial terminal, and it'll boot directly on the VC4 without even starting up the ARM.

Whoa, does that mean there's a chance of replacing the proprietary bootloader on the Pi? It's the most common complaint I see here.

You can replace the proprietary bootloader on the SD card. You can't replace the one in the ROM. And while running executables is easy --- just drop a VC4 binary called bootcode.bin onto the root directory of the card and the ROM loader will load and run it --- there's a lot of hardware setup which still needs to be figured out; such as initialising the SDRAM.

I originally wanted to get Prex (a Minix-like microkernel OS with a Unix interface which doesn't need an MMU) running on the VC4, but I need an actual working compiler for that.

May I ask you: why programming VC4 (apart from taking over the control of the rest of Raspberry Pi?) It's only 250 MHz? And if you want the new control of the Raspberry Pi, why, as it's so proprietary?

Because the VC4 is really cool, basically.

Clock speed is only a very small part of the performance of a chip when comparing across completely different architectures (like VC4 vs. ARM).

Pi 2 apparently has the exact same GPU/VPU as the Pi 1.

The only thing they changed was the arm cores and the memory subsystem.

Gcc code is arcane: it relies heavily on macros to define some kind of DSL for the compiler backends (check out the .def files), but if you want to contribute seriously you need to learn that DSL and how it works behind the scene somewhat, whereas llvm is C++. That being said, it's an interesting experience to dive into that code.

As opposed to LLVM's tblgen?

I give you: https://github.com/llvm-mirror/llvm/blob/master/lib/Target/X...

I agree with you that gcc is much less pleasant than LLVM+Clang, but to say that LLVM is better becauses it uses C++ everywhere is simply incorrect. In fact, using this kind of DSL for backend definitions is good design.

That looks very readable. Maybe the gripe was that GCC isn't readable? I don't know, I haven't looked at GCC code.

The LLVM DSL engine is really annoying to use, mainly because the pattern matching engine is pretty simple, and it's possible to express patterns in the DSL language which the pattern matching engine is unable to actually match. Until you know where the limits are it's really frustrating; you write a pattern which should work and the compiler blows stack or produces an impenetrable internal error because the pattern matcher has given up or gone into a working loop.

A large part of any LLVM backend is actually custom C++ code for matching and expanding patterns which the pattern matcher either can't or is unwilling to do (e.g. custom backend-specific DAG nodes). I direct you at this file:

https://github.com/llvm-mirror/llvm/blob/master/lib/Target/X...

github will give you a warning before loading it because that single C++ source file is over a megabyte...

I must admit that my knowledge of llvm is very limited. I did make the (perhaps wrong) assumption that it would be easier to understand on that basis alone (well, and the fact that the people writing it know how to code C++ properly, I guess for major projects like Gcc and llvm you may expect well engineered code). I felt that the whole macro business isn't as solid nowadays as what templates could allow in terms of type checking (and perhaps some macros involved for syntactic sugar alone; caveat with the cryptic error messages for errors in templates), hence my previously stated opinion.

Note that I am not complaining about the DSL, just how things are quite difficult to figure out (and untyped macros, as stated above).

LLVM's target backends are also written in a custom DSL (tblgen). Is that not a good thing?

GCC developers say that things are better now. I really don't know; I haven't checked.

But the simple fact remains that they did intentionally create hard to maintain code and spent several years rejecting patches that would clean up the monstrosity as a defense against people using GCC code in nonfree programs.

It's hard to trust their judgment now.

Things are better, yes, but there's still a long way to go. This LWN article covers one developer's ongoing frustration: https://lwn.net/Articles/629259/

Yes, the efforts to make it hard for nonfree programs to use parts of GCC also make it hard for free programs to use parts of GCC.

> compromised by RMS' desire to keep it tightly coupled for licensing reasons

It's certainly better to have an intermediate language and a modular design to compile to it and from it, but I thought this was introduced by LLVM and not gcc because (1.) compiler performance mattered more at the time, so it was better to couple everything to save some performance with dirty tricks, and (2.) people just hadn't realized that this was the right design. (The same way that it took time for the relational database abstraction to come around and replace the tendency for each program to use its custom internal database.)

Is there any specific reason to believe that gcc's design was influenced by licensing reasons?

See https://lwn.net/Articles/629259/

OK, I was wrong. Thanks for pointing this out. This is very interesting (and, in my opinion, quite depressing).

Reply on news.ycombinator.com