Without a bootstrap process like this one, it could happen that you run git pull and then can't build the compiler because it's using a new feature that the version of the compiler you have doesn't support yet.
The wasm process ensures that you can always build the latest commit.
I maintain what i said, i do not understand how this is a good idea
Imagine a scenario where you are testing out a brand new RISC-V development board. The vendor has only provided a C compiler for this board as is often the case. You want to be able to use the zig language to write programs for your new development board but no zig compiler exists for this board yet. That means you need to compile the zig compiler from source. The latest version of the zig compiler is written in zig. Again you don't have a zig compiler so how will you compile the zig compiler from source? You need a way to go from C compiler to Zig compiler. That's what this is describing. It does not make sense to maintain two completely separate versions of the compiler. The "real" one written in Zig and the "bootstrap" one for compiling from C. So the zig source is compiled into WASM and stored in the repo. On a system with only a C compiler this WASM can be ran instead of a native zig binary. The WASM version can then be used to compile an actual native zig binary.
They are compiling the current compiler to wasm and then using that compiler to build future versions of their compiler.
In other words, they are doing the described rust approach but instead using a platform agnostic target instead of doing a binary. That allows them to build on any platform that has a C compiler and to use current language features without needing manually backport them.
They could directly target C or C++ but that runs a greater risk of accidentally generating UB. Targeting a bytecode decreases that risk.
WASM file is produced by compiling new compiler, one written in Zig.
EDIT: since my performance critical sentence is getting misunderstood: https://news.ycombinator.com/item?id=33914718
As a generalization, the people who are motivated enough to work on a language, want to use that language. It's only natural that they would want to write their compiler in that language too, if practical. Contributors to the Zig project would on average probably be more proficient and productive in Zig than they would be in a language they don't care about so much.
It's also just helpful to have the people who are designing the language working in that language regularly in the context of a sizable and nontrivial project.
Have they written am incremental compiler then? Or just an old-fashioned slow batch one? Compiler architecture matters much more than whether it's runtime has a GC or not.
It's more important than the choice of programming language. Not all C++ is fast.
> You can't implement an incremental compiler in Zig?
I never said this. This is a thinly veiled ad hominem.
I understood your original comment to imply that you disagree with their choice of self-hosting the Zig compiler because they should instead have focused on architectural improvements, in which case I disagree because the two things are completely orthogonal in my mind—the benefits of self-hosting have little to do with performance, and certainly don't come at the cost of it. I apologize if that's not what you were implying.
> This is a thinly veiled ad hominem.
I never personally attacked you, so I'm not sure why you think this is ad hominem. That's a very uncharitable interpretation of my comment.
https://kristoff.it/blog/zig-new-relationship-llvm/
https://mobile.twitter.com/andy_kelley/status/15651098597708...
Actually, yes.
A compiler is not just a dumb filter that eats text and spits out machine code. It can provide infrastructure to other tools (like linters, analyzers, LSP servers...), and even allow importing and using parts of it in user programs.
Some of this can't be done in a different language (like Haskell) without some very crazy FFI, and might add an extra runtime dependency for those tools, which might not always be desirable.
The number of ignorant comments in this thread is astounding. All this criticism feels like it was written by back-seat drivers who have no clue about the complexities of language design or compiler implementation.
The alternative would be to port your Haskell compiler to the new CPU too in order to set up a self hosting toolchain. Much more work involved, because you not only have to be proficient in Haskell, but you need to have Haskell compiler implementation skills in addition to your own compiler.
Thank you!
My full-time job is making a compiler for a high-level language, and I only considered systems languages (e.g. Zig, Rust) as contenders for what to write it in - solely because compiler performance is so critical to the experience of using the compiler.
In our case, since the compiler is for a high-level language, we plan never to self-host, because that would slow it down.
To me, it seems clear that taking performance very seriously, including language choice, is the best path to delivering the fastest feedback loop for a compiler's users.
I honestly fail to see why would a lower level language be faster, especially that compilers are notorious for their non-standard allocation and life cycle patterns, so a GC might actually be faster here.
The nonstandard allocation and lifecycle patterns are a major part of the reason I want a systems language and not a GC - it means I have strictly more control over when allocations happen, I can do cheap phase-oriented allocations and deallocations with arenas, etc.
Rust's compiler is an interesting example. It was originally implemented in OCaml (which has a reputation for being a GC'd language with good runtime performance), and then rewrote to Rust in order to self-host - and got faster. In contrast, the Go team rewrote from a systems language to Go (which also has a good reputation for runtime performance), again in order to self-host, and it got slower.
Nonstandard lifetimes are not really helped with arena allocators though, and not everything is needed in each phase, or is there that divided phases at all. But you may be right, I honestly can’t tell with certainty.
Didn’t know about that backend though, will check it out, thanks!
Why wouldn't you? I understand why not for high level languages like Python or Ruby (since they're interpreted) but not for low level ones. Rust for example is also bootstrapped.
Bootstrapping is actually a very old tradition that any programming language must implement in order to showcase the power of language.
Rust was originally written in OCaml before being self-hosted, and it wouldn't be as fast (or would be even slower ;) ) today if it was still OCaml.
And remember, low-level =/= poor abstractions. I think there are several novel abstractions available in Zig which the compiler devs probably want to make use of themselves.
They might have good language abstractions, but manual memory management is simply an orthogonal implementation detail to solving a problem — dealing with that is simply more work and more leaky abstractions.
https://media.handmade-seattle.com/practical-data-oriented-d...
Is there a reason why keeping a compiler in, say, C would be a bad idea long-term?
You'd want faster compilation so that you can test your changes without it breaking your flow where having to wait a 1-5 minutes means you'll end up reading HN or checking chat for 10 or so minutes. That's also why there's interest in hot-code reloading and incremental linking to make it faster as it will further reduce compilation to just the changes you've made and nothing more.
I haven't seen much opportunity to improve algorithm performance because which algorithms are applicable is heavily constrained by language design.
In my experience the difference between something like Zig and a language with managed memory is large.
Today’s managed languages are very fast. For example, if Java is not fast enough for your HFT algorithm, than nor is C++ or generic CPUs even! You have to go the custom chip route then. Where there is a significant difference between these categories is memory usage and predictability of performance. (In other applications, e.g. video codecs you will have to write assembly by hand in the hot loops, since here low-level languages are not low level enough). Since these concerns not apply to compilers, I don’t think that a significant performance difference would be observable between, say a java and zig implementation of a certain compiler.
It's not about code size or efficiency. They are talking about the ability to use the latest versions of Zig for further development of the Zig compiler. This improves and speeds up the compiler development cycle by removing the limitations imposed by the bootstrap process.
There are of course many places you might run code that cannot host a compiler for any language. You use a cross compiler for those.
The benefit of this compared to the previous approach is that the developers no longer need to maintain a second compiler written in C++.
There was a time when self-hosting your language made sense.
I did notice that you did not identify the system which can run a C compiler but cannot run a C++ compiler.
Why doesn't it now?
Once, good compilers were rare and expensive, and coding a compiler in your new language proved to people that it was adequate to implement a compiler.
Nowadays good compilers for everything are common and free. Coding a compiler for your language in your language demonstrates only that you could afford to waste the time, because literally every language suffices to code a compiler in.
You seem to be presupposing this is true, but this is a pretty controversial claim. Do you have any justification for it?
As other people in this thread have mentioned, there are other benefits to programming your compiler using your own programming language:
- You learn your language (and its rough edges) first hand
- You can try out & experiment with new language features
- The authors of Zig clearly think zig is better to use than C++. (If not, why would they bother inventing zig?). It makes sense that if you think Zig is a better language than C++, you'd want to write your compiler in Zig and not C++.
Lots of compiler authors (you know, the people who get a vote) disagree with you on this: Clang, VC++, etc are written in C++. Rustc is written in rust. The main zig compiler is written in zig. DMD is written in D. And so on.
And anyway, whats so great about C++ that it would make a good language to write compilers in in the first place? Why shouldn't I write my compiler in plain C, or Rust, or Zig, or Go, or whatever strikes my fancy?
People can write whatever they like in whatever language they like. We need not invent spurious reasons. Surely the creator of Zig already knows Zig; coding a compiler in it teaches him nothing new. He could experiment with draft language features in any program at all: nothing about the task of compiling Zig code is better for that purpose than myriad others.
Obviously the second compiler for Zig was done because the author wanted to. But having done so demonstrates nothing meaningful about Zig. It only tells us its author took time to code what we know, with utter certainty, was a strictly unnecessary program, instead of doing something actually useful.
Doing stuff that is not useful is allowed. We need not pretend otherwise, or make up excuses.
But it remains manifestly unnecessary, because everything done in the "second system" (cf.) could have been done with less effort by altering the first. Achieving it demonstrated nothing more about Zig than that there were people willing to take time to do it.
Arguments from "velocity" have, in particular, always been unconvincing. People code faster in a language they like, and credit the language, but other people are equally quick in other languages.
I’m curious - why does all this seem to bother you so much?
C got its legs from a whole OS coded in it. The compiler was very incidental. I don't have any programs in Rust on my SSD. I have one Haskell program: pandoc. I have no Java, C#, Scala, Clojure, Kotlin, or Erlang programs. I have one in Dart I don't use anymore.
A substantial, useful program in Zig that would be too much work to re-implement in a mature language would do orders of magnitude more for Zig than a second compiler. Maybe a collection of pipewire modules, or matrix gateways?
Perceptions of velocity by language advocates have very, very little to do with actual velocity in industrial settings. The latter is what matters, but is actual work to measure. Anybody measuring seems not to be publishing.
You would not want anything that must be depended upon coded in it, in case it fizzles like almost all languages do, but that leaves huge scope for valued additions on top.
No, there wasn't. They had a half-implemented compiler in C++ which implemented just enough features to compile the proper zig compiler. It didn't have feature parity.
> Obviously the second compiler for Zig was done because the author wanted to. It only tells us its author took time to code ... a strictly unnecessary program, instead of doing something actually useful.
You think dogfooding isn't useful? Wow, thats a really bold claim. I know legions of product engineers who would line up to disagree with you. Dogfooding is tremendously useful.
Failing to dogfood creates an empathy gap with your audience. (You don't understand your own work as well!). If you don't immerse yourself in your own creative output (eg, rust developers writing rust code, musicians listening to their own music, etc) then your output is usually lower quality because you aren't training your taste. How do you know what design decisions are good and bad, or what to focus on, if you aren't using your own output?
And not dogfooding is a motivation killer on passion projects. Why build something for fun when you don't even get to enjoy the fruits of your own labor?
I suppose Andrew Kelly could have a second medium-to-large project that he uses Zig for in order to understand his users. And he could go back and forth between coding Zig in C++ and coding that other project in Zig. (I think Jonathan Blow does that with Jai.) But that sounds like a huge waste of effort when he could just be spending all his time writing the zig compiler in zig, which is what he wants to be doing anyway.
I'm not making a technical argument. V8 seems to work fine despite being written entirely in C++. But V8 is just programming to a spec someone else wrote. Programming language specs don't come out fully formed. They need to be thought through, experimented with and obsessively argued over.
How would Zig become a good language if the team that invents it doesn't even programming in it themselves?
People do often run compilers on RaspPis and the like (though rarely for any good reason), but those compilers are most often coded in C++, whatever the language it is compiling, just like Zig's original compiler.
Most of the Rust compiler is in C++, and they have no plan to change that.
What? I just went and searched the Rust compiler, and the only C++ code to speak of was LLVM itself, and a thin wrapper around LLVM to provide a C API. Which is pretty much the same state that Zig is in right now. (Although in Zig's case, LLVM is optional)
Yeah, it's a lot of C++ that the Rust compiler relies on, but I don't think most people would really compare things like that.
Real embedded is more challenging, but even then... DOS ran on 640 Kb of RAM for userspace, and it had C and even C++ (albeit pre-ISO) compilers, and many embedded platforms these days come with something comparable. Forth is really only necessary if you want to bootstrap on really low-memory stuff.
"Why" is a different question. ~
Reason why it still is running poorly, because it is the same implementation, just went with optimized code gen steps, making the source non human readable
Seems like i was right when i made this comment few days ago: https://news.ycombinator.com/item?id=33763102
Not everyone is born with knowledge, we learn by igniting discussions
Now the 85 interesting comments responding to you are lost to many HN readers.
It would have been better if you had been more moderate.
lol, very few things migrate from C++ to C. Rather they do the reverse (like how GCC migrated from C to C++).
It's not that they're too lazy to rewrite this in C but rather that it's impractical when there are still breaking features to be implemented.
Also the process is `zig -> wasm blob -> c -> zig` where they're not commiting C source code to the repository at all, it's just used as a stepping stone to compile the real compiler from the bootstrap compiler.