Crafting Interpreters: Calls and Functions
craftinginterpreters.com
craftinginterpreters.com
Like you, I don't have a CS degree and find a number of topics like this to be intimidating. So my hope is that readers will come away from this both knowing more about languages and also less afraid of diving into other topics that seem scary.
Where before I haphazardly ignored features like quoted strings or escaped characters, I now have extremely high confidence in my parsers’ completeness and correctness. Also the error reporting rocks and the code is easy to read!
Thank you!!!
I have seen problems that probably do merit a special DSL or language, though I find it somewhat unfortunate if you need such a thing.
One example that comes to mind, Soul lang: https://news.ycombinator.com/item?id=19791154
Even if you don't think the whole book is for you, I recommend checking out the introduction sections. Or at least just the discussion on the terminology of "interpreter" vs "compiler".
https://craftinginterpreters.com/introduction.html
https://craftinginterpreters.com/a-map-of-the-territory.html
I've been following your progress since the beginning and I really appreciate the work you've done. Crafting Interpreters was really helpful for me when I began building my own language.
Writing did get somewhat harder when I transitioned from part 1 (the Java interpreter) to part 2 (the C VM). C is a more finicky language in a lot of ways. Things like the order of declarations matter. So figuring out which order to introduce snippets of code within a single chapter tends to take a little longer, but it's not a big difference.
My rate of progress has generally slowed over the past few years, for a lot of reasons. My initial enthusiasm has naturally waned. There isn't much novelty left in the project for me — writing each chapter is much like writing the previous one. So it's mostly an exercise in discipline now. Meanwhile, of course, I keep stumbling into other new things that I am excited about, so staying focused is hard.
The last few years have also been personally difficult for me in a number of ways. My mother got cancer (in remission now!), two family members died, two friends died, my dog (the one on the back cover of my previous book) was in ill health for a long time and then had to be put down. Lots of other personal stuff like that. 2018 was one of the worst years of my life, even though I'm still very fortunate in many ways.
But through all of that, I've managed to keep going. I work on the book every day, so I worked on it in the hospital waiting room while my mom got a CAT scan. I wrote a measly 59 words the night after we put down my dog. I wrote in the hotel before my friend's memorial service and on an airplane on the way back from my aunt's funeral.
In some ways, the project is a constant comfort in my life, and it's also taught me a lot about the power of perserverance and discipline. I'm really looking forward to it being done and taking a long break. But I'm also looking forward to seeing what I'll be able to do with this discipline next.
It's interesting to hear how you had the code pegged down for this book at the start. The book definitely feels like the result of such considered and planned design. But one of the features I most appreciate in your book are your reflective asides, such as the Design Notes [0]. I also attempt that in my writing, but almost always end up dragging myself down rabbit holes, including rewriting/renaming/restructuring entire code scripts/snippets, if not the project codebases that I've drafted and refer to. Guess I can't delude myself into thinking such bikeshedding is inevitable for this kind of writing!
[0] "Logic versus History" https://craftinginterpreters.com/parsing-expressions.html#de...
"What's in a Name" https://craftinginterpreters.com/introduction.html#design-no...
One of the things I tell myself is that many of these very difficult experiences were also growing ones for me. I, of course, would prefer that the friends and family I lost or who got sick were still here and healthy. But I also have a greater empathy and understanding when I interact with other people going through these same tribulations.
> It's interesting to hear how you had the code pegged down for this book at the start. The book definitely feels like the result of such considered and planned design.
I basically had to write it this way. My previous book was like a recipe book where each chapter is relatively unconnected so I could just start writing. But this book walks through two complete implementations of the same language, which I also designed for the book. So just about any tiny change to anything can affect very distant chapters.
I wasn't even sure if the book would hang together until I designed the language and wrote the code. If I couldn't get to a little toy language I liked or if the implementations had been a sprawling mess, I would have passed on the project. But a couple months in, everything kind of clicked and I realized I basically had to the write the book. It's like the thing was already there and I was just responsible for putting it all down on paper.
> I also attempt that in my writing, but almost always end up dragging myself down rabbit holes, including rewriting/renaming/restructuring entire code scripts/snippets, if not the project codebases that I've drafted and refer to.
I do a lot of that too. Fortunately, having the code pinned down helps. I basically can't touch the code now because doing so would destabilize existing chapters. The fact that I publish incrementally also keeps me from endlessly noodling.
But it is always a struggle to rein myself in and stay focused on making forward progress.
I started the second half in Rust about a week ago, I don't have enough vacation days to work through it at the pace I did the first half. Finding it a bit harder so far, but I think that's just because I'm still new at Rust.
You may have already mentioned this but will you be publishing a physical copy when your done? I'd love to get one for the shelf.
Yes. Just like with "Game Programming Patterns", once the chapters are done, I'll put together a print edition, eBook, Kindle, etc. Based on the length, I may end up doing it as two separate volumes, which is crazy to think about.
*Off topic: Very excited for Luxe 1.0 and Wren as well.
[0] https://github.com/munificent/craftinginterpreters/blob/mast...
[1] https://github.com/munificent/game-programming-patterns/blob...
Yup, though I ended up significantly expanding it over time. This book is a lot more complex because the snippets interact across chapters. Also, I wanted the ability to test the code as it exists at different points in the book.
> but did you ever look into other bookmaking frameworks
Briefly, yes. But, in general, I'm not much of a "framework" person. I'd rather spend time rolling my own thing than reading docs or fighting with the limitations of some other framework.
Of course, doing it from scratch can be a large undertaking, but I think people often overestimate the effort of writing bespoke software. The build script for my book is not a general-purpose build system for generating book sites. It is literally hard-coded to be used for only one single book on Earth.
That makes it much simpler. It doesn't need extensibility or configurability. It has nothing like an "API" or even docs or tests. The output is its tests — if I like the resulting HTML, the build script is by definition working correctly.
It's also easy to add hacks in there that only make sense for the book, like:
# The "(?!-)" is a hack. scanning.md has an inline code sample containing a
# "--" operator. We don't want that to get matched, so fail the match if the
# character after the "-- " is "-", which is the next character in the code
# sample.
EM_DASH_PATTERN = re.compile(r'\s+--\s(?!-)')
Replace "book" with "scientific paper" plus a few other terms, and you have basically given a perfect explanation why the vast majority of academic code is so terrible :p
Hmm. I may be getting rusty, but do modern compilers actually still pay attention to the "register" keyword?
Also, I may have read this too quickly, but if I'm not mistaken you limit the number of parameters to 8, yet the number of arguments passed to 255. Does that make sense?
Thank you once again for a brilliant read!
You know, I'm honestly not sure. If nothing else, it sends a signal to human readers of the code.
> if I'm not mistaken you limit the number of parameters to 8, yet the number of arguments passed to 255.
You are not mistaken. I changed the limit, but missed that part. Filed an issue to fix this:
https://github.com/munificent/craftinginterpreters/issues/46...
Thank you!
I know, hardly authoritative. But it matches precisely what I remember from when I was hacking on perl with an eye to performance at the time.
Maybe drop that bit regarding 'register'? The rest of that section is still perfectly valid. That is, assuming the compiler doesn't actually perform the local variable/keep in register optimisation itself, but I think it can't do that at least in the general case. I only remember vague things about this. At least aliasing may defeat the compiler optimisation.
Caveat: Like yourself, I picked this stuff up without studying it as part of a degree. Anybody with a proper compiler background want to jump in and enlighten maybe?
Didn't read this one, I don't have much time to read these days. But we definitely need more hands on practical information to counter the avalanche of academic fluff out there, and many seem to enjoy Crafting Interpreters.
Having spent 34 years writing code and around three years full time designing my own languages, most other kinds of code make me fall asleep sitting up these days. Nothing really compares to controlling your world from the outside.
Lately I've been working on a stack machine devkit [0] to use as foundation for new languages, providing convenient access to most of what's needed and an integrated assembler to control it from the outside. The general idea is that you link the library and hook in a custom reader for your syntax, possibly embedded as a separate opcode. It's 5kloc of reasonable MIT-licensed code, small enough to own.
Feel free to have a look and I'll try to answer any questions.
[0] https://github.com/codr7/cidk
Edit: Was it the academic toes? Because I'm just sharing some code I wrote to help people get started designing their own languages. What did you do for others without expecting anything in return lately?
Deleting the 2nd graf would probably change the perception; it doesn't add any insight, and honestly, no one expects you or any commenter to have read the entire book before remarking on it.
The point of the message as a whole and the code I shared was the opposite, we need more of this. I included the second paragraph as a disclaimer, I didn't read the book because I don't read much these days but I've heard great things about it.
Nah, there's just no way to protect yourself against this these days. I did absolutely nothing wrong here. But if you're looking for trouble and something to project your self loathing on, you'll find it. Ask and you shall receive.