Hacker News Reader

Comment by lispm | Hacker News Reader

SBCL has an interactive/incremental/in-memory native code compiler and a file compiler.

https://www.sbcl.org/

Lisp seems like the most incredibly obvious choice here.

Looking at the list of requirements: 0. Only 1-2 years of dev effort to implement. 1. Predictable, explainable performance. 2. A small, embedabble runtime. 3. Support for repl / incremental programming (ie adding one function shouldn't require stopping the process and re-compiling the entire project). 4. Support for debugging and profiling.

Lisp satisfies 1, 3 and 4 I’m pretty sure, and maybe 2 depending on your definition of small. Idk about 1 though.

I glossed over this the first time around, and it seems like others did too, but once you add the following constraint, it takes many languages off the table -- Java, basically all Lisps, Julia, OCaml, etc.

> Ability to use static dispatch, static field offsets and stack allocation.

> Control over memory layout - at minimum being able to express arrays-of-structs without pointer soup.

---

So reading between the lines a bit, I believe the author is asking for an REPL'd, interpreted, and fast running Go, Swift, Rust, C++, or Zig. (which to me is an interesting problem -- I basically use shell as my REPL for C++)

OCaml with new value types would qualify, and Java with value types would qualify, but they're not there yet AFAIK.

I think Julia/Clasp can get that kind of performance in some cases, but it's not predictable performance, with language control.

(copy of lobste.rs comment)

How much of these things make an interactive language difficult? Many languages can be used interactively, but only in very simple ways of being interactive. To be truly interactive one also needs dynamic behavior. Let's say we define a struct with fixed fields (which Lisp can) and we want to interactively redefine the thing. If we want to stay in our interactive flow the runtime needs additional changes. It needs to deal with existing and future allocated structures. It needs to deal with inlined accessors etc. One of the simple ways to deal with that is to kill the program and start over. But that is not 'interactive'. Static types also get in the way of dynamic behavior.

That's why the interactive versions of languages like C, Rust, Haskell typically are not very interactive and come with a lot of compromises. Often compromises at the boundary of compiled and 'interpreted' code.

What does 'interpreted' mean and why would we actually need it? Lisp as its bases has a radical interpreter: it interprets source code in the form of s-expressions. Most people think of interpreted code "byte code interpreters" - that's an entirely different form. The source code gets compiled to a byte code interpreter. In Lisp its quite often that one uses incremental native code compilers. These achieve much faster running code, in a spectrum of possible optimizations: optional type declarations, reduced runtime safety, code inlining, stack allocation, type specialized code generation and more. Many of these optimizations make the code harder to write. SBCL by default always uses the native code compiler. If we want "byte code" interpreted code, there is no way to make it fast without either some form of native code compilation (typically a JIT compiler) or writing the code in some other language (Python + C). The alternative is to get rid of byte code and having a native code compiler, with the drawbacks: one or more hardware architectures to support.

> Control over memory layout - at minimum being able to express arrays-of-structs without pointer soup.

That's a problem. Lisp needs to deal with it when interfacing 'foreign' memory. Thus it typically has a foreign-function interface.

> I believe the author is asking for an REPL'd, interpreted, and fast running Go, Swift, Rust, C++, or Zig. (which to me is an interesting problem -- I basically use shell as my REPL for C++)

There are compromises to make. "Interpreted" already is vague, see above. "Fast running" - is it always "fast running"? Is that possible? While being "interactive" / "interpreted"?

"REPL" is another concept with a range of possible meanings. Is a Read Eval Print Loop like in Lisp integrated into the language (read, print, eval, loop are primitives in the language) and is it the base for interactive programming. Is it a glorified command line interpreter which basically can call functions on data?

I agree with you, the language design implications are just as interesting / important. What I imagine the problem statement as is to make a definition like this:

    // layout.h
    struct MyRecord { ... }
    #define N 1024*1024
    MyRecord myarray[N];

And then suppose you have 1000 translation units that #include layout.h

So whenever you change N, you have to recompile a lot of code.

And then you want to compile myprog quickly, and run the following quickly:

    cat 2gigs.txt | ./myprog

It's an interesting problem, but I think from the perspective of an interactive language, it does make sense to have some kind of fast mode and slow mode ... the classic debug/release build split is precisely that. I would settle for an interactive C++/Rust, with a good incremental build system, where you can choose the build variant!

C++ build systems suck, and Rust often has to interface with them. And both C++ and Rust are not designed for fast interactive use. They have a lot of syntax that allows the control over memory, and that's perhaps a bigger problem.

I also don't really like typing code into a REPL -- I use an editor and a shell (and Ninja, which turns out to be crucial for fast C++ dev).

Reply on news.ycombinator.com