klibertp

I would recommend trying something like PyParsing[1] instead. Libraries like this allow you to compose the parser from language-level entities (object and functions, on top of regex and string literals). This means you can attach comments to those entities naturally within the syntax of the language. You also get much better error reporting out of the box, as well as a well-defined way of attaching transforming code to parts of the parser.

There's a place for simple regexes, but complex regex DSLs (with comments and non-significant whitespace, etc.) are almost always less convenient than simply using your language directly.

[1] https://pyparsing-docs.readthedocs.io/en/latest/HowToUsePypa...

While true in principle, writing grammars in regexes is problematic in practice: the syntax for the more advanced features (named submatches, lookahead, backreferences, etc.) is pretty complex, and refactoring the expression means you're working within a string literal, with no help whatsoever from your editor or IDE.

My "go to" solution for parsing (and validating/matching) non-trivial grammars is a library that wraps regexes and allows you to structure the grammar with entities above substrings of a string literal (including arbitrary code for transformations). PyParsing for Python, scala-parser-combinators for Scala, Grammar in Raku, PetitParser in Smalltalk, PEGs in Janet, parser combinators in F#, and so on. These are mostly internal/embedded DSLs, which makes them much easier to use than the typical lexer/parser generators, while giving you all the power to structure and evolve the grammar easily.

For simple grammars, a well-written library adds little overhead over plain regexes. However, grammars rarely stay simple - very often, during the course of development, you find edge cases or the need for extensions. If you started with a structured parser, you're fine: there are specific ways of evolving the grammar, and you can use normal refactoring tools to perform them. If you started with a regex, you quickly end up with a monster regex literal that becomes more brittle and harder to change with each modification.

One important property I look for in parsing libraries is the support for left-recursion. Memoizing/packrat parser generators can handle it gracefully, which is important, because if I'm implementing a published grammar, I want to encode it as closely to the original as possible. For the same reason, I prefer having dedicated tools for associativity and precedence (so that I don't have to invent names for intermediate levels).

TL;DR: yes, regexes are much more expressive than the "regular" in the name would imply, but they still have their limits. For parsing things, it's better to start with something that can work in the simple case fast (so no lex/yacc-style codegen from 2 separate external DSLs), but which also provides enough structure that adding good error handling, extending the grammar, attaching arbitrary code transformations, etc. won't be a big problem later.

> But they generate objectively bad text.

By default, yes. That's exactly the same for code: by default, even with planning and patient nudging towards best practices, you get passable code at best. Not elegant, not performant, and not particularly readable, either. Basically, an uninspired salaryman type of code.

In both domains, you can get much better results in some specific circumstances. Prompt, skills, memory, and the task must align, but when they do, you can get good quality building blocks that you can then work with. It's crucial to recognize the instances where there's a chance of getting better-than-average results and ones where you could dump your whole week of tokens into and still end up with a mess. This is a skill of the LLM operator.

Once get that skill, the only issue is capitalizing on it: basically, how you fit handling of the generated building blocks into your workflow. If it's seamless, it can be a big win. If it's not - yeah, writing by hand from scratch is often better.

10x or 60x of very little is still not much. "Working programmers" unable to write FizzBuzz exist.

10x or 60x of a negative value is a 10-60x bigger problem than that initial negative value. We all regularly commit things that are actually net negative.

Finally, 10x or 60x more productivity channeled into yet another static site generator, CMS, or programming language doesn't really help in improving products or developing new interesting features.

In short: LLMs offer real productivity gains, but they are still tools, and they are very new, so we misuse them a lot, in various ways (not limited to examples above). Unless they replace programmers and can consistently produce better than average code (even if they fail catastrophically in a few cases), they won't fundamentally change programming. They did change a lot, but not enough to get the result you're describing.

> Some times the true nature of the problem is revealed while implementing it and by deferring everything to an LLM you spend days throwing tokens at the wrong thing.

If you look at the code the LLM spat out (and you really, really should!) you will immediately notice that its shape is not what you thought it should be. You might not notice immediately if you're learning, but if you really "mastered" the domain, you will "just" see it. You will also recognize the problem with your assumptions, and immediately (or after some research) correct the prompt.

Looking at the diffs for everything the LLM does slows you down, of course. Not looking - or looking and not recognizing problems, for one reason or another - can be initially faster, but a single pathological case can eat both the time saving and tokens. For domains you truly "mastered", the current models can generate code as if they read your mind (because you can be that precise in the prompt, and quickly), so it's really glaring and very hard to miss when they somehow misread your mind.

It only works at the "mastered" / "unconscious competence" stages, and only in those narrow domains you truly mastered, but it does seem to work. Is it 10x? No idea, but there is a marked change in the speed boost when crossing from conscious to unconscious competence area, with everything else (harness, model) staying the same.

GToolkit[1] - a Smalltalk environment that's based on Pharo but replaces the UI framework and some other parts of the stack. It uses Rust through FFI to interface with and bundle native dependencies.

I don't remember the exact numbers, but when I checked, the GT+Pharo ecosystem (available packages, number of people in Discord, tools with support for the language, etc.) was ~2x smaller than Common Lisp's. It also comes with its own problems, some of which CL doesn't suffer from (the GIL, performance, startup time). But it's very fun to use and play with, which is the most important quality for me in my hobby/side-projects. :)

[1] https://gtoolkit.com/

The only language with Fexprs that I used was Io. In Io, unevaluated code is represented as a tree of Message nodes, and for each call, an activation record is instantiated and provided to the called method body. That reified Call object lets you access the caller's environment and the raw Message chains passed as arguments. You can then decide whether to evaluate those messages, which ones, how many times, and in what context/environment. It's a very niche language, but if you want to see Fexprs "in action", that's the best place to look at (at least to my knowledge) - they are central to Io's design rather than an advanced feature nobody touches. It's even mentioned on the front page[1]:

> Messages as Code — Messages form trees that can be inspected and rewritten at runtime. Argument evaluation can be deferred, so if, while, and for are implementable in Io itself.

[1] https://iolanguage.org/

Good point. I was thinking about meta-interpreters (and it's still possible I misunderstood or misremembered how they work!), but you're right; I should have at least been more precise.

I don't think that moving the evaluation to compile time is something you can just ignore when discussing macros. It's an important differentiator between macros and Fexprs[1].

IME, the primary use case for macros in Lisps is language extension, a step before a DSL. For every `loop` there are thousands of `with-x` macro implementations. See the macro-writing macros in Alexandria/Serapeum for the most common patterns.

Also, some dynamic languages (Io, Prolog, Groovy) support DSLs through non-macro mechanisms. DSLs in those languages are as easy (or easier) to implement as in Lisps; however, they tend to be less performant. So, again, compile-time macro execution does matter for the DSL use case, too.

[1] https://en.wikipedia.org/wiki/Fexpr

If your language has macros, you don't need to wait for it to gain async/await (and it was quite a wait even in Python and JS, while Java got its virtual threads very recently); you can just[1] implement a CPS transform as a macro:

https://github.com/nim-works/cps/blob/master/docs/cps.svg

[1] Yeah, I know, this word is extremely overworked in that sentence. Sorry, that's an exaggeration for effect. In reality, it's not that simple, and not always possible: it depends on the language and macro system. It's still more likely to be implementable in library code if there is a macro system, though.

IIRC, the criticism is of syntax-rules and syntax-case kind of macros in Scheme; I don't recall (happy to be proved wrong) LoL directly mentioning syntax-parse that's specific to Racket. syntax-parse is no less powerful than defmacro; it's just (much) more structured and featureful. It's a difference between hand-rolling a recursive-descent parser and using a (really robust) PEG library.

Racket's define-syntax/syntax-parse has escape hatches into procedural generation (through which you can easily break the hygiene), and the equivalent of CL's &environment contains more information and allows more operations in Racket. Racket also supports reader extensions but often doesn't need them because it has a macro-based API for module definitions (a feature completely absent from, or even alien to, Common Lisp; OTOH, Elixir borrows some of Racket's ideas in this space).

For a practical example, Coalton and Typed Racket implementations show that both systems are capable of large-scale language extensions. I never studied their respective codebases, so I can't say anything about relative ease of introducing those extensions, but they are possible in both cases, at least.

On the contrary, CLI, TUI, and desktop GUI apps are basically the only kinds of apps that benefit from CL and can live with its shortcomings. The startup time of a tool written in CL is short: you just need to load the image into RAM, run some hooks (if configured), and you're good to go. If you don't include loads of dependencies, the dumped image is also not too big, so loading it from an SSD is almost instantaneous. The startup is of course nowhere near that of a small C or Zig binary, but for larger tools it will be tolerable. For TUIs and GUIs, you can work on them interactively and see the changes in the source immediately reflected in the interface of a running instance.

Web services need either cooperative concurrency or M:N concurrency due to the "10k problem". CL only supports threads (OS-level) and promises; everything else is incomplete (eg., delimited continuations, which could be used to build coroutines) due to missing parts in the spec and some language features (eg., conditions and restarts). Of course it can be done, but it won't be as pleasant as using Elixir and Phoenix.

A game engine would work, most likely, unless it was for an MMO (again, concurrency handling). I personally never worked on one, but I see examples of game engines in CL, and they tend to look nice.

In general, single-user desktop (CLI, TUI, GUI) apps are still a good fit for CL, even today (you need to put some work into packaging the app for different platforms, but it tends to be easier to set up than it is for C or C++; harder than Go or Rust, though). It's unfortunately not as good a fit for the backend, at least not until an implementation with good support for concurrency appears. It's nice as an extension language in a larger app (through ECL), and as far as dynamic languages go, it's quite performant, so some computation-heavy apps can benefit from using CL (with SBCL). On the other hand, the ecosystem is quite small, which means dependency-heavy apps are better written in something like Python or a mixture of CL and another language (there are two-way bindings to many dynamic languages and there's mature FFI support for compiled languages).

To be perfectly honest: as much as I love Lisp, I personally gave up on trying to use it, for now. For hobby stuff, I found an even more niche solution that is more enjoyable to work with in the GUI/TUI/CLI space. It also doesn't support OS-level threads, but instead provides coroutines for concurrency - I find this side of the trade-off to be useful/beneficial more often, at least in the code I tend to write. I don't believe the time spent learning CL and other Lisps was wasted, but it's become harder and harder to justify going for CL over the past 15 years, and I finally reached a point where I stopped trying. YMMV though, and I would still give CL a chance if it's your first language of this kind (i.e., providing image-based interactive development, a dynamic language with a native compiler with inline assembly support, a multimethod-based object system, and homoiconicity/macros, etc.).

I suspect it's a bit like tree-shaking/dead-code elimination in an untyped, dynamic language. You can probably delete a lot of things, but each deletion has a chance to break (or silently degrade) some functionality. Good coding practices, good docs and comments, and good tests can help determine whether a given definition is safe to delete or quickly verify that the deletion was a mistake after the fact. LLMs' weights lack all of those. It's like trying to perform DCE on a binary linked from outputs of ten different compilers with different optimization levels (and the linker is outright bonkers) without running the binary once. Oh, and the binary is many tens of GB in size. Yikes, I'm getting goosebumps just imagining it!

> western colonies fared much better than eastern ones.

Sure, Iraq and Afghanistan definitely benefited a lot from being bombed, occupied, and then handed over to even worse tyrants when the West got bored.

No, sorry, but "capitalism and democracy" is not the right answer everywhere, and when they're not, pushing them by force is no better than forcing communism. In general, forcing an incompatible ideology on people historically and culturally opposed to it will end in tragedy, no matter how great that ideology may be.

> In that regard Lepiter (and GT by extension) seems immensely powerful

Yes - I think this all comes from the insistence on using a single rendering tree for everything displayed on screen. It allows mixing all kinds of UI elements everywhere, including nesting of arbitrary widgets inside a text field (and arbitrary text inside any other widget). It's a genuinely powerful model for UI composition :)

> I can't summon spell-checking willy-nilly like I can in Emacs

Yes, that's why I mentioned "features I use regularly in Emacs which are (and will be, for a long time) missing in GT" - it's not that you can't build them; they just need to be built, and with how few developers work on GT, it'll take ages before GT Coder gets those features.

It doesn't help that GT editors seem a bit less ad hoc scriptable than Emacs. I'm not sure why this seems to be the case for me - I suspect the need to structure everything as classes and methods adds non-trivial cognitive overhead for simple scripting. At least compared to the pretty flat, procedural shape of typical Elisp code. Then again, I did not invest enough time into learning how to customize GT's editors, so it might be a shallow impression that would go away with time.

> Also there seem to be official examples

Oh, nice, thanks, I missed that. I mean, it's obviously possible to implement an LSP client for GT, but I didn't know one existed. I'm especially interested in how that is integrated into Coder, since it should be able to provide a lot of functionality that I'd also want to script in my custom Coder instances.

> you already think we are odious

I had to look up the word to double-check as a non-native English speaker. A quick definition I'm working with is "Arousing or meriting strong dislike, aversion, or intense displeasure." The synonyms assumed are: evil, cruel, hostile, nasty, wicked (all from Grammarly).

The experience of "you" - Americans - for non-Americans is shaped by cultural exports, personal contacts, and interactions with available American institutions. The last of these is likely to start with the immigration policy and its execution in the form of an embassy visit to apply for a visa and later with the officers at the airport. How confident are you that these interactions cannot be described as "odious" in a significant percentage (maybe even the majority) of cases? I've never experienced it myself, but in my social circle, almost all who tried were pretty bitter about it, even when successful. There were some outright Kafkaesque situations as well. Does it come as a surprise that a completely ruined "first impression" ended up biasing them to some extent?

I think that's rather expected. I'm pretty sure the same is true for a portion of people who want to visit or immigrate to my country. I can hope it's maybe less prevalent - we're not as sure of our greatness here (still better than those others around, obviously! /s) - but I wouldn't count on it to be an entirely positive experience, either. Admitting this and trying to fix it (including, unfortunately, convincing that dude who had to deal with his fair share of assholes to get here that not everybody is an asshole) would probably be a better course of action than isolationism you seem to favor, but to each their own.

I might be wrong - I investigated Pike just once, a long time ago - but I think Pike dropped a lot of the "liveness" that LPC provided (ie. reloading objects from a (changed) source at runtime while keeping their identity and updating clones). It's still probably a good fit for a lot of more advanced MUDs, but it's even more niche than CL and Smalltalk, so again, not much of an advantage over a home-grown solution. Plus, I've been recently looking more closely at MOOs, and for that, even LPC would not be enough (too much of a separation between code and data) - LPC/LPMUDs weren't really built for persistence as much as MOOs were.

Still, I can't disagree that, out of the box, LPC and Pike are probably a (much) better fit for a mudlib language compared to Lua, Python, or JavaScript :)

Almost no languages are built for the environment where all code and values must be live and persistent and accessible to many people at once. Sandboxing and permissions need to be much more granular than what you get in most implementations, and persisting complex state (nothing in-world can disappear or break just because a server is restarted) is hard if that state is not first-class in a language (imagine a half-consumed generator in JS). There's more, but in short: the requirements this environment enforces on the "mudlib" language are unique enough that no mainstream language is a good fit out of the box.

Roguelikes have had 2D UIs since the start (just rendered with characters), while a MUD with a 2D interface is not a MUD but an MMORPG. There's a bit of a middle ground in rich MUD clients that can reorganize the 1D stream of words visually into a dashboard, but that's as far as you can go while still calling it a MUD.

MUDs (and closely related single-user text adventure games) are a well-defined genre; you can improve various aspects of the presentation, add rich formatting, images, background music even, but the moment you stop using text as a main medium for gameplay, you're simply not doing a MUD anymore.

> Modern MUDs and similar will just include a mature script language, like LUA

Not necessarily. The persistent, live nature of some types of MUD programming (LPMUDs, MOOs) is unique enough that many typical scripting languages are not a good fit. The only general-purpose languages that come close to the required execution model are Common Lisp and Smalltalk (and Prolog, if you squint hard enough), which are so unpopular that you don't really get much out of choosing them over a custom language. That's one of the reasons so many MUDs developed their own execution models and language dialects.

Writing a transpiler for a large MOO code or LPC codebase would require generating tons of boilerplate to emulate the unique semantics of the languages, as well as providing a runtime with support for those features. It's not that easy.

For some MUDs, scripting with Lua is more than enough. There's a book on writing MUD engines[1], and the second half shows exactly this setup (I think C++ for the engine and Lua for scripting). It will work in many cases, but it'll break down if you try to make the MUD easily programmable by users. It can still be done, but at some point, it might be easier to start from scratch rather than trying to make a stock Lua or Python interpreter work in that environment.

[1] https://www.amazon.com/MUD-Game-Programming-Development/dp/1...

EDIT: Argh, I misunderstood your point: you'd like to use GT as an IDE for non-Smalltalk code, right? I took "LSP integration" to mean "GT providing LSP server implementation", while you meant "GT implementing an LSP client", I think.

That's not something I ever thought I needed, though there is some support for syntax highlighting for JavaScript and Python, I think. I don't think GT is supposed to be a general-purpose, polyglot IDE: it's a bit more specialized than that, IIUC.

I'm leaving the off-topic rant below; it's a bit too long to just delete :(

> Please correct me if I'm wrong and things changed in the meantime.

Sadly, you're not wrong, and as an Emacs user (since 2012) myself, I share your reservations. The text editing widgets in GToolkit are usable, but nowhere near what you get in Emacs. Multiple cursors, iedit, re-builder/visual-regex, editable occur, AST-based editing (paredit/combobulate) are just some of the features I use regularly in Emacs which are (and will be, for a long time) missing in GT. On the flip side, rich formatting, inline images, and interactive widgets are way easier to implement in GT than in Emacs, and Lepiter is a good alternative to Org.

The conventional answer to that is that you shouldn't need those more advanced editing features when working with Smalltalk: the method bodies should be short, you should be using refactoring commands instead of hand-editing code in many places at once, and you should just auto-format after each edit. Even if it's not technically incorrect, I don't like that answer. Because this is the Smalltalkers' mindset (on average), things like LSP implementation (or other support) for external editors[1] never gained traction.

Without writing an $EDITOR-GT wiring yourself, the best you can currently do is some level of external integration. You can add a context menu option like "Edit in Emacs" in GT that will run emacsclient on a temporary file with the method body dumped. It's obviously underwhelming: you have either a powerful editor (Emacs), or a smart editor (GT), and you have to deal with two completely different UIs. Yes, it's bad. It's still better than trying to use just GT for all editing, though.

I ended up structuring my GT project a bit like a Web app. The sources on disk are a source of truth, and coding agents work with them. I have scripts that load the Git repo into a fresh image and either run the tests, run a one-time eval, or open the GT GUI (equivalent of loading a page and opening DevTools in the browser). In GT, I have an Emacs escape hatch for editing, and I use the Git tool to export any changes back to disk. So far, it works OK, though it's closer to how you work with Common Lisp images than traditional Smalltalk "100% in-image" development. It's not ideal, but it allows me to leverage GT while still integrating other tools into the project.

[1] There were attempts in the past; for Emacs, somebody wrote a frontend to Smalltalks called `shampoo`. It's been 14 years since the last commit.

> A git commit [ideally] gives you the reason for the change

FTFY ;) I broadly agree, but it's important to note that, in practice, Git commits are hit and miss: they might save you a lot of time if they are well-structured and described properly, but they might also completely fail to give you anything of value (for the problem you're facing at the moment). Turning Git history into a broadly useful artifact requires a lot of discipline from everyone working on a codebase over its lifetime.

It's never totally useless, but it's often less helpful than it could be.

Recently, I turned to JJ for this exact reason: crafting a meaningful change history is much easier there, especially if you need to retrofit it after you're done with the implementation. jj's split, squash, and describe are a joy to work with, also because the interactive version lets you easily split or squash hunks (selected changes in the same file).

I would ignore it if I spotted it once. This is the second time, though, so I probably should point it out: 5 years is a very short time in terms of personal experience and development. While the whole industry moves very fast, it does so thanks to slow, grinding effort parallelized over hundreds of thousands of individuals. For an organization, 5 years is an era. For individuals, it's just 3-6 projects (+/- a few, depending on the context you work in).

TL;DR: "never have encountered in 5 years" is a meaningless data point.

My professional career can legally drink beer since a few years back, yet I'm still regularly hitting "first times" on various things in my day job; I'm also frequently mind-blown reading about others' experiences in domains I never touched. Don't make those "5 years" into a badge to hide behind; use it as a springboard to reach higher.

In this case, try to actively think of uses for the commit history. It's already there; there's an expectation (admittedly, to various degrees depending on context) that it will be maintained. Try to maximize the value you get out of it. Underutilizing a tool you have access to may not be that much of a problem on average, but in some specific circumstances can make a difference between being able to complete a task at all or not. You shouldn't rely on luck to never land in a situation like that: familiarizing yourself with as many potential uses for a tool as possible gives you a better chance of handling it more easily.

EDIT: formatting, last paragraph.

> The Smalltalk approach doesn't really integrate well with anything else.

That's not true. GToolkit (built on Pharo Smalltalk) integrates tightly with Git for VCS and Rust via FFI for external libraries. It integrates with external Python and JavaScript through IPC. The same IPC mechanism is used to work around the GIL (the same way Python used to do with the multiprocessing library).

Squeak descendants are also not the only Smalltalks available. Smalltalk/X is polyglot, supports Java and C code inline, and has a native AOT compiler (through C). Smalltalk/X and VisualWorks have built-in version control, namespaces, and packages with visibility modifiers.

TL;DR: a live Smalltalk environment doesn't have to be an isolated island. The most popular open-source Smalltalks come from education and hobbyist coding, so they tend to miss a lot of "programming in the large" conveniences, but that's just a historical accident. There's no technical reason why a Smalltalk that's integrated into a wider ecosystem and suited for large teams couldn't exist.

> building alternatives and nobody used them

Right. I kind of forgot that MUDs are supposed to be social. Seeing as the popularity of MUDs plummeted, I also changed my goals over time: I want to build a persistent, text-based (but with rich formatting support, including eventually GUI widgets) live programmable environment with network transparency and multiple concurrent sessions allowed. Since it's personal/local space first, and only allows others to "visit" (and even that is mostly for my own remote sessions), I don't need many users initially. If I was trying to build something that emphasizes the multi-user aspect, the choice of continuing one of the historical lineages would be much harder to avoid.

> Secondly, existing "mainstream" languages are not appropriate, mostly. None of them or their runtimes are really based on a properly sandboxed persistent model.

You're right. And even if you broaden the search so much that "mainstream" is barely visible from the cut-off point, you're left with basically just Smalltalk. My current attempt is based on GToolkit distribution of Pharo Smalltalk - it does check some boxes, but it's class-based (not a big problem given the dynamism) and VERY single-user by design (quite a big problem, and thinking about eventual security/permissions/capabilities handling gives me a headache). Still, from all the prototypes I made over the years, this attempt went the farthest; I think I'll stick with it for a while longer.

> Finally, I have all sorts of ideas on what I think is "better" than MOO

Thank you, I will definitely check it out :)

> But the TLDR is that one of the arguments you pass to the system at start is basically a source directory of ("objdef") human readable / editable source files, and that gets compiled into a persistent binary database.

Thanks. That's exactly how a Smalltalk image is built, too, at least with the tooling I'm using (GT). You start an image, point it to a set of Git repos with Smalltalk code, which is then loaded into the image, classes are created, and methods get compiled. You then dump the new image. When you start it the next time, it already contains all the code, and you can change it inside the image at runtime, as much as you want. At some point, you export the changed code back to Git repos, and regenerate the image. Since the image contains all the live objects as well, you can close it, and all the state is restored when you reopen it.

Thanks for the Discord invite, I just joined. I'll go over the book and try to run mooR and mica first, and will reach out for help if needed.

For mica specifically: I'm not 100% sure, but I seem to remember you describing the idea behind it much earlier (here on HN) than this spring. I think it was in comments under a Python-Prolog bridge or something like that. Good to see you had the time and motivation to implement it :) I'll definitely give it a read!

Very cool! Why reimplement the "moo code", though? Is it "just" for backward compatibility, or did you determine that none of the existing languages can be easily modified to live in a MOO environment?

I started playing on an LP MUD in the late '90s. Over the past 20 years, I tried a few times to implement a similar environment, only with everyone being a "wizard". LPC was also a prototype-based OO language with multiple inheritance, and it supported live development, with code for objects stored in files and dynamically reloadable. I would always hit some kind of blocker, no matter the language I chose for the implementation. It looks like MOOs did (and do) what I wanted, but I just wasn't aware of their existence. :( Maybe if I used MOOs as a model, I'd get better results... but I'm not sure if I want to go all in on DB. I tried looking up how the code is loaded into the DB in the book (https://timbran.org/book), but couldn't find it. Is there an explanation somewhere of how MOOs are bootstrapped from sources, how you can maintain sources in files on disk, and how new definitions are pulled into the DB at runtime?

It tends to be. While it does use a GC, it has both bytecode and native AOT compilers. When compiled to native, it's closer in performance to C++ and Rust than to C# or Java. It's a reasonable middle ground in PL design that Nim also tries to occupy. In recent years, OCaml's runtime also shed the GIL, making it a good solution for parallelizable, CPU-bound tasks. As a language, OCaml is one of the most impressive ones (in terms of features and capabilities), yet it still provides strong, practical tooling, good performance, and access to low-level-ish features.

TL;DR: OCaml is usually going to be a bit slower than C++, but faster than Java, while rivaling Rust and Haskell in terms of expressive power.

Caveat: a lot of the power comes from the tooling. Last I checked, syntax extensions were external preprocessors, just with standardized APIs. And you need those extensions for things that are handled by (in-language) macros in Rust (like deriving string representation for a record). Without the tooling, OCaml can get a bit tedious/boilerplate-y. OTOH, after a period of instability and competing solutions, the tooling mostly settled down, and enabling the important parts is often a single line in the config. It's not a situation unique to OCaml, but if you look at the language docs only, you'll miss half of what OCaml development normally offers.

> They all have failed.

That's not true. Most have failed, but those who used the right tool for the job - a rich, static type system in a functional language - did succeed. It's just that such type systems are rare, and even if nominally a type system is good enough, the required boilerplate might be uneconomical to maintain. Scala and F# are probably the only two languages that are mainstream-adjacent, at least, and have type systems expressive enough that using them to track escaping is not an absolute hassle. And they're both tiny in terms of the number of users.

In the end, we did settle on APIs that hide the escaping process (prepared statements, innerText vs. innerHTML, etc.) just because it's a) good enough; and b) possible to implement more or less uniformly across the TIOBE Top 20. That's good, but if you happen to use a language that's powerful enough, you probably also should track the escaped/unescaped status in the type system - it can be a cheap, additional safety net.

Somebody checked[1], and they discovered that the "stages" past the first had minimal impact on the unsafe blocks count. That's just one metric, but it makes me doubt your claim that there was/is a lot of work put into making the translated code idiomatic Rust.

[1] https://news.ycombinator.com/item?id=48967630

Sure. But you know, from personal experience, you can be a happy BASIC user writing simple games without knowing about the existence of arrays. It's a bit hard, but nothing copy & paste couldn't defeat. And while learning about `DIM` was quite a breakthrough for my 8 y.o. self, I'm not sure I'd welcome an article about it on HN's front page.

The same is true for iterator protocols: as noted in my other comment, they have existed since 1975 in almost the same form (with a few closely related variants). It's not easy to find a mainstream language that doesn't support them "since forever" (in "tech years"). They're just not news, no matter how you slice it.