Hacker News Reader

> There is always a need for two types of languages, higher level domain languages and general purpose languages.

I never suggested otherwise, just that when you're in a domain you should be in that domain. That solution requires something more general purpose to glue domains together, which is the crux of the problem. What does such a language look like? How do you ensure you don't lose all the good properties you gain from the domain specific languages/editors when passing between them?

I think you present a false dichotomy though. General purpose languages are just as much about encoding a process. The distinction between compiling to the machine vs some abstract machine also isn't really relevant: this is about semantics, not implementation. And if you let implementation dictate the semantics you won't get very far from where we are now.

> What I strongly disagree with is claiming that the complexities that plague general purpose languages are somehow mitigated by building more domain specific languages.

I never said that :) I said that programming would be greatly improved by being observable, direct, and incidentally simple. And again those have nothing to do with what "level" you're programming at, they're just principles to apply. I do think there is a general solution that can encompass most of the levels (though I'm not interested in trying to do that any time soon), but there is a common case here and it certainly isn't high level theorem proving or embedded systems. It's stupidly simple automation tasks, or forms over data apps, or business workflows. The world works on poorly written excel spreadsheets and balls of Java mud. You don't have to fix everything to make a huge impact and the things we learn in doing so can help us push everything else forward too.

> How do you ensure you don't lose all the good properties you gain from the domain specific languages/editors when passing between them?

That's a very interesting (and I'm betting hard to solve) problem! However, it's very hard for me to see how Aurora would help with that. From the demo, it looks like yet another visual programming system; such systems don't seem particularly interoperable.

The clever part of the strategy I was using in that demo was that all domains are expressed declaratively as datastructures. This meant that the glue language only needed to be very good at manipulating vectors and maps. You built up a structure that represented music, html, or whatever and then just labeled it as such. Interop between domains then becomes pretty simplistic data transformation from one domain's format to another. And given how constrained the glue language could be, you could build incredibly powerful tools that make that easy. You could literally template out the structure you want and just drag/drop things in, fix a few cases that we maybe get wrong and you're done - you've translated tweets into music.

We ended up abandoning that path for now as there are some aspects of functional programming that prove pretty hard to teach people about and seem largely incidental.

Here's some meat. So how does FP fall down?

Explicitly managing hierarchical data structures leads to a lot of code that isn't directly related to the problem at hand. A lot of attention is dedicated to finding the correct place to put your data. Compared to eg relational or graph data models, where that kind of denormalisation is understood to be an optimisation made at the expense of program clarity / flexibility.

The pervasive use of ordering in functional programming inhibits composition. (Even in lazy languages the order of function application is important). Compare to eg Glitch or Bloom where different pieces of functionality can be combined without regard for order of execution/application. This better enables the ideals that BOT was reaching for - programming via composition of behaviour. In a BOT plugin you can not only add behaviour but remove/override other behaviours which turns out to be very valuable for flexible modification of Light Table.

A more concrete problem is displaying and explaining nested scope, closures and shadowing. As a programmer I have internalised those ideas to the point that they seem obvious and natural but when we showed our prototypes to people it was an endless source of confusion.

Functional programming is certainly a good model for expressing computation but for a glue language the hard problems are coordination and state management. We're now leaning more towards the ideals in functional-relational programming where reactivity, coordination and state management are handled by a data-centric glue language while computation is handed off to some other partner language.

> A more concrete problem is displaying and explaining nested scope, closures and shadowing.

That's what really killed it for me and also one of the things that I found pretty surprising. Tracking references is apparently way harder than I realized. And while I thought we could come up with a decent way to do it, it really did just confuse people. I tried a bunch of different strategies, from names, to boxes that follow you around, to dataflow graphs. None of them seemed to be good enough.

OK, you don't need to convince me! I've been saying for years that the central problem is coordinating state updates, and have been proposing that languages should automatically manage time for us like the way they automatically manage memory. Sean McDirmid and I have submitted a paper on it to Onward.

But as to graphs vs trees I don't think it is so clear-cut. Good arguments on both sides of that issue.

> Let's not forget that "programming" runs the whole gamut from embedded systems programming in assembly all the way to very high level theorem proving in Coq and understanding anything about the nature of that entire spectrum is difficult indeed.

True, but one of the problems which Bret Victor and Chris Granger set out to solve with LightTable (and is mentioned here in §"Programming is Unobservable" and §"Programming is Indirect") is that the tooling for using current programming languages hasn't meaningfully changed since the 70s or 80s.

I agree that generalising over all programming languages is near-impossible, but even the most 'bare-metal' languages only manipulate models of the computer hardware.

> they're complex because the problems they solve are genuinely hard and not because we've artificially made them that way.

I have a ServiceControllerServiceProvider which disagrees.