The Economics of Programming Languages [video]
youtube.com
youtube.com
Case Study: Zig Software Foundation by Andrew Kelley
https://archive.org/details/fossy2023_Case_Study_Zig_Softwar...
Also zig is cool and thank you for the technical and gubernatorial guidance you've given to other language authors all over.
Imagine how amazing our languages and tooling could be if there was a happy path for talented people to work on this.
For example, the error messages in Elm are just insanely better than in any other language I know. Like, it's not even in the same ballpark. Why doesn't every language have this?
But that's just an example. In general, I think we under-invest in tooling.
I think this has led to many modern open innovations only realistically being useful for solving the problems of very large companies.
On a cynical day I might even accuse certain tools of being designed that way for job security. Or at least it looks that way because the majority of people who are using the tool could only dream of getting approval from management to make a better alternative.
The priorities of many of the tool maintainers, no matter how honest, just don’t match like they used to, and this is in an ecosystem awash with telemetry.
[1]: https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-com...
And in most cases, it's just a matter of sitting down and really thinking about what is the best error message you could provide in that specific scenario, what context is needed any so on. Instead of just writing down the first thing that comes to your mind, and never iterating on it ever again.
It's a combination of economics (who pays for the work? It could be people scratching an itch, but these people also have to have the privilege of being able to pay their bills somehow), difficulty (it's not a trivial problem space, you need to turn your compiler into one that understands not only its own language but also every potential language people try, and the errors might crop up in unexpected places where the context you need to provide good errors is long gone, so you must change the compiler to a bowl of spaghetti design, so you can access, for example, the type checker during parsing) and inertia (developers are used to building tools for themselves, so they dont spend time on diagnostics because they are already familiar with the inner workings od the tool, then other developers use those tools and because every other tool is like that they don't expect good errors, so no one considers them important enough, and the cycle goes on and on).
But let me be clear on one thing: although there are many things in your language design and compiler implementation that you can do to make things easier on yourself (don't throw anything away related to the causal relationship for why a certain constraint is present!), the work is mostly elbow grease, and spit and polish, little more.
The downside is the nature of LSP, which is to say it is necessarily a common denominator API. As a result a tool built on LSP-based API tends to be significantly inferior compared with similar tools built on comprehensive, proprietary APIs. For instance, the IntelliJ IDEA Open API illustrates this advantage quite well.
Thus, the economically rich tool authors maintain a clear advantage _if dev experience is the key differentiator_.
Utilizing the vernacular put forth in the talk, the sophistication of the IJ API is both the means and the cost of establishing a presence on the IntelliJ-dominant Language Tooling Avenue.
Jokes aside, that’s a really interesting use case. I’ve often wondered how laws, and changes to them over time, are recorded. It seems like the kind of thing that could be made more efficient with a formal syntax along with version control.
LLVM is one. That's pretty successful.
The JVM is another. That's very successful, there are lots of languages that build on the JVM to reduce their tooling costs. Java, Kotlin and Clojure are examples of that.
GraalVM is another. If you use their Truffle framework then you get JIT compilation, debugging, tracing, instrumentation, different kinds of interpreters, advanced strings, some LSP support, language interop and FFIs and more.
But what you see, generally, language authors tend to start by hacking together a compiler and standard library starting from a C hello world program. Not that many survey the landscape and look for ways to reduce the costs.
This was a problem in writing an LLVM backend for both the Glasgow Haskell Compiler and the Standard ML of New Jersey Compiler. It seems like it was easier for the GHC implementers, because they were already able to compile their semantic IR (the spineless tagless G-machine) into C--/Cmm, which conceptually isn't that different from LLVM IR. The SML implementers compiled to the continuation passing style, which is harder to map to LLVM IR. However, the GHC implementers still had to extend LLVM with a custom calling convention, so it was still challenging.
Paper on the LLVM backend for GHC Haskell: https://llvm.org/pubs/2010-09-HASKELLSYM-LLVM-GHC.pdf
Paper on the LLVM backend for SML/NJ: https://people.cs.uchicago.edu/~jhr/papers/2020/ifl-smlnj-ll...
I think I get the Clojure guy now.
I used to use Elm, but after all of their hostility to their own users (which you can see with a cursory search online) and the growing popularity of TypeScript, I stopped using it, even though TS is in some ways worse. But sometimes, worse is better, and sometimes, you just want to get things done.
The one that's most talked about is the top one as of this writing, Why I'm Leaving Elm. You'll find some examples in there.
- the compiler detects whether it's compiling core libraries and disables some features if it's not
- you can't choose your own package set, even locally
Ideally Elm core development should have kept up to make up for these restrictions, but it didn't. Yet, IMHO, it was still the right choice.
My spiciest take for 2023 is that the development community is going to start waking up to the realities of asymmetry between the kind of open source models made possible by the largess of massive billion dollar corporations and the kind of open source made possible by individual developers working on passion projects.
It probably helps that I am aligned with what Evan is saying in some meta sense. We really need to zoom out from the viewpoint of being the benefactors of seemingly "open" projects like Chrome. I even see in these comments some pot-shots at the Elm community and leadership for being extremely protective of their project when faced with unwanted and undesirable intrusions.
It is also interesting that this comes not long after I ran into Golang debating an updated json library for its stdlib [1]. What I want to say about that is, there is almost no universe I would be willing to put up with that community feedback. Any human with the stamina to respond to aggressive and persistent haranguing and nitpicking has more patience than I will ever have. So if the majority of criticism of Elm is that they exclude those kinds of people from their community then I totally understand it. Even if you paid me a 300k TOC FAANG salary there is no way I'm dealing with those kind of people, forget about it for a project of my own passion. Thank you for your interest, but please take your "help" anywhere else.
His conclusion is bleak and a bigger warning than most realize. How many people have amazing ideas on their laptops in a proof-of-concept state right now but they don't know what to do? One on hand is the specter of entitled expectation from a legion of idiots who have been conditioned on the open source models made possible by huge corporations. On the other are the corporations salivating at Apache, MIT and BSD licensed free software they can host for almost no cost. And as he points out - the current alternatives for funding are totally inadequate.
I get the point you're trying to make here, and I agree that funding from big corporations has fundamentally distorted the open-source landscape in ways that might not be the most desirable, but we can acknowledge that without invalidating people who believe that the Elm project has been mismanaged by just assigning them to some "those kinds of people" bin.
But to continue on the topic you brought up - I am also not interested in calling out people who decide a project is not for them and decide to walk away. I am calling out people who jump into discussions about the core fundamentals of projects that they are not currently contributing to and turn those discussions in a hostile direction. Once they realize that their technical contributions are not wanted they turn to character and reputation attacks against the project owners and core contributors. Even if one out of one hundred users matches this description, you end up with a situation most sane individuals want to avoid at all costs.
So there is no invalidation of people who believe Elm went in the wrong direction. There isn't invalidation of people who wish to express that opinion publicly and even on project hosted forums, chats, bug-lists, etc. It is against those who go the "extra mile" and persistently push for changes even after they have been rejected and who resort to attacks on a person's character or reputation when they fail to get their way. And it is against those who defend such behavior by invoking some unspoken rules of open source development.
This was and I believe still is disallowed on any communication venues controlled by the Elm core team. Any discussion of alternative designs or workarounds was shut down and deleted, regardless of tone.
Or as I like to point out, living as street artist.
Sure one does get some money out of it, yet on its own its isn't going to pay all the bills.
That's a weird example, because it supported a huge industry for decades.
Artists get their money from people going to concerts.
From albums being sold, after everyone takes their slice, it is a buch of pennies, assuming they aren't being pirated in first place.
To that end, it's worth asking why our "mechanisms for getting creators rewarded for their efforts suck". Is it because copyright is too weak/too strong? Is it because patent process is overly burdensome/costly/a joke? I don't know, but I empathize with your points entirely and think it's worth thinking about.
Personally, I think there's room for more nuance in the commercial/non-commercial licensing dichotomy. Specifically, instead of using that as a breakpoint for charging licensing fees, maybe having some revenue threshold after which payments need to be made. I was about to say into a general fund to support open-source developers generally, but that would be ripe for corruption, rent-seeking, and self-dealing (like union administration). Better to go straight back to sponsor the project, IMO. Some products do this - free for organizations with less than x revenue. I've not really seen it much in open source though. Some tooling to make this "10 times easier" could be the next github.
But this problem has plagued inventors/creators of all eras. It does seem like we have better tools than ever to deal with it - telemetry, micropayments, etc.
This means the code might be tackling problems that small companies don't have, though. It might be more complicated than you need.
Maybe it's bleak if you care about control issues (governance) or you want to make money off open source? But as a source of free, possibly useful stuff, it's pretty good.
Flutter seems to be a revenue maker for Google (which should explain why it has gained so much attention - just look at the StackOverflow tag for Flutter, or their Youtube Channel - it's really, really active and the pub.dev package manager has grown immensely lately, to the point there's packages to do nearly anything in Dart/Flutter), but I am not sure how exactly given both Dart and Flutter are open source! Are the big users of Flutter giving Google big money?? Or how does that work?
Also, the ads codebase is enormous, so there are plenty of technical challenges from getting language infrastructure to scale to their needs. (Consider getting a language server for an IDE to scale up as a codebase grows, or making the language tools generate efficient JavaScript that's not unreasonably slow to download, or loading and displaying large tables in the browser.)
I have no idea whether Flutter generates revenue directly. I'd guess not, but internal usage of Flutter is likely enough to justify having a team devoted to maintaining and improving it.
Lots of other links you can find that explain the choice.
Blew their chance at what? And what trade-offs would have been necessary to take that chance?
Those questions are at the heart of this talk, and I think make it clear that there is a much larger range of definitions for "success" than is often assumed. And that perhaps we should question the default definitions. Perhaps the decisions, if not the ones you would have taken, are not so obviously bad (or are at least understandable) with that in mind?
Tools people complain about And tools nobody use.
Elm had a massive shot at making the big time circa 2015-2016. If they hadn't blown that shot, yeah, they'd be the kind of tool a lot of people use and complain about. But what I see when it comes to Elm is a lot of people who don't use the language shaking their heads in frustration at what they see as missed potential, while a handful of people who do use the language insist that it's perfect and unchangeable and that Evan could never ever make a bad decision, all the people who disagree must just be short-sighted.
I think there’s an interesting discussion to be had about why all these programming tools (languages, frameworks, version control, other infrastructure) have made the job of software engineers so much easier, and many more people have entered the profession as a result, but the wages of software engineers has continued to rise.
More demand. As you’ve stated, we probably just wouldn’t have services like Google Docs if we were still writing code in Fortran. But with the advent of better languages and tooling, people replace old tasks (e.g. spreadsheets copywriting) with automation, and create new types of software (video games, social media, ads). Software engineers have become more efficient but there’s a need for more software engineers, because there’s a need for much more programming.
It’s the same reason people still work a lot despite having more and more automation: our expectations for a decent quality of life (housing, shopping choices, machines) have increased. Workers have become more efficient but there is much more work to be done.
A lot of it is inefficiently too. I believe there are a lot of “bullshit tech jobs” (see: ad creators), and software produced with real money which really just isn’t necessary. Similarly, there are bullshit jobs in other fields (see: call centers); things like huge houses and shopping choices which aren’t necessary, and there’s a lot of waste (food, old stuff, products designed to fail), which all cause society as a whole to work much more than we really need to.
But it’s not just inefficiency. There are a lot of problems we simply couldn’t tackle with old technology (be it software, hardware, chemical manufacturing, healthcare) which we can tackle now. So, better tools don’t reduce jobs, they just change how the jobs are done and what they’re for.
The "relations" in Prolog and SQL ("relational model") are the same thing, and Prolog is arguably "nicer" than SQL?
It seemed like a good idea worth exploring. I can see how he'd get Jeff'd there.
Lamdera, fwiw, is storing Elm states in a database right now. It's a closed source fork of the Elm compiler. It's honestly amazing, and it may be what Jeffs the official Elm on the DB. IDK.
But it sounds like Elm is relatively easy to rewrite/reconfigure a backend to target a different language.
Thanks for the lead on Lamdera, which is helpful.
If it is it's a capability of a branch of the compiler that only exists on Evan's laptop
> something only known to those on the Elm mailing list.
As far as I know there is no Elm mailing list.
> But it sounds like Elm is relatively easy to rewrite/reconfigure a backend to target a different language.
I don't think it targets LLVM or anything, so you're gonna be on your own for that.
I've watched Elm for many years from a distance, and I think that with a little more "social skills", Evan could've probably done the same as Elm had quite a lot of hype a few years ago... notice that even though Andrew may be a bit more charismatic (though I find Evan really nice too) he also has a bit of a hardline regarding features, specially requests by sponsors are not treated specially in any way - there's a talk from an Uber employee where he mention that after getting Uber to sponsor Zig, they asked about some issues they'd created and were told very, very clearly that sponsorships don't come with "perks" like having your issues given priority!
I'm not saying there needs to be only one programming language out there, but I get overwhelmed at the fragmentation. I am someone who programs to get things done not to sit back in awe at the beauty of programming languages, and there have been small projects that I have wanted to do that were impossible because my dependencies were spread across different languages (some of which I was not familiar with).
I know a lot of people like Java but Java is exactly that.
You want to benefit off the work of others, and bear zero cost. What if you just built those libraries yourself, or paid someone else to do it for you?
And on language creation and development, why should we stop exploring, discovering and innovating? To make you more productive? What a joke.
No one can do that to any large scale. That is all there is to say about that.
And all my projects are open source. I give back to the community and to science (see below).
> To make you more productive?
There's intent, and then there's reality.
I work in scientific programming, and scientists are absolutely struggling with software development. Not programming, but everything related to package management, version control, distribution, different languages, incompatible ecosystems, etc.
There is complexity in the boundaries of packages, languages, and ecosystems. That is unsolved and no one seems to care about it as long as you can do this cool things with your own language.
Someone's intent could be to make python package management easier. The reality is that having only a few people adopt that can make the entire community suffer.
We have Turing's Universal Machine, we have transistors. Science and capitalism have delivered technology and wealth. We won history.
The obvious thing to do is apply our technology and resources to provide a basic level of quality of life to everyone (i.e. knockout the lower levels of Maslow's hierarchy.) Build a simple and efficient regenerative machine to provide food, shelter, clothing, some medicines, building material, etc. It would look like a neighborhood.
Then people like Evan Czaplicki could just do whatever they wanted to without having to worry about making money. Also everybody else.
We do this because: A) We can. B) Not to do so is cruel.
corollary: if no killer app, not killer language
https://www.welton.it/articles/programming_language_economic...
Your choice of examples also dates you as being roughly 20 years older than Evan. ;)
The economics of creating a programming language are similar to most other open source software economics, although even proprietary languages need a way to get an initial user base. In some cases that's easier, as a large company can force a language for some important environment like writing iOS apps.
I think your article and Evan’s talk are largely complementary.
Build a successful product with new lang https://scrimba.com, make sure the product's very hard to Jeff and take VC money.
Now you can work on the language as you please, and they can't Jeff you since nobody else can build something similar (not in a reasonable amount of time anyway)
P.S: taking VC money is optional. Scrimba is profitable so it wasn't necessary.
How do you sell people on enrolling for a course that uses a language that isn't used by any other company, which makes it pretty useless outside of possibly teaching some high level concepts?
I liked the ending of 'now that I've proven myself to be capable and likeable, someone give me money for this new thing I've been working on'. Very shrewd 'self marketing'.
For all this talk of free markets and whatever, we still live in a world of rich but bored and creative but poor.
They are the problem.