The rewrite in Rust thing, for darklang at least, comes from the 50th time we've had to do something bad and ugly to fit into what is supported by OCaml.
7,962 karma · joined January 26, 2009
Previously founder/CTO of Dark (http://darklang.com), and founder/CEO of CircleCI (https://circleci.com). PhD in CS.
https://hachyderm.io/@paulbiggar
https://bsky.app/profile/paulbiggar.com
https://twitter.com/paulbiggar
https://blog.paulbiggar.com
Contact: paul@techforpalestine.org or paul.biggar@gmail.com
The rewrite in Rust thing, for darklang at least, comes from the 50th time we've had to do something bad and ugly to fit into what is supported by OCaml.
I actually confused my facts here - it's tipping I was thinking of that comes directly from slavery. Employer-paid health care just _continues_ because of racism, as opposed to being created by it.
In this case, the employer is the parent's employer. Which means that the people who can take the risk to create startups is limited to "people whose parents have health insurance". Which excludes a large section of society, including for example poor people.
(also--and this might be a bit of a jump for someone just learning about inequality of opportunity--but a fun thing to research online is "why is the US the only country with employer-based health insurance", especially once you discover the answer is "slavery")!
> I quit my job and thanks to the insurance changes in the US a few years ago
So you're saying that you were able to take the chance to build a startup because of the safety net?
About a decade ago I did research on sorting and searching, which was the same type of work going on here (https://github.com/pbiggar/sorting-branches-caching). I found it was extremely difficult to accurately say whether one algorithm is faster than another in the general case. I found a bunch of speed improvements that probably only apply in processors with very long pipelines (like the P4)).
Execution speed is probably the right metric here, but it makes it hard to understand _why_ it's faster.
PS. Looking at "boundless interpolated search" (https://github.com/scandum/binary_search/blob/master/binary-...), it seems it's missing a `++checks`. I initially thought this could be the cause of a misreporting it as being faster, but I see the benchmark is run-time based so that wouldn't cause it unless the incrementing itself is the bottleneck.
As an example, Dark has "live values" which is where at any point in your program, we show you the actual value for the expression you're on. We use "traces" for this, which means the input (usually a HTTP request) as well as saved results of every expressions.
To accomplish this, we need a language where we can mark which functions are pure, where we can instrument the runtime, and where we can play the runtime back in our editor, filling in the saved trace values.
So nothing wrong with python, but it doesn't have any of those properties.
Similarly, we want your program to deploy instantly. That means that we need a technical way to deploy in 50ms (not sure that's available in python), but we also need language features that support that (such as feature flags built in, and the ability to safely make changes to programs). Otherwise, you'd find it's super easy to break your program with a typo.
This stuff is plausible in Python and many other languages, but seems super difficult.
Startups are hard, and we were trying to grow something that hadn't gotten to product market fit yet. I'm working on getting to PMF, and thinking hard about what's needed to do that, while making sure Dark works in the long term.
People generally use what we call "trace-driven development", where they make a HTTP request to the code they're editing (typically, people are making APIs or backends to JS apps), and then they always have "live values" of what they're working on (similar to a REPL, or to running unit tests on a loop, but instantly).
We've looked at allowing people to paste text (eg from Stackoverflow) into Dark. We think we can do it by just inserting each character one at a time (though there's a technical flaw with that at the moment, which could be addressed). In that sense, Dark is sort-of a parser (I don't know if it technically counts).
To get to the root of the issue though, there is not a parser.
Well, one could consider the function that takes each keystroke to be a parser - though I don't know if that's technically correct either. So perhaps the more accurate thing to say is there is not a step that takes a textual source file and parses it.
The main idea is that you when you make a change (let's say, you type a key in the editor), that change happens directly on the AST (the internal set of objects that represent the program). So for example, if you're seeing:
let x = 5
x + 2
we represent that internally as `Let ("x", Int 5, BinOp ("+", Var "x", Int 2))`In a normal programming environment, you type up text, and the parser converts it to the internal representation. With Dark, we always keep everything in the internal representation. That means, if it isn't a "syntactically" valid program, you can't create it in the first place.
In practice, this means that if you type, for example, `z` in the middle of `let`, in a traditional language you'd get an error like `lzet is not a valid token` and in Dark, the `z` just wouldn't appear (nothing would happen). So it's not possible to get a syntactically invalid program.
The overall idea with Dark is that programs are always live (and also safe). So to make that be true, we need to ensure that you don't spend a period of time in a state where we can't understand your program (such as when the syntax is invalid).
> Syntax matters
The approach we took with Dark is that there isn't a syntax, per se, in that there isn't a parser. There's certainly a textual view of Dark and so it's important that the code looks good (it currently looks only OK, in my opinion). But as a result, we have other options for minimizing keystrokes (autocomplete built-in), parsing (again, no parse), and minimizing keywords (you're allowed have a variant with the same name as a keyword, which isn't allowed in languages with parsers (well, lexers, but same idea).
> context free grammars
No parser, no need to have a grammar. His point about IDEs is great - we only support our own IDE (a controversial decision to be sure!)
> Redundancy
This is an esoteric parsing problem, that only applies if you have a parser. No parser means no syntax errors. We are left with editor errors (how does the editor have good UX) and run-time errors.
> Implementation
He's right about how hard error messages are, so I feel good about our "not parsing" approach.
> Compiler speed
Our compilation is instant. The way it's instant is:
- very small compilation units: you're editing a single function at a time, and so nothing else needs to be parsed.
- no parser: The editor directly updates the AST, so you don't have to read the whole file (there isn't a "file" concept). Even in JS, that means an update takes a few milliseconds at the most.
- extremely incremental compilation: making a change in the editor only changes the exact AST construct that's changing.
> Lowering
This is really about compilation. One thing you can do, which is what we do, is have an interpreter. Now, interpreters are slow, but we simply have a different goal with the language, which is to run HTTP requests quickly. We do run into problems with the limit of the interpreter, but we plan to add a compiler later to deal with this.
Really what I'm suggesting here is that compiled languages look at having interpreters in addition to compilers.
> i/o performance
> memory allocation
I think he's approaching this with an implicit goal of "it must be as fast as possible", which isn't necessarily a goal for all languages.
> You’ve done it, you’ve got a great prototype of the new language. Now what? Next comes the hardest part. This is where most new languages fail. You’ll be doing what every nascent rock band does — play shopping malls, high school dances, dive bars, etc., slowly building up an audience. For languages, this means preparing presentations, articles, tutorials, and books on the language. Then, going to programmer meetings, conferences, companies, anywhere they’ll have you, and show it off. You’ll get used to public speaking, and even find you enjoy it (I enjoy it a lot).
Well this is certainly correct!
I don't know if Dark (my startup trying to solve this) is going to take over, but it's pretty clear that the current complexity is unsustainable.
This goes double if recession/layoffs/etc is happening due to COVID. People are going to need to spend a lot less on engineering, and the amount being spent on engineers that maintain infra is an obvious starting point. I would expect people to switch to much simpler infrastructures, and reduce the complexities of their stacks by necessity.
They what?!?
No, you're not. And so every dude on this thing is going to use "he" everywhere cause "no-one's complaining". And then when people who prefer different gender pronouns come in, they're going to see people being assholes about gender pronouns, and they're going to say "fuck this" and leave (note, they're not going to reply either, cause they also are sick of it).
And so to prevent that, people other than the "supposedly offended person" need to step in, even if they weren't asked.
I'm not going to reply to further discussion here. I hope this was useful to you.
Don't be an asshole.
Parent misattributed the pronoun, sure. And then they were asked to not do that.