Rust long compile times are mainly caused by LLVM, and the amount of IR that is given to it.
"Lisp on Small Pieces" from 1994
https://www.amazon.com/exec/obidos/ASIN/0521562473/acmorg-20
It is not 70's, but 26 years should be enough to get hold of such knowledge.
Or "Modern Compiler Implementation" from 1998, a little more young.
https://www.cs.princeton.edu/~appel/modern/
Or rather "The Implementation of Functional Programming Languages", from 1986, now getting closer to 70's.
https://www.microsoft.com/en-us/research/publication/the-imp...
I can still have a look into my SIGPLAN paper collection to dust off some ML papers.
The large amount of IR given to LLVM is closely related to the amount of abstractions that Rust uses. It's especially relevant for iterators where a "simple" `for` loop expands to `into_iter()`, `iter.next()`, `match`, `drop`, and arithmetic expands to calls to overloadable operator methods, all moves are `memcpy` to clean up, and so on.
Speaking of which just use OCaml bytecode backend or Haskell interpreter to see how slow complex languages are.
This is what is missing to Rust and cranelift might help to deliver, alongside not having to keep compiling the whole world from scratch.
Avoiding that means doing the flattening earlier, in a higher level IR.
* GHC does exactly what I suggested rustc needs to do- it does the flattening on an earlier, higher-level IR than LLVM's. (It is also not exactly a fast compiler, though!)
* I'm not sure if this applies to F#, but in C#, LINQ compiles faster by not doing as much (if any) flattening in the first place.
https://play.rust-lang.org/?version=nightly&mode=release&edi...
for _, m := range maps { ... }?In general though I think the functions used in map and filter should be total. Most languages are not expressive enough to handle errors fluidly in long chains like that, you can do it with monads but apparently everyone hates those except me.
Agree to disagree I guess.
> Combining a map and filter into one loop body tends to create specific, imperative code that doesn’t need to be specific or imperative. For one, loops require mutation to have the same effect as filter and map, and as soon as you’re maintaining state, things get confusing.
I agree that immutable, declarative code is easier to reason about in general than mutable, imperative code, but it's not a binary proposition. I posit the cost of managing the tiny bit of very-local mutable state for a for loop is a lower cost than trying to model a problem in terms of map and filter. While I can appreciate the elegance in "immutable always", I think it's too puritanical for real world software development, especially in these cases which are simply expressed as for loops.
Note that languages like Python have strong support for immutable expressions and `for` loops, and it seems very common (even idiomatic) to use imperative `for` loops for anything more complicated than the simplest list comprehensions.
> In general though I think the functions used in map and filter should be total. Most languages are not expressive enough to handle errors fluidly in long chains like that, you can do it with monads but apparently everyone hates those except me.
I think it's just that monads tend to allow for very abstract code at the expense of understandability, and the abstraction that monads facilitate is often well in excess of what a given problem requires. If you're writing code to maximize abstraction (which is a lot of fun), monads (and map/filter/etc) are great, but if you're trying to work with a team to build software to solve real problems, you probably care a lot more about readability than gratuitous abstraction and the monad (/map/filter/reduce/etc) juice frequently isn't worth the squeeze.
My workplace uses Elixir exclusively in production for our backend services, so we’re all functional programmers and we do not find that map, filter, and reduce make things harder to read, quite the opposite. Breaking up loops does not introduce cognitive overhead, it makes every step in the chain more explicit. You can call this personal preference but there’s nothing concrete in your belief that for-loops are more readable or easier for teams of programmers to maintain, many people disagree.
The other time it's very useful is when translating algorithms specified in an imperative form. I translated a bunch of algorithms specified as for loops to elixir, and it was annoying to have an extra source of complexity when comparing the implementations for correctness.
Hardly. You can absolutely have multi-line lambdas, just not multi-expression lambdas (the real limitation is that there are no try/except expressions so certain fallible manipulations aren't so easily expressed). Similarly, Python lets you overload operators so you could always override `|` or whatever. Of course, operators are just syntax sugar for function calls so this strikes me as a cop-out. In either case, neither of these excuses are pertinent to the community's rejection of complex comprehensions or higher order functions.
> My workplace uses Elixir exclusively in production for our backend services, so we’re all functional programmers and we do not find that map, filter, and reduce make things harder to read, quite the opposite. Breaking up loops does not introduce cognitive overhead, it makes every step in the chain more explicit. You can call this personal preference but there’s nothing concrete in your belief that for-loops are more readable or easier for teams of programmers to maintain, many people disagree.
You could argue that any two models which evaluate to the same thing are fundamentally equally easy to understand. You could argue that monads are just as easy to understand as a sequence of statements in an imperative language, but experience suggests that people have a harder time understanding the more abstract concept rather than the more concrete concept. This shows up all over--Newtonian physics are more easily understood than the more abstract quantum physics and in mathematics "more advanced" math tends to mean roughly "more abstract".
I can only assume you haven’t done a lot of functional programming because Python is just not expressive as a functional language. Lambdas are extremely cumbersome. JavaScript by contrast has a huge FP culture and does not even have comprehensions. But it does have syntactically lightweight, multi-expression lambdas.
> You could argue that monads are just as easy to understand as a sequence of statements in an imperative language, but experience suggests that people have a harder time understanding the more abstract concept rather than the more concrete concept.
Except I’m not arguing that, because I don’t believe that monads are easier to understand, I believe that functional programming with pure functions is. This is just another restatement of the same argument you’ve been making the whole time. We onboard people who have never used FP into being productive Elixir programmers in two weeks or less. There are no monads in Elixir, nothing more advanced than recursion and function composition. My experience differs from yours. Our new programmers have very little trouble grokking how FP works. They have much more trouble understanding the Erlang concurrency model and the various primitives available, but those are unique to Erlang and uniquely powerful.
You assume wrong, but more importantly you missed the point of the snippet of mine to which you are replying—namely the expressiveness isn’t the reason the Python community rejected those features, but rather the complexity and cognitive burden.
> Except I’m not arguing that,
I wasn’t arguing that you were arguing that, I was using it as an analogous argument. We’re not going to get anywhere if you aren’t going to respond to what I’m actually saying.
But we don't do that because for the entire history of computing, we've come to realize that abstractions enable us to reason about programs at a higher level without having to be bogged down in irrelevant details and that getting those details right once lets us avoid the inevitable bugs that occur while reimplementing them thousands of times.
By absolutely every objective measure we know in software engineering
let triples = ints.map(|x| x * 3)
is strictly superior to var triples = []int
for value := range ints {
triples = append(triples, value * 8)
}
The fact that this even has to be argued is just bonkers to me.The second one is trivial to introduce bugs into, reading it requires mentally filtering out 90% of the code as irrelevant minutiae, and it's even worse from a performance perspective unless you remember each and every time to allocate a result array of the appropriate length to avoid reallocation. None of the details of allocating a result array or iterating over indices/values are important from the purpose of expressing the problem to be solved, and yet every time someone has to read this solution those things are of greater visibility than the actual business logic.
Worse, repetitive and unnecessary boilerplate like this hides bugs. Did you catch the fact that I'm multiplying by the wrong value in the golang version? Did you catch that I'm accidentally tripling the indices and not the values themselves? If you did, would you at least acknowledge that the first bug is much easier to identify in the first example than the second, and that the second bug is strictly impossible?
As an added bonus, the first one (in Rust at least) is automatically vectorized using SIMD for optimal performance. This happens even as you chain additional operations to arbitrary complexity.
If you still somehow think the second version is "better", then whatever your argument is can trivially be repurposed to say this version is better still:
int* triples = malloc(sizeof(int) * array_len);
for (int i = 0; i < array_len; i++) {
triples[i] = array[i] * 3;
}
If those details of creating a result array and explicitly enumerating are important, why isn't maintaining your own counter and directly indexing? At least the C version somewhat encourages you to allocate a result array of the correct size. if res, err := fn(...); err != nil {
fmt.Errorf("Look ma, I'm a human exception handler: ", err)
}
This is held up as a positive thing, because handling errors is explicit and good. Somehow, the Rust equivalent of `?` is implicit and bad, despite desugaring to almost literally the exact same thing and being actually impossible to forget or overlook due to the return type of the function enforcing a Result.In the same breath, devs will assert that the language is "low boilerplate" (despite 50%+ LOC being just one type of boilerplate) because Error Handling is Important. Which seemingly ignores that the important part of error handling is actually ensuring that errors are handled and not the mechanistic act of self-flagellation by wrapping every line of code in nearly-identical error handling ceremony. Which is unfortunate because golang doesn't do so great a job with the first part. It's surprisingly easy to actually forget to do something with the error while carrying on with a return value whose semantics are undefined (unlike Rust's Result, where you can have an error or a value and not both).
I would prefer my language wasn't designed around corporate bureaucracy personally.
It is possible to have company policies against abstraction and cleverness when working in more expressive languages, and this would achieve almost the same intent as Go, it's just that actually using Go is more convenient. The language is corporate bureaucracy. The whole point is what's not there. What can't go off in a weird directions on a per-project basis. What junior engineers in a high-turnover environment can't screw up. It's not by accident that the native toolchain will give you a test coverage report with untested error return branches in red. Google is just happy to pay people for that drudgery, I guess.
I fundamentally disagree with that. the only thing you know in java is that it throws an exception that's maybe caught somewhere. That's an untested branch aswell.
You do have to test cases where exceptions are supposed to be caught and handled/eaten, which I agree is meaningful and relevant in all languages.
In general your org’s policies for what to test seems like a recipe for a low-value, high-effort test suite. Reminds me of the unit tests for getters and setters in Java.
I agree, given that Go is what it is, it's a bad policy.
* it has been actually demonstrated that shorter programs contain less bugs,
* code is read much more frequently than it's written, and to the point code is easier to read than code full of boilerplate and meaninglessness syntactic elements,
* menial tasks and distractions introduce mental fatigue that consumes your limited brain resources that you need to think about the structure of your code that is actually dealing with the problem at hand.
With respect to the second bullet, I agree that code is read more frequently than it is written and that readability is paramount. I strongly disagree that minimizing the character count is the same as optimizing for readability. In particular, I find very syntactically terse languages like Haskell or Ocaml far harder to read—indeed ReasonML exists precisely because so many people find OCaml so difficult to read.
With respect to the third bullet, you should think about your error control flow. The structure of your code doesn’t change whether you’re using Go’s “if err != nil” or Rust’s ? operator. The difference is a dozen characters. I can’t believe that the real cognitive load is in the actual pressing of keys, but if that is your experience then text editors often have a “snippets” feature or extension that will let you map that boilerplate to a hot key.
It was also an oblique reference to this talk which is one of my favorites: https://youtu.be/mZyvIHYn2zk
There’s no universally correct answer to the right balance of “smart compiler” vs “simple language”. Having worked with Go on a daily basis for five years now, I really like the simplicity of the language. Compared to the cognitive load imposed by the language when using C++ and Rust, I much prefer the naive straightforwardness of Go. There’s no devilry about it.
Go vs Rust is not a fair comparison, because Rust's type system is much more powerful that Go's and enables applications that Go's doesn't (like embedded systems programming). Also some of the things I said about C++ apply here.
There are also features of Go absent from C++ and Rust (reflection, garbage collection). So it's really puzzling that you'd pick those two.
To have a fair comparison, you'd need to look at something like Kotlin sans the class system. There Kotlin is a clear winner, and with the same engineering resources put behind it, a cut-down version of Kotlin would easily be able to achieve build times in the ballpark of generic-enabled Go.
Despite its claimed simplicity, Go has inconsistent design and bug-prone unwieldy features (try to write code that merges streams of data from two channels without blocking unnecessarily). It also has unnecessary constructs that contradict its claim to include only what's necessary (interfaces are just structs of function pointers, type methods are just functions with a glorified first argument).
It may be enough to write one-off command line applications, but it doesn't give developers the tools to create complex distributed systems in a systematic, clean way, which was supposed to be its main area of application.
So basically no real example. It is only what would be possible if things were to happen certain way.
> It may be enough to write one-off command line applications, but it doesn't give developers the tools to create complex distributed systems in a systematic, clean way
This is just arrogant opinion. Distributed tools and applications like NATS, Kubernetes, Docker, CockroachDB and so many other exist and written in Go. Developers who can't write distributed systems themselves can ofcourse buy or license tools from vendors. And at that point, honestly, it could have been written in any language.
It is also a problem of langage design though: JS’s HoFs are spec-defined as returning Arrays, which makes fusion way more complicated.
Internal / external iteration also have different optimisation profiles e.g. Rust initially used internal iteration, switched to external because that was much more conducive to optimisations, but re-introduced an internal supplement because some combinations / patterns optimisé much better through internal iteration.
Source? I thought conventional wisdom was that internal iteration was much easier for compilers to digest.
With https://www.reddit.com/r/rust/comments/5ez38g/rusts_iterator... as the… not counterpoint but reintroduction of internal iterators.
Linking to the Reddit thread rather that the article directly because lots of comments from important rust folks.