Comment by pcwalton | Hacker News Reader

Comment by pcwalton | Hacker News Reader

Parent Full thread

> (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).

Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.

> [1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used again in the same scope. It's definitely possible to extend `go vet` to handle this, and it may even be possible to write this in as a compiler error in a future version of Go.

That trivially fails to solve the problem, due to aliasing.

> [2] Incidentally, one of the reasons that it's so easy to do reliable static analysis on Go code (compared to other languages) is that the grammar is incredibly simple - it's almost entirely context-free, which is very rare among non-Lisps. Having a more complex type system usually requires at least some additional syntax to along with this, which means you'd have to start sacrificing this design goal as well in order to create a more elaborate type system.

Rust's grammar is also context-free, as far as I know.

Your proposed static analysis is not reliable. The "more complex" type system in Rust exists precisely so that we can do more reliable static analysis.

> Rust's grammar is also context-free, as far as I know.

I believe bytestrings are context-sensitive :( I don't remember the exact details here, so I could be wrong. But other than that...

Technically, byte-strings (and nested comments) mean the lexer is not regular, but the grammar is still context-free. But anyway, parsing is not typically the bottle-neck these days.

Oh, that's just lexing though.

raw strings: https://github.com/rust-lang/rust/blob/master/src/grammar/ra...

> Thanks to the module system, the Rust compiler never rereads a file more than once.

Are you sure about this? When you use a generic function accepting and/or returning values of type T, I guess the compiler has to generate a version of the function for each instance of T. But the compiler cannot know all the possible use of the generic function without having walked through the whole code first, which implies at least two passes. How does it work?

The compiler reads the source code into an internal AST and manipulates that, never touching the external files again.

Ok, you're right, but we're playing with words here :) I understand the file containing the source code of the generic function is read only once, but I guess the AST of the generic function is processed many times, at least once for each instantiation of the function? And I'd venture in guessing that the biggest cost is in processing the AST and compiling it, not in reading/parsing the source?

Generating specialized code from a generic AST is in no way analogous to the exponential-time explosion that is the C++ header system, which is what Go is referencing when it says that it has been designed to read each source file only once. All languages with proper module systems have this property (which is to say, basically all languages that aren't C or C++).

Yes, designing a proper module system and getting rid of the header system is the best way to improve compilation time. But it was not my point. There are other factors impacting compilation time. Look at Scala for example: it has a proper module system, but compilation is still relatively slow, even if better than C++. This is the reason why I'm interested in learning how Rust compiles generic functions, and what is the impact on the compilation time (because the compiler has to generate a version of the function for each possible T), the binary size, and the pressure on the CPU cache (because having many versions of the function makes harder to keep all of them in the cache at the same time).

The Rust compiler actually has built-in diagnostics to let you profile the time that each phase of compilation takes. On mobile but I believe that it's `rustc -Z time-passes`, and whenever I use it on my own code 95% of the time consists of LLVM performing codegen. Nearly every other compiler pass takes approximately 0% of the compilation time (coherence checking being the notable exception, for reasons I have the discern).

> the exponential-time explosion that is the C++ header system

No I don't think so.

It's not playing with words: the parser only runs once, and the processs/reprocessing AST is much faster than interpreting the original source multiple times (e.g. it can be indexed cheaply).

> > (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).

> Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.

As I said right at the beginning of my post, I'm not trying to compare Rust and Go directly, because I don't think that's meaningful. I'm explaining why these particular features would be difficult to incorporate into Go. Note that I never said that Rust reads a file more than once, or that Rust's grammar is not context-free. In fact, the word "Rust" doesn't appear anywhere in my comment at all except in that very first paragraph.

I love talking about PLT and would otherwise be interested in having a discussion about static analysis and hearing why you think it would not solve the problem, but I have to say, it's both frustrating and discouraging to post an in-depth response and then get downvoted twice, with the only reply being one which very clearly ignores the very first line of my entire response.

Bystander here, neither upvoted nor downvoted you, but I think one reason for the skepticism is that people have hard the "It shouldn't be hard to implement X in language Y..." refrain many times before, and until somebody actually does implement X in Y, it means nothing. "It shouldn't be hard to make Python at least as fast as V8." "It shouldn't be hard to implement static typing on top of Python 3 function annotations." "It shouldn't be hard to add lambdas to Java [well, they finally did with Java 8...which we still can't use on Android]".

If it's actually not that hard, go implement a checker for Go that does check that statically-allocated rvalues are never used again, and post it here. You'll probably shoot to the top of Hacker News, it'd be a nice open-source project for the resume, and it'd provide a very useful tool for the Go community.

> I would otherwise be interested in having a discussion about static analysis and hearing why you think it would not solve the problem

Stack allocation isn't in the semantics of Go, so that's a pretty weird thing to use a basis of a static analysis. It's also not sound because of interfaces or closures. You would want something more like "fully by-value data with no pointers in it, no interfaces, no closures".

Not to mention no arrays or slices! This burned me pretty hard during my first week with Go.

You outlined why they shouldn't be compared because they are different and outlined differences. The response (from one of the rust devs, if I'm correct) explained how some of your purported differences weren't actually different. The response wasn't ignoring your first line, it was explaining how portions of your evidence backing up that first line were factually incorrect.

It's impossible for your comment not to be an implicit comparison of the two. No amount of disclaiming the intention to compare can prevent that.

Reply on news.ycombinator.com