Comment by hota_mazi | Hacker News Reader

Comment by hota_mazi | Hacker News Reader

Parent Full thread

> The reason for this is that, as much as I know, that a compiler does type checking for every single file. So if you spawn 200 processes of Swift compilers, it needs to 200x check all the other files and make sure that you're using the correct types.

I'm a bit baffled by that. Is the Swift compiler that naive?

Surely you know how to assess the number of processors/cores on your system and you spawn threads in a way that doesn't lead to diminishing returns. You use a bounded thread pool and you stay within these bounds.

Seeing such a speed up from simply merging text files is really puzzling to me. You have to type check the code anyway, surely the overhead of opening a new file is completely negligible compared to running this type checker on the same file? Especially since all these instances of the type checkers have to share a lot of data anyway.

Swift is very cool and I'm excited to see it and Kotlin become our next generation languages for mobile, front end and back end alike, but it seems to me the Swift compiler is still very immature.

There are a lot of 'obvious optimizations' like this lurking in swift. I get the impression they are somewhat engineer constrained in an interview with chris latner.

Is there a link where Latner says the Swift team is engineer constrained?

Ex: "But the real reason for doing it is that it was a small amount of work that moved Objective-C forward, which allowed the compiler and language team to focus on Swift because Swift was a very large amount of work. " http://atp.fm/205-chris-lattner-interview-transcript/

"So if you spawn 200 processes of Swift compilers"

I don't think that implies those processes all were spawned at the same time (or even that Apple's compilation process did the spawning. Given the effort they spent on their build processes, I do not rule out it is their tooling that does the spawning)

The thing is: all these processes are independent of each other. Uber likely has tons of files that are imported into many, many files. Each compilation process that imports such a file has to individually parse it.

I can see two ways around this: reusing intermediate results between processes and using fewer processes.

The first is like precompiled headers in C/C++. The second could mean having a queue of to be compiled files and a fixed number of compilation processes that pick up files from it.

Risk in both cases is that unintended state may leak between compilations. For example, compilation flags may be different for files f and g. Does that mean g can't use preprocessed module m generated while compiling f? Because of that, I would go for something like precompiled headers, because it makes it easier to reason about what information flows between compilations of f and g.

I don't know whether that would be sufficient, though. The biggest concern I have w.r.t. Swift is that all its cool features, combined, make it essentially impossible to build a fast compiler (Yes, languages such as C# have most of them and aren't that slow, but adding protocols and overflow detection (which you may want to optimize away a lot, even in unoptimized builds) may just tip the balance)

The Swift compiler has some serious issues with the way it does type inference. Try compiling code with a line of basic arithmetic on a few numeric literals.

That has literally nothing to do with what the parent comment is talking about. And that's not naivety either. That's because the way type inference, operating overloading, and literals work together ends up being a combinatorial explosion.

Kotlin has all of these features and build times don't (noticeably) suffer.

As does Rust.

From what I've heard Rust has serious build-time issues as well, mostly because of it's inability to perform separate compilation of generic code. Unlike Swift, Rust specializes everything at compile-time, similar to C++ templates.

That's not exactly true, while that is usually what Rust people write, you can choose to have stuff not monomorphized. I'm also not exactly sure what you mean by "separate" here.

As usual, compile times depend on what you're used to. It is something we're working on improving though; we want it to be very fast! We expect incremental recompilation to move out of nightly soonish; that will help quite a bit.

No it doesn't. Rust doesn't have overloading, or operator overloading, or literal overloading. It literally has none of the pieces that cause the combinatorial explosion in Swift.

Rust does have operator overloading though the traits in the std::ops module. There are only a few (&&, ||, etc) that can't be overridden.

Oh you're right about that. I was thinking of custom operators when I said no operator overloading, but of course that's not actually what we were discussing. Still, no literal overloading. Also, there are other differences between Rust and Swift's type systems that allow Rust to do type inference in a way that Swift cannot.

Reply on news.ycombinator.com