Hacker News Reader

Comment by ohnoesjmr | Hacker News Reader

I congratulate the effort, as I always thought that Spark is great, but the fact it was written in Java hinders it quite badly (GC, tons of memory required for the runtime, jar hell (want to use proto3 in your spark job? Good luck)).

I do however worry that rust has a high bar of entry.

View on HN

I'm quite sure as soon as there are performant WASM engines deployable outside a browser (e.g. parity-wasm) then Ballista will converge on it and it would hence support anything that can compile to wasm.

In fact, I think the time to begin that type of project is now because by the time you do the ground work for such a project and integrate, etc. a wasm engine will be ready.

The first EA releases for value types are out, even if it takes a couple of releases more, it is much easier to improve existing stacks than restart from zero.

I agree. Rust has a very steep learning curve compared to JVM languages. My hope in building this platform is that it can provide value to other languages (especially JVM) by taking query plans and executing them efficiently.

> compared to JVM languages

Apache Spark is written in Scala, and I wouldn't describe that as having an 'easy' learning curve, even compared to Rust! If you want something 'easy' on the JVM, Kotlin (and perhaps Eta or Frege) might be more appropriate.

Scala is way easier to use, given it uses a tracing GC, has three good IDEs, and more mature ecosystem.

Coming from Java, i found Rust easier to learn than Scala.

I expect you were exposed using a heavily functional programming approach. That's a whole 'nother (larger) area to pick up along with the different language.

If Scala is used as a "better Java" to start with, and then the developers explore FP at their own pace, I think there's a better outcome. Granted Java 8+ has done some to close the gap with Scala as well.

I wonder if Scala-Native will absorb some Rustish features as other languages have or are attempting to...

(What I said above echoes Odersky's ideas about Scala developer levels: https://www.scala-lang.org/old/node/8610.html)

Reply on news.ycombinator.com