The more I read about Scala it makes me wonder why (1) I am not using it (or my team) (2) Java teams across the world are not at least seriously considering changing to Scala. It seems likes it's miles ahead of both Java and even Kotlin.
Kotlin can be introduced very easily into most Java codebases as interoperability is a tenet. Scala not so much. Holistic interop between Scala and Java can fill an entire intermediate-level book.
At the startup, we had a difficult time hiring for Scala. A number of candidates we moved forward with who were well-versed in it wanted to explore the language and embrace the FP aspects, which led to incomprehensible code for the engineers used to procedural OOO. In full retrospect, this was an issue with our hiring standard; Scala should not have been a specific part of our bar. It was a startup, we had crushing customer demands, there was no time to teach every engineer what an IO Monad was. It was a constant battle and perhaps the most frustrating aspect to me leading all the way up to acquisition. None of these issues would have existed if we chose e.g. Go; we would have quickly found a team of engineers who wanted to build applications instead of tinkering with the language. To me, that's the best part about Go -- it's boring.
Java is another boring language, but it's also a very verbose one, even when fully utilizing Lombok/G-libraries/etc.. Kotlin fixes the verbosity issue to a large enough degree for my needs without introducing enough interesting features to attract those darn academic types. To me, Kotlin is about as good as a language can get while remaining on the current iteration of the JVM. We have been slowly introducing Kotlin and plan to use it for most new non-Spark-related projects. I would only consider Scala for the same purpose if 1) interop was trivial to the degree that Kotlin's is, and 2) it was reduced to a reasonable subset of its full complexity. I believe the right steps are being taken with the language, but it still has a ways to go in those regards.
Incidentally, I find golang to be more verbose than Java. The amount of one-off functions you have to write to get around the lack of generics and/or map/filter calls on collections makes the code longer and harder to follow, and harder to debug. Not to mention error handling.
I generally agree with the gist of your post though.
What ended up happening with the startup you worked at out of curiosity?
We made two poorly judged business decisions: 1) build a marketplace without enough proof of demand, 2) treat certain types of financial instruments as securities ahead of a fed decision that never materialized (I call this the bizzaro-Uber play). Outside of the things we could have controlled, the sector we operated in did not meet its expected growth projections in the aftermath of certain participants not playing nicely. We were acquihired as part of the typical VC risk sharing.
The ultimate failure had very little to do with Scala as a tech decision. I was extremely proud of the products we built from an engineering perspective, but I don't credit Scala with that either. The high-performers on the team could have repeated the same technical success with any of the languages I discussed above. Perhaps our hiring was slower than our funding level would have allowed. Perhaps our worse-performing engineers could have contributed more if it was Go or Python. Neither of the alternative realities would have gotten us past the two aforementioned business decisions.
I've used Scala before. It has challenges that would make it a non-starter for many teams. It has some cool features, but those (IMO) don't outweigh the negatives that come with the language: it's overly complex (different people have very different styles for writing the language), the compiler is slow, it generates more garbage than Java, lots of implicit behavior, the standard library is not up to par, among others. Java is improving significantly in the coming few releases: pattern matching, records, switch expressions, etc. will bring to programmers the bulk of what Scala has to offer without the added weight. Kotlin is a much better contender for most teams anyway, and even that will be a challenge IMO.
What does Scala's pattern matching do that Java's (upcoming) doesn't?
> For example, does Java 14 Stream have any method equivalent to `collect` in Scala, which is a combination of filter and map?
Not that I'm aware, but then again, I don't see the large advantage compared to making two separate map + filter calls. Streams in Java are lazy, whereas calling `map` or `filter` on a concrete collection in Scala is eager, creating temporary intermediate collections only to discard them in the case of chaining calls.
case class Person(name: String, age: Int)
// a collection of 3 people
val c1 = Vector(Person("A", 5), Person("B", 6), Person("C", 4))
// now I want a collection of people who have age > 4 and then their ages are doubled
val c2 = c1 collect { case Person(name, age) if age > 4 => Person(name, age*2) }
data class Person (val name: String, val age: Int)
val c1 = listOf(Person("A", 5), Person("B", 6), Person("C", 4))
val c2 = c1.filter{ it.age > 4 }.map{ it.copy(age = it.age*2) }It removes a few things and adds a whole lot of new stuff to it.
It's a bigger, more complex language, even if you credit all the cleanups that were already in Scala 2 to Scala 3.