So even if feature wise the 2 equalize (a good thing not a bad thing IMO) stylistic differences will remain. Many of Kotlins syntactic differences are unlikely to make it to Java for instance.
I'm ok with this. I use both for different things. Kotlin for most app level development, Kotlin for libraries that are only useful in the context of Kotlin, Java for everything else so it has minimal dependencies and can be used from Kotlin/Scala/Java/Clojure equally well.
The truth of it is that Kotlin is incredibly easy to learn, even for non-Java devs. It is definitely focused on a very low barrier to entry. I don't think this was an original goal but it's become a significant focus of the language especially after being selected to be the next platform language for Android.
This isn't true of Scala at all. Beyond things like SBT, comparatively poor IDE support (less of an issue these days) it is also just a very complex language, with many complicated semantics in it's standard library and even more complex/fancy 3rd party libraries piled on top of that.
Scala became an incredibly powerful language, not a simple or easy to use one however.
Kotlin is a very user friendly language that is easy to learn but still has enough power to keep most expressiveness focussed devs happy.
Beyond the learning curve Kotlin integrates with Java a lot better, calling in both directions.
These 2 factors alone probably account for why it was chosen over Scala for Android and in general why it has value over both Java and Scala.
I think with Gradle moving from Groovy -> Kotlin, Android obviously and big companies adopting Kotlin w/Spring on the backend the ship has indeed sailed for Scala, it doesn't really have a place outside of places that specifically want a more FP lang.
The point about tooling and developer experience I'll concede. It has been at times patchy in Scala in the past but it's pretty good these days.
As I say, I have no hate for Kotlin, and I'll possibly use it one day being a JVM developer familiar with Java as well, I just find it unfortunate that a language which is so similar to the languages it's influenced by was invented.
It can be used as a better Java but requires discipline. Things like implicits can easily cause an unnecessary amount of pain. Kotlin on the other hand implemented extension methods in a way that doesn't detract from the better Java paradigm. This is only one example though, there are many many reasons why Kotlin is more suited for the "better Java" role. Namely it's simply not that different to Java. It's effectively Java with a more modern syntax and polished extensions to the Java standard library.
That's not my truth.
I tried it, after not doing "real" programming for a while, and found it the most obfuscated, incomprehensible language I've ever run into. What new concepts are there in Kotlin that justify it?
"Expressiveness" sounds toxic to me - along the lines of "facilitates a personal idiom that makes your code impossible to understand and you irreplaceable".
After several days trying to do the simplest form of asynchronous processing in an "hello world" Android app, I switched back to Java, and everything made sense again.
I also wrote a small program in C# and that too was just as easy as I recall from ~10 years ago.
But then, I've seen some shit Java too.
I'm surprised you feel Kotlin is incomphrensible though, it definitely can be written that way because it has features that allow for extreme terseness.
I think there is a minimal set of things you need to learn over Java for Kotlin to make sense, nullability operators, block passing syntax and the differences in OOP (inheritance/interfaces are slightly different, classes are closed by default, companion objects replace static methods).
So yeah. Kotlin is strictly larger than Java so it takes longer to learn all of it.
That said because of the look and feel of Kotlin it's more accessible to developers of Ruby/Python/JS/Typescript. This tradeoff is personally worth it for me.
Not because I care that much, I can just as easily use Java. However it's -very- difficult in modern times to convince my team and company to use Java when there is hipster options like Typescript taking all the air out of the room. Kotlin competes in the same area but provides the same battle tested runtime and excelent libraries, it's almost like a compromise between the new age and the old age.
This is why it's such a great language for me in particular but I realize not everyone is in the same boat.
SBT is inexcusably awful (and so is ScalaTest, which is a lot of people's first exposure to the language), and you might be right about the zeitgeist having moved on, but it's a shame; it's honestly a significantly better language design.
Also it is quite common on embedded Java for the vendors to have their own implementation with code translation.
What isn't normal is having stagnant compatibility.