Hacker News Reader

Comment by coffeecloud | Hacker News Reader

Scala is an organizational red flag for me, especially if their recruiters are looking for "scala engineers."

Same.

It's a fairly accurate predictor of a company downturn after getting sunk in total complexity and ensuing certain employee churn.

The only reason for Scala adoption was Spark 10 years ago. Scala/Spark engineers were pulling the top buck. With pyspark and other cloud big data options on the scene, no one sane is using scala for this anymore.

> after getting sunk in total complexity

Plus, the cost of teaching Scala to your new hires means they won't be productive for literally _months_

I agree that other big data tooling providers are eating into spark but I'm surprised people are choosing python over Scala for Spark.

Spark wasnt even released 10 years ago :) I think Twitter, Linkedin, and some others gave Scala adaptation big push. Twitter in particular, when they killed their fail whale by moving to Scala/JVM

What did you think of the organisation described in this HN post?

BTW, the described organisation does not hire "Scala Engineers", but Software Engineers. Scala is just a useful tool.

Same. Scala is the worst of the category of "functional looking languages built on the top of an imperative ecosystem".

I think F# is doing much better in this category.

Scala can surprise you many unexpected ways. https://www.youtube.com/watch?v=v1wrWQcqLpo

F# has some nice features that scala is lacking, such as a more terse syntax for ADTs or type providers.

But other than that, Scala is miles ahead. And if you want to do pure FP, then Scala is the only option.

Also about that video, almost all of the points mentioned are either not available in other languages like F# anyways, or they already fixed in the latest Scala version.

> And if you want to do pure FP, then Scala is the only option.

I don't think that Scala is especially good for "pure FP".

I also do not want to do "pure FP".

> I don't think that Scala is especially good for "pure FP".

I meant when compared to F#. Of course there are other languages for FP as well.

> I also do not want to do "pure FP".

That's your choice!

> Scala is the worst of the category of "functional looking languages built on the top of an imperative ecosystem".

Could you elaborate? Because that's not been my experience at all. I have found Scala to provide many tools to wrap around the idiosyncrasies of the underlying imperative that they are barely noticeable anymore.

I found F# really good at with this -> "wrap around the idiosyncrasies of the underlying imperative".

In the video Paul Phillips elaborates on many subject much better than I could ever do.

I also found the .net ecosystem libraries better, even though there are fewer libraries available.

Some links that I found great on the subject:

https://www.reddit.com/r/scala/comments/4246qc/heres_why_sca...

https://techblog.bozho.net/i-dont-like-scala/

To be honest with you my social circle of software engineers gave up Scala long time ago and we do not even consider it to try it ever again. Most of the projects we do we simple use Java for JVM (since version 8 there are many useful features that you can use and have relatively terse code that is also easy to read) for the .net ecosystem we use C# of F# depending on how relaxed the engineering team and the stakeholders about it.

For data engineering (ETLs, glue code, etc.) Python works. Since Spark has a Python interface it is also not needed to use Scala there.

For frontend obviously Scala was probably never a serious option.

There is no use for Scala for me or for the companies I work for. The last time I tried Scala it was horrendous. SBT? Seriously? Binary incompatibility between minor versions? I could go on an on of the collection of anti-patterns I have encountered. Since I have limited time and there are a ton of other software environments that we can use and have much better experience why on Earth would I want to waste time on Scala ever again?

Paul Phillips misses the point, and the "more than one way to do it" complaints are superficial. There's no language that prevents you from writing two functions that do the same thing, and that's all that most of those examples are (e.g. a -> b is just calling a function that returns (a, b)). Most of the time there aren't multiple semantic ways to do the same thing, which is what matters.

SBT is awful, don't use it.

"Minor versions" is overstating things - 2.11 was released in 2014, 2.12 was released in 2016, 2.13 in 2019, and 3.0 onwards are backwards compatible with 2.11. So that's one compatibility break every 2-3 years, which is better than most languages.

Scala is the only vaguely mainstream language I've found where you can do a whole enterprise-scale project in the language itself - no reflection, no AOP, no decorators, no macros, no monkeypatching, no nothing. So until I find another language where I can do custom context-like types (e.g. a type to represent "this must happen in a database transaction", and libraries already offer me functions like "take this list of database actions and combine them into a single database action" that work with my custom type) and "walk the object graph" style tasks (e.g. JSON serialization) in plain old type-safe code, I'm sticking with Scala.

It's not experience with Scala at all, but we all have different journeys. Should you ever find yourself using Scala again, I hope you make the same pleasant discoveries as I have.

Thanks for elaborating your point of view.

It really should be. When you're trying to solve a business problem, Scala gets on the way.

And when I say Scala, I mean libraries like akka http, spray-json and slick. I like FP parts of Scala overall.

You would think that Scala 3 released in 2021 would have first class net/http and json that we can get rid of spray json. But no.

I worked at a company full of Scala zealots and elites. It was insanely unproductive and people spent a lot of time trying to design around all the floors in Akka, trying to get Akka working in modern cloud computing environments and the like.

I think at some stage specialists from some Akka pro shop had to be air dropped in to try fix it all, but it didn't help much.

It always made me laugh how well all the services written in Python and Go worked and how easy they were to extend, operate and deploy comparatively.

My bad experience with Scala has more to do with the FP side (cats and all that). Akka has been quite pleasant to me, so I'm quite surprised by this.

Is the business problem not suited to the actor model in the first place?

I've never seen a case where the actor model actually helped with the business problem. Every time I saw Akka used, replacing it with straightforward Futures-and-for/yield code was an improvement.

Cats et al are frequently abused - 90% of the time rather than using a complex tool to solve a complex problem you want to think about it a bit and simplify it. But when you need them they do things that you can't do any other way, and they're vastly better than the reflection/AOP/bytecode-manipulation that people use to solve the same problem in Java or Kotlin.

> I've never seen a case where the actor model actually helped with the business problem.

I've inherited and built some actor based applications and libraries, both in Erlang and .NET (via Akka.NET). Usually these systems are:

- heavily Domain/event driven

- have 'funnel points' (example; bids on specific cars)

- have strong (but not extreme) requirements around consistent, fast response.

- be reasonably maintainable without a whole lot of gotchas

In all cases the actor model was the easiest way to solve the problems presented. That last point may be a bit contentious, but what every shop I've been at has found that while sometimes domain evolution results in extra boilerplate code to handle different versions of events, it is usually not the same sort of nightmare to do larger redesigns of the system as it ages and needs evolve.

> Every time I saw Akka used, replacing it with straightforward Futures-and-for/yield code was an improvement.

I'll admit I've over-used Akka.NET from time to time. I'd say the most overkill case is 'I need background workers running on timers with automatic recovery.'

Not using Akka, but I have a side project right now being done primarily in Erlang that felt natural to architect with Erlang “actor model” and the OTP (not the only reason I chose it of course, Erlang’s history and certain other features from that history made it a natural fit). There’s a small JVM-integrated component as well, but I’ll likely just be developing it in Java as it’s really just involved implementing an old JSR and I don’t foresee more advanced feature coming too much in handy.

Can’t speak to category theory. Not familiar with it, but I have done my fair share of “reflection metaprogramming” that might get me me shot (dynamically constricting and using generic types through reflection gets pretty nasty).

Same here, Akka is really amazing

The standard library is where modules go to die, just like in Python. And unlike Python, there's a good build/dependency manager available for Scala (Maven). Keep that stuff out of the core language, let it be libraries released on their own schedules.

JSON has a standardized spec and the API you design has to do a few things: Marshal and unmarshal.

How frequently do these need to change?

The problem with the library landscape is that one library will be promoted by the purists and another by the company with the money. Suddenly you have to learn both to understand Scala codebases.

Well literally today I got a notification from GitHub that I need to upgrade the version of jackson-databind that one of my project depends on because of a security vulnerability. And apparently that's the 8th such fix to this patch version of jackson-databind.

Well, you can always use tried-and-true Java library like Netty, Jackson, and plain-old JDBC API. You get to enjoy all the goodie like `val`, `case class`, `pattern matching`, etc... without all the headache of those pure functional libraries.

Those are the least confusing libraries in the Scala ecosystem lol. Slick is just like any ORM-light framework. Akka-HTTP abstracts away all the Akka-specific things and you just create routes that return Futures essentially. How is spray-json complex? You're just using macros against your case classes to create (de)serializers.

If anything, you should be saying things like Shapeless provide the extra complexity.

> Those are the least confusing libraries in the Scala ecosystem

Now that says something about the Scala ecosystem. Doesn't it?

I'm comparing this to how easily I can do the same task in nodejs and Go.

Go is at the same level of complexity. JS I can agree with but we're starting to compare completely different things at that point. Type systems make things more complex at the benefit of safety.

Having used both Scala and Go for the past 6 years, I can tell that for the purposes of creating a database backed web API, they're definitely not at the same level of complexity.

This isn't about the type system. This is about complicating things by making everything FP while getting nothing in return.

That's the great thing (and the point of many of the comments in this thread) about Scala - you don't have to do any FP with Scala if you so choose. You can operate in a completely OO fashion and gain all of the benefits of the language (primarily the type system) for free.

People promoting this view fail to understand that people work in teams and it only takes one FP zealot to force everyone to do FP and your PRs don't get merged unless it's idiomatic Scala.

Why not just use Jackson? Sure, it doesn't have a clever DSL. But then, that's a feature.

Jackson has its own parallel "type registry" to make up for not being able to use something like Shapeless, because the problem still has to be solved. Getting a failure at compile time if there are any cases that wouldn't serialize properly is a huge advantage; a nontrivial proportion of production outages I've seen were Jackson.

I'll be honest, I had a quick look at Shapeless, and couldn't see how you'd apply to ensuring an entity serialized correctly.

But I'm keen to learn more :) I always prefer compile time to run time errors.

What Shapeless gives you is the ability to treat value-like types - case classes and sealed traits - generically. And so you can do "walk the object graph" style stuff at compile time - particularly, typeclass derivation.

https://circe.github.io/circe/codecs/auto-derivation.html is one popular library that does this - note that the `.asJson` will fail at compile time if you add a member to Greeting or Person whose type can't be serialized/deserialized (e.g. File or Lock or something) i.e. one that doesn't have an Encoder typeclass instance available.

Okay, that's pretty cool. Cheers :)

These decisions aren't made by me.

can never convince the FP purists to let you use a non FP library.

Reply on news.ycombinator.com