Time spent profiling and optimizing inherently inefficient technologies is an undervalued factor when deciding what stack to use.
Most businesses fail. Those that don't fail, usually don't have interesting scaling issues. (You can go a really long way on a boring monolith stack.)
So in most cases, whatever gets things out into the world and able to see if the business can be validated makes sense, and then you optimize later. A nonscalable stack that you can iterate on 50% faster is more likely to produce a viable company than a more scalable stack that's slower to work with.
If you're a hired employee, it's easy to forget that the place you're working for is already a big exception just by the virtue of it grew large enough to hire you.
Productivity and Scalability(in performance sense) aren't opposites.
Take Bash. Performs bad and is a guarantee for terrible productivity in a large category of software. But perfect for a niche. Take Java. Performs better than many, and allows for good productivity (if you avoid the enterprise architectures, but that goes for any language). Or take Rust. Productivity much higher than most C/C++ and in my case higher than with Ruby/Rails, and also much more performant.
They often clash with each other. Rust for example is a lot less pleasant to debug than interpreted languages and that is a loss of productivity.
The other reason is runtime speed. A typical Ruby test-suite takes me minutes to run. A typical Rails test suite tens of minutes. A typical Rust test-suite takes < a minute to compile and seconds to run. I run my tests hundreds of times per day. With a typical Rails project, I'm waiting for tests upwards of an hour per day (yes, I know guard, fancy runners with pattern matching etc).
The last reason, for me, is editor/IDE integration: Rust (and TS) type system make discovery, autocomplete and even co-pilot so much more useful that my productivity tanks the moment I'm "forced" to use my IDE with only solargraph to help.
And debugging: sure! I've had reasonable success with gdb and ruby debuggers in the past. Rust's gdb isn't much better. But stepping through a stack in a rails project is a nightmare: the stack is often so ridiculous deep (but it does show how elegant and neat it's all composed!) that it's all noise and no signal. Leaving a binding.pry or even `throw "why don't we get here?!"` also works, but to call that "productive" debugging is a stretch, IMO.
Then I did Ruby+Rails fulltime for 9 years. Just recently moved on.
With Ruby, I'd have to write hundreds of
edge-case unit-tests just to cover stuff that,
with Rust is enforced compile-time for me.
It was one of my major concerns about Ruby, prior to starting out. But like... it just wasn't a problem.
It turns out that we just don't pass the wrong kind of thing to the other thing very often, or at least I and my teams did not. It certainly helps if you follow some sane programming practices. Using well-named keyword arguments and identifiers, for example.
# bad. wtf is input?
def grant_admin_privileges(input)
end
# you would have to be a psychopath to pass this
# anything but a User object
def grant_admin_privileges(user:)
end
With Ruby, I'd have to write hundreds of edge-case unit-tests
just to cover stuff that, with Rust is enforced compile-time for me.
I strongly dispute that you need extra tests for "edge cases" because of dynamic typing. Something is deeply wrong if we are coding defensive methods that handle lots of different types of inputs and do lots of duck typing checks or etc. to defend themselves against type-related edge cases.
(yes, I know guard, fancy runners with pattern matching etc).
The last reason, for me, is editor/IDE integration
But stepping through a stack in a rails project is a nightmare
Those warrant unit tests. Those will regress. Those would never exist in a strongly typed language (though Java still has null...ugh)
It's true that 99.9% of production log errors are NoMethodError exceptions.
annnnnd 99.9% of those NoMethodErrors are just code not handling nils/nulls correctly
annnnnd 99.9% of those unhandled runtime nils/nulls are from external data (user inputs, database data, etc)
So strong typing doesn't help you there at runtime, it just blows up differently.
It really does, though. Not with the Java-type of strong typing (still allows null) but with the Rust type of strong typing. Simply because it moves all this to the edge. At the point where you read the CSV/database/HTTP-response/user-input.
Everything inside of this edge (a strong boundary) doesn't need to to deal with "can this be nil" because it can't. Your `the_outlier(items: Vec<Measurement>)` will simply not compile if the type-checker sees that `items` can be nil, `items` can contain a nil, or, internal to that function an items[].measured_at might be nil, or maybe items[].measured_at is a Date instead of a DateTime.
You don't need a bazillion tests to deal with this situation around `the_outlier()`. That doesn't mean that the part that reads a Vec<Measurement> from a CSV (or json, or database or whatever) is covered by this typechecker. But it means this layer, the edge, the boundary, is where you put the protection. Validation, whatnot.
I had an app once where we used user objects, and later switched to ids to save db calls. Now you have some functions that can accept both, and some that accept one of them, and without type hints (that was long time ago) you can easily make a mistake.
Ruby has had keyword arguments since Ruby 2.0 from 2008 and earlier code could certainly use option hashes.
So I don't see a reason for any confusion there.
# probably malpractice in a system where many methods
# take IDs and some take Users
def do_something_with_user(user)
end
# how hard is this? trivially easy and unambiguous.
def do_something_with_user(user_id:)
end
# this is obviously wrong
do_something_with_user(user_id: User.first)
Primarily it's not the language that makes people more or less productive, though it does have some influence. It's mostly the frameworks in those languages. And traditionally the most modern / full-featured web frameworks haven't been in systems languages. The major counterexample at the moment (while still obviously not a systems language) is that modern JS VMs are actually really fast, so while I don't love JS, it does hit that sweet spot at the moment of performance and mature frameworks.
Also, I've never worked in Rust, but am mostly a systems programmer, and while I understand that Rust is supposed to be easier than C or C++, I'm skeptical that it's as easy to work with as higher level languages, or that you could throw most web developers into Rust without some serious additional learning.
That's another problem I have in this narrative. Productivity isn't measured by throwing an inexperienced developer at something and then looking how fast they get stuff done. That's learnability.
I'm an experienced Rails developer (some 15 years in) and my productivity has plataued for years now. I've been doing Java and Rust work for years too now. Web and application dev. It took years, but my productivity in both Java and Rust, on anything that lives longer than 6months, has vastly surpassed that of my Rails.
Productivity of a senior, or experienced dev, of a (large) team, of a team with high turnover, of a project over decades, all that is productivity too. And in all those categories, Rails isn't great.
That's all correct.
But the point I'm making is that if an MVP isn't accruing technical debt, it's over-engineered. Most of them will be thrown away, or rescoped, and so taking on technical debt is an advantageous strategy: you only have to pay the technical debt on the few survivors.
Shopify at its offset was a CRUD app (fun fact: it started as a snowboarding shop), and in 2006, Rails was a great choice for that.
Your notions are fine for an established company building a piece of infrastructure they're certain they'll need. But that's not what Shopify was, and it's not the spot most startups picking a framework are at.
Your thing about developer quality is kind of meh. Building the first versions of a shopping platform isn't rocket surgery. You don't need Anthony Bourdain to make a sandwich. Particularly if you're not sure anybody wants a sandwich.
What I'm arguing, however, isn't to take on debt¹, I'm saying that productivity and performance aren't always opposites.
Sure, Rails trades in performance for productivity. But, I've learned, this is mostly just Rails. There are many languages and frameworks that are just as productive (for a certain definition, see previous comment) as Rails, but also performant. And I'm arguing that performance affects productivity: a performant, scalable software is easier to work on, because it gives faster feedback (tests, ci, manual testing), wastes less time waiting for stuff (hundreds of single seconds add up over days and weeks weeks), and decreases friction (I'll postpone running the full test suite if I know it'll hog my machine for the next half hour. I'll gladly run it, if it takes a few minutes or less).
Edit: and, if what you say about tech debt is true (I think it is), wouldn't Shopify be at a position now to pay it back? Many startups that used Rails paid it back by migrating elsewhere. So maybe Rails in its entirety is Tech Debt?
¹A cautionary sidenote, that I've learned the hard way, is that taking on tech debt is an art in itself. Not all debt is alike. Many kinds will cripple my project. Where at the unlikely moment that I do need to scale, that's impossible. Or when I do need to pivot for the umpteenth time, we cannot, without that Giant Refactoring.
They don't need to do any of this. The product is fast enough. They make money. It's purely to fatten the bottom line.