Cost only goes up and up when engineers go overboard with "clever" Ruby magic... which is human error, don't blame the tool.
I've only used a few languages in production for my non-developer job, but I've noticed very different frequencies of "clever" code between them.
R: This is my primary language. I love it, but it almost encourages clever coding. There are 10 ways to do any task without even touching third-party packages.
Python: My secondary language at work. Clever coding is definitely possible in Python, but the language's syntax makes it easy to spot. Detection isn't as good as prevention but it's better than nothing.
(I also grudgingly use SAS, but won't waste my time looking for ways it encourages clear coding.)
And to be fair, most of the premature optimization and bloated abstractions are just your architects sitting around bored for 6 months while the project sponsors argue over reporting minutiae (but won’t approve the designs until their pet features are pulled in on the roadmap). They know the entire feature list is going to get changed at the last minute as the political winds shift, so they’re designing an overbuilt architecture to CYA for whatever random bullshit someone will pull a week before a deadline.
I'll grant that it is easier in more stable API environments. But we are our own worst enemies in that race.
In Java? Not really. Spring has been around since 2003 or so.
I honestly can't compare to a rails app, as I don't have the experience. I can say that modern dev practices worry me with the constant upgrade train on api standards. On the one side, progress is quite nice. On the flipside, what you describe in "all your dependencies have completely changed their APIs" is just as annoying in Java as it is in any language. More so if someone has helpfully split your codebase into as many supposedly independent parts as they can.
More than that, it is amusing that many of the refactoring tools that you reference actually began in more dynamic languages. I've seen that folks that rely on their IDE to be able to refactor have a habit of making codebases that they have to use an IDE to refactor. I grant it could just be confirmation bias.