I'd caution against the term "over-engineering." It is a straw man[1] — no one is pro-over-engineering — so it is only ever used to discourage long-term thought. It's a good thing to try and avoid, but maybe pick something neutral like "predicting the future."
Hacking, on the other hand, is sometimes advocated for (e.g. Facebook) and can be good or bad depending on circumstances.
I've had to argue against over-engineering many times in my career. I wasn't about discouraging thinking. I was fighting against actually implementing specific features that attempted to solve poorly-defined problems whose details had not yet come into focus. Developing those features would significantly impair other parts of the project.
For example, the problem might be "What if our database becomes overloaded one day?" Choosing to implement, say, sharding, could lead to an over-engineered system which gets in the way of solving the many different future performance problems that could crop up.
I think it's quite reasonable to use the term "over-engineering" in this sort of situation.
In your example, if you tell the other person that their database sharding is over-engineering, they can say one of two things:
a) No it isn't. b) Yes it is and I think over-engineering is a good thing.
I have never heard someone say (b); if they thought it was over-engineering then they wouldn't have advocated for it. The negative quality is part of the definition.
What you're likely disagreeing on is how far in the future you're team should be thinking or how much data is necessary to make a confident decision.
I don't think it was your intent to stop long-term thinking, but if someone says "we shouldn't over-engineering things", I don't think that solves any ambiguity because no one was intentionally over-engineering before.
It's akin to saying "don't do bad things." People never do bad things, only things that are bad under a different set of principles or based on other values.
My feedback is that this document should define what over-engineering means in their context (since it, like what is "bad", is context-specific) and lay out what engineers should look out for.
Absolutely, and that's why I agree that the term Overengineering is an unhelpful shorthand that implies criticism to an unnecessary extent.
Stating that you are against over-engineering makes it explicit and as a value it is something you expect your engineers to think about. "No over-engineering" should sit at the heart of any good set of coding practices and it should be there because it means engineers have to think about and understand what it means.
"there are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies; the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." - Tony Hoare
I completely agree that over engineering that takes the form of "this feature will be cool, lets build it" without validation or demonstrated need from the market is generally a bad idea.
However taking time to understand the fundamentals of a system and build a robust core on strong principles is almost always a good idea.
Not exactly.
The term is used to discourage long-term thought that almost certainly won't pay off. As such, it usually will cause more trouble in the long term. So it's not just a failed investment with a missing long-term win, but an "investment" that will cause a long-term loss.
Over-engineering usually happens when people either over-estimate their ability to predict the future. Or, when they just learned a bunch of exciting technologies or patterns, and drift into some kind of hammer/nail thinking.
It is when you hand them a blank check and allow more engineering ot justify writing down a bigger number.