2,220 karma · joined October 30, 2018
I sometimes do that but I'm never able to quit it permanently.
This pretty much describes exactly where we are right now. We've been able to migrate the big customers to a new, less overloaded database server. We could continue to do that. I believe it's what you call a "bridge" architecture, so the compute layer is stateless and can serve any tenant. It's also got a queue/service bus to offload a lot of stuff that the web servers shouldn't be doing. That stuff is all on autoscaling but even that's not a panacea.
So... Yeah. Thanks for denying my reality I live every day.
Upgrading a major version of an ORM for instance. It seems there are certain key improvements that boil down to "the only way this can be improved would result in a big-bang release affecting the entire of the system and take a dedicated team quite a long time to do". It's just not palatable for anyone.
For the parts we carve out as microservices we are essentially able to erase that technical debt. That comes at the cost of increased ops complexity. We're not ready or capable of having 50 microservices from an ops perspective, but we are capable of handling more on the order of ~5.
Perhaps the strangler application approach could work to resolve that aspect over time.
It's a combination of three different problems working against us in concert.
1) compute layer is multitenant but the databases are single tenant (so one physical DB server can hold several hundred tenant database with each customer having their own).
2) We're locked into some very old dependencies we cannot upgrade because upgrading one thing Cascades into needing to upgrade everything. This holds us back from leveraging some benefits of more modern tech.
3) certain entities in the system have known limits whereby when a customer exceeds a certain threshold the performance on loading certain screens or reports becomes unacceptable. Most customers don't come near those limits but a few do. The few that do sometimes wind up blowing up a database server from time to time affecting other clients.
For most of the domain stuff, to be honest I'd like to fix the performance problems and deadlocks by just making data access as efficient as possible in those spots. I think that could get us quite a bit more mileage if we took it seriously and pushed it.
For the single tenant database situation, I don't really know how to approach fixing that. I don't see us having enough resource to ever reengineer it as is. Maybe it's possible for us as a team, maybe it's not. The thinking is that for the parts of the domain we're able to split out, we could make those datastores multitenant.
There's also a bunch of integration code stuck in the monolith that causes various noisy neighbor problems that we are trying to carve out. I think that's a legitimate thing to do and will be quite beneficial.
But yeah... It's a path we're dipping our toes into this year in an effort to address all of these problems which are too big for us to tackle one by one.
In reality there are maybe 1 ~ 3 hotpots in the domain that reveice many more transactions than the rest.
There are many hueristics that are not mental models. There are many things that are frameworks that are not mental models.
Labelling something a "mental model" as used in this context serves the purpose to communicate "here is an abstraction with unusually good power-to-weight ratio that you can leverage to aide your thinking/reasoning about situations that meet a given set of criteria".
It's actually a very specific thing.
Whether a mental model ends up with a name on a list is largely down to it's utility in aiding that for a wide audience.
In my experience they have been extremely useful.
I stopped participating in Christmas years ago. Was very freeing.
I find it helpful to read over them even when picking up a new language / framework to see how it has evolved over time.
Leadership is the other side of that coin.