They could use vitess, citus or alloydb. They could use read replicas for read operations and single master in a shard for write. They would get many SQL features (upgrades, referential integrity etc) for free. It would allow them to extend their business logic considerably.
Just wondering if anyone considered using Postgres or another relational db. I understand it won’t do multi master replication as well but it is much more stable and predictable if you give it right amount of traffic. I guess the team had to do that part anyways for ScyllaDB
Great observation, I wrote a little wrongly. What we want ideally is guaranteed delivery to one random free worker. Uuid strategy is better than locking but this could mean that if one worker gets a longer job, all the others on this worker are delayed even if another worker is free.
Thats exactly what we do but taking a lock takes 1 RTT to the database which means about 100ms. it limits the number of events receivers can handle. IF you have too many events, receivers will be just trying to take a lock most of the time.
We use listen notify extensively and it is great. The things it lacks most for us is guaranteed single recipient. All subscribers get all notifications which leads to problems in determining who should act on the message n our case.
A different take: All companies are built on relationships. Large companies are a form of nepo companies where you get paid for a relationship to the management chain.
This is so true! Going after the pie in the sky only works for a short while, then you have to work for your pie. the worst is that employees in windows used to get slower promotions than these new tech groups (live,mobile etc ) which means more of them are in powerful positions to continue this trend. They tasted success with this strategy in their early days when they built sq
L, xbox and word. Azure was the last successful play like this but at least they wrote own software.
I think dbs like etcd mongo Postgres all have some notion of events and notifications. The only drawback is that you have to still poll from the ui if you use these. This is sufficient for most use cases. If you want real time updates, Kafka provides long running queries which keep Kafka still relevant.
If we forget how the current system is architected, we are looking at two problems: First problem is that Linux capabilities are also dealing with isolating processes so they have limited capabilities because the user based isolation is not enough. Second problem is that local identity has no relation to the cloud identity which is undesirable. If we remove user based authentication and rely on capabilities only with identity served by cloud or kubernetes, it could be a simpler way to do authenticating and authorization
I think a admin would better understand the system if there was only one subsystem doing a particular type of security and not two. Two subsystems doing security would lead to more problems down the road.
It could all be simplified and map one to one to your identity provider credentials at a higher level. Having a complicated user system on the servers makes it a problem.
you could potentially create multiple containers in that machine which are single user and give to every user who needs access. CPU/Memory/GPU can be assigned in any way you want(shared/not shared). Now no user can mess up another user.