Level 502 already? Who knew the Gateway class would be so easy /s
756 karma · joined February 17, 2023
Event Sourcing dictates that Events are never deleted which means that the data volume keeps growing and growing. There is - in this system - no way to delete old events. When I brought this up, the response was "Just add another hard drive". In the modern Cloud era, adding a hard drive is extremely expensive.
The system uses CQRS and all events generate reports that are stored in Elasticsearch. Data is never deleted, only an extra event gets added /saying/ it's deleted. The data is still cached in Elasticsearch. All of it, all the data back to 2013. Added an extra 32GB of RAM just to keep up with it is ludicrously expensive.
We're in Europe. Guess what a system like this does to GDPR. Can you tell me which events I need to delete when somebody says they want to be forgotten? Yeah.
I can't delete old data. It's impossible without collapsing the entire system like a house of cards.
Finally, and this is the piece de resistance, the developers decided to develop a relational database structure ON TOP OF EVENT SOURCING. We're talking primary keys, foreign keys, cascading and non-cascading deletes. Importing an Excel sheet of 10000 rows takes TWO WEEKS because it generates hundreds of events per Excel cell that is being read. We brought it down to 10 minutes and there is plenty of room for improvement it's just that we have other priorities right now. Currently, simple flat "tables" with no foreign keys take several seconds to import (just like in a regular RDMS).
Oh yeah note that this is a system that is used by 4-5 users at a time, not hundreds or thousands of users.