I don't want to be like "that guy" or anything but there is definitely a racoon in there so like 14,332 cats and a racoon.
78 karma · joined December 19, 2013
I think there is a big jump from the tech literacy to install Arch and the tech literacy to maintain an Arch system over time. These derivatives feel like they are there to help solve the latter.
They don’t even mention their product till the very last section. Overall think it’s an excellent blog post.
Legit made me chuckle
I think one of the big tools is Shuffle Sharding. The article talks about standard sharding by itself as not being enough to provide robustness in multitenant queues. But Shuffle Sharding I.E. assigning users to virtual groups of underlying queues and enqueueing to the queue with the smallest size gets you pretty far. It can limit throughput for individual users but implementing some simple work stealing logic on the consumer helps make sure you keep your throughput up.
Has some good visualizations as well.
Since many of those use cases probably already run extensive on-prem infrastructure this could appeal to them. AWS outpost talks about industries like healthcare, telecom, media and entertainment, manufacturing, or highly regulated spaces like financial services. I've heard of media companies that process through things like IMAX cameras that have just tons of TB's of data sometimes just for 5 minutes worth of footage. That would simply be too cost prohibitive - in bandwidth alone - to try and move around in the cloud and you don't want to have to wait for things like AWS snowball or whatever.
While I think the space is "niche" those niche spaces are not small. Big companies with big budgets.
I saw in your socklimit project looks like `fasthash64` and `hashlittle` I'm not familiar with those any insight or recommended reading to understand these hash functions?
p.s. googling pairwise independent hash functions did get me some college class reading but doesn't mention any named hash functions developed out in the world.
https://techblog.wikimedia.org/2020/10/26/impact-of-using-ht...
TLDR they used envoy proxy as a sidecar container. So to your point its a pain but still achievable
It's not stored in searchable text rather they simply have some heuristics that they modify the password submitted and retry automatically.
Not sure what about terraforms performance is so bad. Seems hard to blame a tool who's main execution path is potentially 100's of network IO requests with 3rd party API's. Most of the "split stacks" I've seen is more for code organization and security reasons rather than performance. Seems safer to know 100% that deploying infra for my app isn't going to mess with my VPC settings and can be executed with a lower privileged role.
> Furthermore you can no longer effectively plan any change that affects the boundary between the two stacks.
That's fair -- you do end up with these "foundational" modules a lot of the time. Like an 'aws-account basics' module or something that other modules expect the account to be setup with that base for being able to query data objects for subnets ect... planning changes if that changes be difficult but not impossible. Good versioning is critical. Feels in the same vein as apps that need to manage framework updates and things like that. (though can be made more difficult or easier based on how you've broken up using your cloud provider -- multiple accounts by buisness unit or all in one).
But if you're already in the cloud most of the k8 solutions already exist as individual tools, with little operational maintenance needed. "I need 4GB RAM, 2 cores, I need access to the postgres secret to talk to it, and here is my hostname" is all achievable with lambda's or ecs fargate task definitions and parameter store. Managed services for queues and databases lower operational burden. All of this while not having to risk a cluster upgrade bringing everything down or now having to increase operational overhead by running multiple clusters to have some operational blast doors.
All of this to say if kubernetes is working for you thats great! Congrats. But I'm on the hill with OP. Most platform/infrastructure engineers that I've heard pitch kubernetes because they are trying to solve one problem I.E secrets management or deployment ect... not all of the problems at once and bring in kubernetes complexity when there is already a fully managed, safer and cheaper solution to migrate to doesn't usually make sense.
There's no 100% right or wrong answer but I think you really have to evaluate what you're migrating from when evaluating moving to kubernetes