star_juice

The heavy usage of 3rd party tools does sound about right wrt what I've heard get used to vastly expand apps after they've got a minimum viable product people actually want to use though, particularly the usage of CDNs to reduce latency. However I haven't heard much about the details regarding ssr/issr vs ssg though, so if you have any reading that elaborates more on it I'd love to take a look.

Additionally, taking this scaled t-shirt app approach as a template can you share any insight into how a more complicated platform (eg Youtube, Amazon) would further leverage such in-house/3rd party services to most efficiently use their resources (ie at the fully mature platform stage)?

So a lot of the scalability complaints arise from the fact it's written in Python, but as far as I can tell they're largely newer developers trying to grow some web app they made rather than people presenting at PyCon about why it's awful. Fortunately I am not trying to bootstrap a business from this thread, but I agree that it's far better to spend a few weeks getting my hands dirty with some framework or language that I'm familiar with than just reading about all the cool things I can do in some other language or framework that I need to still learn.

Is it possible you could share more specifics about these simulation figures? While I'm still very much pursuing technical/organizational info about what underpins the big scalable services (or the meta-scalable ones like firebase, which itself helps other devs make scalable apps) I'd be very interested to know how stark the difference in cost/performance is for this managed crud/firebase-heavy solution vs other offerings, or even just as a baseline of whats out there in the space of easier-to-architect scaling options.

I've heard this book (assuming it's https://dataintensive.net/) recommended before, but some have mentioned it's overly theoretical and doesn't offer much in the way of helping you build working examples of the material (although I suppose the complexity of the material does make toy examples a bit tricky to come up with). Do you know of anything that does have such offerings, or possibly if I'm thinking of a different book?

I've heard it recommended before, but some have mentioned it's overly theoretical and doesn't offer much in the way of helping you build working examples of the material (although I suppose the complexity of the material does make toy examples a bit tricky to come up with). Do you know of anything that does have such offerings?

Ah gotcha yeah at the small scale(?) scaling we're talking about monolithic applications, being a bit simpler to organize and run, do still make for a compelling solution. That's a great tip regarding how to handle ballooning storage issues via managed cloud offerings (in the weird case I make something that really works) that I hadn't considered. However, it's starting to feel like these scaling questions/solutions are a lot more akin to Factorio bottleneck chasing than I would like haha.

I'd actually been thinking about giving golang a try after seeing some very clever usage of concurrently operating Go routines, do you know of any particularly interesting server examples you could share?

As for the homework...

naive answer: constantly update every T shirt app instance with a record of the current stock, as every transaction completes, and then make sure the system all agrees before allowing additional transactions. Sort of like a really inefficient internal distributed ledger.

horizontial/vertical scale agnostic answer: split the stock tracking into two separate services. One that is focused on managing transactions/maintains the number of T shirts (presumably a hash table for different types of T-shirts with their quantities), the other being updated every time that inventory table changes. The first service can probably be further decomposed into constituent services but for now we'll keep it simpler as a single big one. At page load time the app instance can send a GET (or equiv) request to the secondary service to get quantities of all the shirts on a given page (using matching item ID/hashes) that is then cached in something like localStorage, with ones in limited supply or out of stock denoted as such by the browser/app on render. On a transaction completion you send an update request to the primary stock service, which then begins its process of updating the table/passing the updated table to service two. This will allow customers to add products to their cart with a much lower chance of finding out some are out of stock at check out, but also (hopefully) keeps the stock tracking footprint lower than something like a constantly polling no-matter-what or monolithic tracking system.

Given the bottlenecks I can already see with the checkout transaction part, I'm actually extra curious how to improve it, so I hope you'll share what the exemplar solution would be!

Alright, you win: this answer is fantastic. This is a far, far better way to think about what limits scalability than simple things like pages served per millisecond. I never did expect to find ready answers that will guide me to making the universally scalable app, but now I see the problem can be reduced even further into niche sorts of scaling which, while solvable with hardware tricks, do their very best to escape generalization.

Thanks! Do they include coding examples or is it advised you think of your own wrt the material?

This sounds right up my alley honestly, could you explain a bit more about the affordances you set up for the decoupled container component to allow it to grow from a few containers to an orchestrated K8s cluster? Just some insight into how and when one decides to grow from a container or two to full on clusters would be really good to know.

As for sharing, while it's very generous of you to offer access and I'm certainly interested in using it once it gets a public launch, I'm much more in pursuit of the thinking behind system design choices and better developing the intuition that makes those choices. Honestly the framework you're putting together sounds like it could help a lot of people with similar problems to me though :D

Given I have zero intention of making it any of those things, and this question was aimed at specifically learning more about the technical underpinnings of the rewarding part, I'm not really sure it's the right advice for what I'm doing. You will note that I indeed recognize it as being good advice in general though.

The generalizable advice about data co-location/data sharding is definitely something I will keep in mind (if this weird learner project really involves data in such quantities) however, thanks!

That could work for a smallish CRUD application I think (just using the function as a service approach to really squeeze efficiency out of the compute time), but yeah the specific numbers aren't quite as important as learning about how to acquire a scalable fraction of the power behind the massively distributed platforms that now dominate the world. This seems like one such approach that some devs have very definitely made working services from, based on a quick search.

Technical debt is a made up problem and cannot hurt you :,(

All of this is entirely fair and worth considering, given pretty much all mature frameworks do tend to have some extension or build out option to make them "scalable" as far as I can tell. Now to choose one that's not Django, which I've heard is a bit of a nightmare for this sort of building out (but maybe more because of the problems you mentioned than anything wrong with the framework itself?)

Thank you for the fleshed out answer! These are the sorts of considerations and solutions I was looking for. The first scaling step is probably the most informative of all though, is it possible you could point me towards additional reading about how developers have handled that first scaling step/the decision making behind what should be prioritized to go in the limited cache space (naive example: localStorage in js). Even if you can't, I really do appreciate what you've shared already!

Ah that makes a bit more sense. Could you possibly point me in a more specific direction, eg a current system design guide aimed at developers just trying to get their feet wet with things more complex than an SPA? Yeah, that issue is one I've noticed; you're expected to sort of pick this knowledge up from a job but to get the job where you become acquainted with it you're likely gonna need at least some relevant experience with it.

So, as I mentioned in the other reply I'm not really trying to start a business here I just want to understand more about the processes and tools underpinning these vast scalable web apps that surround us. But this advice sounds pretty reasonable relative to how one would actually go about creating a small tech startup and iterating on it enough to get into YCombinator (for example).

Given it doesn't sound like you're gonna share the info about scalable infrastructure, could you possibly provide some guides or extra reading for the patterns and practices one should be going with at the the small and feisty stage? I might as well take some notes down about this side of things if you have materials that speak to it.

That's why I put the range starting at 1000 simultaneous users funny enough, but I also wasn't sure if 100k was comically low (I suppose if you just wanted to serve static pages with a jpg and one line of text it might be) or high so left it in. That's already some insight to work from, but you may be mistaking me wanting to serve 100k simultaneous T shirt buyers with an actual expectation of needing that much margin: I don't, but I do want to know how/why the back of store infrastructure for such a large number of users works, and more about what a solo developer/small team would realistically want to choose instead.

To be clear I am not trying to create a business, I'm wondering what the ecosystem of best practices, tools, and frameworks to solve an open ended problem like this is so I can try them out before trying to build a real one (which I think you're right, would require an actual business to justify its existence) For example: could you elaborate on some possible benchmarks to try reimplementing, to this end?