Ask HN: How to get good at software system architecture
I would like to have the competence to choose the right kind of technology for example like AWS lambda for different components of a software system
ps. I'm the author :)
[1]: https://github.com/mehdihadeli/awesome-software-architecture
Also, often choosing the 'right' technology is less important than designing an architecture that is both simple enough and capable enough.
When doing this in my career, I am always considering can it be more simply (i.e. less complexity is easier to grok and is usually less prone to fail) and can it be done more clearly (i.e. such that the intended use of the architecture is obvious).
Try to think of a project that either needs infra like that to get started, or another one you can deploy in a homelab or a few VPSes or something.
Maybe VMs are a good idea, to give yourself a good place to test things without committing bare metal to anything.
What software system architecture interests you? The scale you're interested in may matter, too. For me, system architecture means which distro (and which set of packages), configs, and other settings that create a cohesive server/client or groupware system. I think a lot of those challenges lay mostly in understanding how computers are being used within a given organization, and designing things in ways that make adapting easier and workflows smoother.
What that looks like, I would imagine is different for every organization.
I'd be surprised if you could simply read a book or watch a video and learn how to architect complex software solutions, honestly. For me working at different orgs and being curious about how all the various layers of the software stack fits together has given me a decent perspective on different approaches to problems.
Also, "good" here is somewhat relative. Being "good" an architecting problems for large orgs operating at scale is far more challenging than being good at architecting solutions at a small startup.
Smaller, younger companies generally don't need anything super fancy or complex, what's important is getting things done so typically you build simple and worry about doing things right later. Some basic AWS knowledge and a strong coding background is probably all you need.
Larger, older orgs are far more intimidating because there's so much complexity already in place, and often there will be entire teams built around managing that complexity. In these cases you need have an appreciation for what already exists and build things that complement and improve on that. You also need more experience with building at scale. At a small startup whether you decide to go serverless or not doesn't really matter, but at a large org with tens of thousands of daily users the cost of a bad decision grows exponentially.
Another thing I'll note is that if you're doing this for career opportunities whether you can do something doesn't matter one bit. Companies won't hire you unless you have a track record. If you want to become a technical architect I think the challenge is mostly knowing how to play the game in such a way that you can take on those responsibilities at the place you work. Learning how to architect new solutions isn't that difficult if you're curious about how things work.
Do the simplest thing that works, if that is hosting something on a VM with SQLite for persistent storage than that is what you implement.
If what you had in memory was a map or a queue and you don't care about persistence then redis is likely going to serve you well when you need to scale to more than one instance, if you never need more than one instance then keep it in memory as a map or a queue.
Don't add things "because that is what :bigtech: does" :bigtech: is not you and if it was, you would have the team to do it or would be building the team and not the software.
On the other hand, try not to paint yourself into a corner, where you can't grow (if you do) without a major restructuring of the architecture. This takes having some idea of what the problems can be, and what people do to handle them. You don't have to implement those things yet, but try to build what you build in a way that will be friendly to those changes if they happen.
An example is for crud, use the (onion|hexagonal|whatever you want to call it) architecture, you business logic has nothing to do with how the data is stored persistently (or not), build your application using abstract APIs.
It's slightly more code up from but if I was using a map in memory it is trivial to move to using redis if I build with the abstractions in place to begin with.
Generally in the prototype stage I won't care about that either but I expect the prototype / poc code to be deleted.
Same idea with SQLite, moving feom sqlite to a full blown database isn't that difficult but this is one of the places where I would conceded and say if you have the infra in place just use what is available, it may be easier to get a DB on thr current cluster in the long run if there is already a cluster that you can get a DB on.
If you however are in a completely greenfields position and really have no idea where the product is going, sqlite scales to a decent amount of RPS and so does a simple go/rust server on a host.
Many webservices really don't need more than that and not worrying about being distributed when you don't know if you ever do need to be distributed makes whole hosts of problems trivial to solve and allows you to use technologies that would usually be inconvenient.
I'm no expert on system architecture, but I have noticed that reading a lot of docs and blogs on how other companies do things successfully has helped me:
Quastor: quastor.org - Engineering blog posts and summaries
Engineer's Codex: read.engineerscodex.com - Case studies and summaries of Big Tech systems and software architectures
SWE Quiz: swequiz.com - Bunch of useful questions that has helped me a lot at work
TLDR Web Dev: tldr.tech/webdev - Good for keeping up on engineering-specific stuff in general.
I have a Feedly set up with these blogs too: https://blog.bytebytego.com/p/79-engineering-blogs-to-level-...
ByteByteGo is good too, though that's an obvious one that everyone already knows about.
You might ask, how about 20% of the rest ?