To me every of these sounds like "why don't we put 10ys(tm) inside ZFWs(tm) inside Funnylettes(tm) proxy-edge-borderline stuff so stuff is so messhy and uncomprehensible that you need an IA to explain your application deployment"
Come on, I just want to deploy my app somewhere. I don't want to know about yamls and kubernetes, I just want a button where I deploy my branch.
It often feels like people working on infrastructure and platform development do it for it's own sake, forgetting that they were supposed to be enablers for other teams building on top of it.
"Leaking" infrastructure details to developers is actually less work for them because they are deferring the finer details of the app deployment to the developers who made it. Your request for a magic button that doesn't require you to understand anything about infrastructure is exactly why we have so much complexity.
I think there's a broader point though, in that very often, infra teams will pursue solutions that solve problems in the perspective of their own lens and interests without good oversight from the broader organisation, for developer experience, and economies of scale.
I've seen and worked in environments at both ends of that scale and the gap in dev-ex, and agility as a result can be absolutely staggering.
So often its to avoid 'vendor lock-in', only for infra teams to become the 'vendor', its complex, so they grow by necessity to be expensive, but then still lack the resources to be able provide a clean experience that can be easily migrated off of, resulting in lock-in.
As a dev, the cost isn't my concern, but whats frustrating is knowing that its possible to deploy a new service, with all the bells and whistles, in an hour, and being unable to.
If you agree with the above statement, please go into management.
My most recent example: deploy a lambda function. Building the image: 1 day. Deploying it to prod: 2 weeks.
I'm getting the impression that the whole industry shifted focus from solving actual problems in the most simple way to draining money and keeping things afloat as much as possible.
If that is not possible, you have failed your job as masters of the infrastructure.
At my company, for this I need to copy-paste some terraform in repo 1, write some scaling config in repo 2 and put in new, encrypted credentials and other configs in repo 3, then finally I can deploy my docker app from repo 4.
I can all do it myself in two hours but it constitutes total failure on the infrastructure teams side.
I find it annoying that there are a lot of young (and not-so-young) developers who don't take the time to properly understand where their apps are running in order to make the best of it and also not to write crappy and slow software.
I see people climb mountains in Java when they had all they needed in the database or in the OS.
Either way, you have to understand the solution at a high level, and I think much of the balking at infrastructure is really just balking at needing to understand the fundamental nature of their own problem.
I suppose we all live at the abstraction level that suits us. I'm of the opinion that a lot of the modern infrastructure we work with is over-abstracted. I'm not sure that most developers even know that an HTTP server exists thanks to the work of modern devops where you push a commit and it gets automatically deployed to some sort of container orchestration system. At some level it has to be this way; we stand on the work that was done before us; "the shoulders of giants" it's often said with more poetic flair. So I wonder if it's just me getting older and not understanding the new, or if we have genuinely gone astray in some ways. And those need to be mutually exclusive either.
I guess I've traditionally built and deployed all my apps myself, without the luxury of an infra person, so I had to learn all this, including Linux administration.
It's a wide-ranging skillset no doubt, but still built on the work that precedes it. Did you build the servers that ran your code? Rack them and set up the networking? Build redundant power systems for them? HVAC? I'm sure you've done some of that, but no one person can really do it all. I think the saying is something like, "fish don't know they're wet"; meaning that they're so adapted to the ocean that they don't really have any idea that there are other ways to live. I get that feeling when I hear older folks talk about working on mainframes - it just sounds like an alien world. I came of age in the era of distributed systems - Microsoft vs Open Source, but it was always on commodity x86 hardware and TCP networks. There's no law of the universe that said it had to evolve that way. It seems like the new way is "cloud-native" which I'm not sure I understand or like (as I've said, it may just be me getting old); my biggest problem is that it seems to give ever more power to Amazon and other members of the tech oligolopoly rather than any technical shortcoming.
Maybe it's the same as before, maybe I'm just less familiar and it seems huge to me.
I think there are at least two reasons for this: the first is that infrastructure engineers (e.g., SREs) just think about reliability and architecture a lot more than SWEs and the second is that the infra eng position generally selects for people who have a penchant for learning new things (virtually every SRE was an SWE who raised their hand to dive into a complex and rapidly evolving infrastructure space).
Also, the best SWEs have been the ones who accepted the reality of infrastructure and learned about it so they could leverage its capabilities in their own systems. And because our org allows infra to leak to SWEs, the corollary is that SWEs are empowered to a high degree to leverage infrastructure features to improve their systems.
No it's not. This was simply just not the case before k8s. Running a stateless app behind a (cloud) LB talking to a (cloud) DB has never been the hard part, and it's even easier today when something like Go is essentially just starting a single binary, or using docker for other languages. People seem to have forgotten how far so few components gets you. But incentives for too many people involved aligns towards increasing complexity.
I think most would be surprised how big chunk of modern apps are within that space without active intervention by stuff like microservices. No need to stop at a handful of VMs though, although I imagine that most companies could easily be covered by a few chunky VMs today.
And yes, if you're a PaaS, multi-tenancy something something, then sure, that sounds more like a suitable target audience of a generic platform factory.
And if that's the territory you're in, you need probably need to set up log aggregation, monitoring, certificate management, secrets management, process management, disk backups, network firewall rules, load balancing, dns, reverse proxy, and probably a dozen other things, all of which are either readily available in popular Kubernetes distributions or else added by applying a manifest or installing a helm chart.
I don't doubt that there are a lot of systems that are running monoliths on a handful of machines, but I doubt they have many development teams which are decoupled from their ops team such that the former can deploy frequently (read: daily or semiweekly) and if they are, I'm guessing it's because their ops team built something of comparable complexity to k8s.
One downside I also see compared to the "old-school" approach, albeit maybe an indirect one, is that it's also a very leaky abstraction that makes the environment setup phase stick around seemingly in perpetuity for everyone rather than being encapsulated away by a subset of people with that expertise. No normal backend/frontend dev needed to know what particular linux distro or whatever the VMs were running or similar infra details, just focus on code, the env was set up months ago and is none of your concern now (and I know there's some devops idea that devs should be aware of this, but in practice it usually just results in placeholder stuff until actual full-system load testing can be done anyway). So a dev team working on a particular module of a monolith should be just as decoupled as with microservices. Finally, for stateless app servers, the maintenance required was much rarer than people seem to believe today.
I realize that a lot of this is still subjective and includes trade-offs but I really think the myth building that things was maintenance ridden and fragile earlier is far overblown.
[1] https://cloud.google.com/compute/docs/containers/deploying-c...
I am afraid people is focused on company products that the will forget what infrastructure is.
It’s also worth noting that sometimes the details leak because app teams want them to leak—they want to build some feature that requires some particular infrastructure solution (e.g., we want our app to run some background task in response to requests, and we don’t want to do it in the VM lest the autoscaler—which is background-task-agnostic—kill it while the background task is running).
You shouldn't have to know about the yamls and the k8s, but you should know about infrastructure concerns and how they relate to you and your design choices, just as infra guys should be aware of what's going on on the front side. Having to work with close-minded counterparts who refuse to elaborate further than "make it work now, or else" is about the worse thing there is. Being willing to participate in good faith in back and forth discussions around those subjects is important and pays dividends.