I always wonder why systems are designed so poorly that it takes weeks of boilerplate to use them for their intended purpose.
At what stage did the engineering fail?
At what stage did the engineering fail?
But the problem identified that is talking to people and having them record data.
The stack certainly could need improvement, but much of the overhead comes from the process being pretty serial because each step depends on the previous. If someone we needs to talk to goes on vacation, sick or has a full calendar this delays it much further. Iteration speed is not linear in such cases. We do a lot of custom solutions and have to implement custom parsers or interface with undocumented ancient software or simply no existing software. Everything is custom, because previously they have no processes, no IT infrastructure or simply only data intended for humans (roughly structured, informally written, ever changing naming schema, typos).
The AI analyses sample data from old and new system, and then builds a prototype deterministic mapping pipeline. The experts can then inspect said pipeline and with their domain expertise catch edge cases/exceptions/uncovered cases, fix them and improve things, and then you use the vetted deterministic pipeline for migration. Yet, the initial AI draft saves enormous time.
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
In a webpage the UI design, authentication, etc. are all “boilerplate” if one just wants to do “X” instead of worry about infrastructure.
I look at stuff like terraform code or AWS infrastructure and it’s annoying and repetitive as hell sometimes but at the same time it’s incredibly impressive that you can just describe something as complex as your own private company’s network existing on global data centers and on a relative level you can just bake in pretty complex stuff like a spiderweb of IAM rules, disaster recovery/backup/databases you barely have to think about, horizontal scaling, security policy and all kinds of other stuff I’m forgetting off the top of my head.
It’s heavy boilerplate because like you said you can basically design an infinitely complex corporation as code and once you’re done with that deploy it instantly which used to be pretty much unheard of.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
That’s pretty much the answer. The simple stuff has already been tried and there’s already some reason why we can’t just make XYZ simpler.
When you look at old-school code that just hacked things together, it was ugly, but also very easy to in turn further hack. There was no elegance or grand base types with 80 levels of inheritance to enable clean extension at every possible juncture of types. And no spending forever to find out the exact function you want to be able to override, only to discover that it's outside your modifiable codebase and wasn't marked virtual. Everything just worked.
What I've seen (at least at one place I worked) is that the code may be very well engineered for further modification - but only by people who have worked quite a bit on that code. For someone coming from outside, it's almost impossible. It takes forever to find your way around, and figure out where the magic place is that you need to modify in order to work within the existing design.
Well, you say, that's what the design documents are for! But they didn't have enough detail. They described the major objects, but not all the ancillary helper objects. They didn't describe the flow of data through the system in enough detail to know what to modify. The knowledge was there, in peoples' heads, but it was hard to get those peoples' time.
So, yeah. "Well engineered", but not engineered for maintenance by people who are not those already in the group.
That's like saying "that's the irony of it, a well engineered bridge sometimes drops chunks of concrete onto cars that pass underneath".
Engineering isn't about creating some perfect thing that is prepared for any conceivable scenario, but about solving a problem as efficiently as possible.