If it takes you 100 tries for a moderately complex jenkins pipeline, you have other problems that are not jenkin's fault.
Or do you mean systemic corporate problems? In that case, I agree.
It still doesn't change the fact that Jenkins does not make my job any easier. I'll spend a day worrying about Jenkins idiosyncrasies ("why can't I use a pipe in sh", "why did my bash escaping disappear completely", "why 'dir' doesn't work with a container build agent?! (JENKINS-33510)", "why this input plugin won't work with blue ocean", "why can't I use a for loop in this piece of code in particular but it works elsewhere" (JENKINS-27421)).
Whereas with concourse or other newer build systems I can write a simple YAML description, which is modular and uses an existing standard, and test that in isolation. And then provide it as a building block for other tasks.
Welcome to the world of enterprise Java or .net programming. Loads upon loads of crap. Best served with multiple frontends (e.g. web + mobile) which need different npm versions to compile and all of it out of a single fucking pom.xml which is a nightmare in itself!
You have an excellent point. Individual microservice containers are not complicated (then again, all they do is call a standardized script). The script will run a Dockerfile and push it to the registry. I would classify it as a 'trivial' Jenkins job, not even pipelines are used.
The pain starts when you want to do more than CI and try to get into CD. Or even worse, automate 'devops' tasks. That's where you run into all those warts.
A job could call Terraform, or spin up VMs, or run vacuum on a database, or any number of tasks. Or it may perform tasks on K8s to deploy a complex app. It may need to call APIs to figure out where to run things. And so on.
Since Jenkins is not only a CI/CD system, it can do anything, so people will try to make it do increasingly complicated stuff. And I'm arguing that this is wrong. If you have complex logic, it should be moved out of Jenkins so it can be more easily maintained and tested, and dependencies isolated. One of the easiest ways to do that is with containers. At which point, Jenkins loses most of its usefulness and other, newer tools shine.
Alternatively, use more specialized tools. If it is for CD, and Spinnaker works for you, please use that instead.
However, deploying containers to environments like openshift and kubernetes is extremely simple with jenkins. I don't think that's complicated at all. As a rule of thumb, you should be able to hide all the complexity in your deployment in the dockerfile. In addition, you can always use jenkins "build with container" functionality to build your application in a dedicated container on the fly. Many ways to hide complexity with jenkins.
I do agree with you that jenkins is abused because it is more than a CI/CD tool. I think that you need some experience using it to know what works well and what doesnt. Unfortunately in the new age "sprint agile" world some random guy has to pingeonhole crap into jenkins in 2 week time windows that shouldnt be there in the first place.
I also think that many devs underestimate what you can do running local jenkins as a tar file on your macbook. I like using jenkins to automate tedious tasks for myself. As an example, it is trivial to write yourself a custom github code scanner that will scan all files and folders in as many repos as you want. I like using jenkins for outside the box things like that.