I'm fine with generic kernel builds and that's what I usually use, but I don't think tweaking your kernel is as painful or manual today as the greybeards here remember.
Back then, we went into "make menuconfig", and meticulously reviewed every single configuration option, carefully tuning them to our bespoke hardware; it only took a few hours. Try doing that nowadays; there's simply way too many configuration options (each new device driver adds a couple, and even the kernel core has grown to hundreds of them). So yeah, I'd say fully tweaking your kernel is a more painful process today than in the past.
(The same for packages; back then, we could open "dselect" and page through all the available packages in a single evening; nowadays, there's too many of them.)
Even back in the Gentoo days, when we measured the differences, they were underwhelming.
They’re not following each dependency, building it, then testing it, debugging etc…
That’s not a judgement on them. I wouldn’t want to do that either. I just meant to say that they “feel” different to me, even if the end result is the same.