Have you tried a MacBook with its trackpad right in front of the keyboard? I like keyboard-only too, but some jobs are faster when you can point at the screen
But I still prefer keyboard shortcuts, they juste feel more natural while the hands are already in "typing mode". I feel like there's a question of "waiting around". Even though I've always found the mac trackpad to be very precise, even for quick movements, you still have to aim for an icon, move the cursor to it, click, etc.
Now in the particular case of a macbook, where the windows will probably be maximized or not move around a lot, you may get some kind of muscle memory. But you still have to look around for the icon, etc. Whereas a shortcut is always the same, you press the keys and you're done, it doesn't matter where your cursor and / or window happen to be. I also prefer hidden icon toolbars, this allows me to gain some vertical space.
But, as another commenter said, some actions are easier with the mouse and wouldn't throw mine away. In my case, that's web browsing.
File management is also something that I find a lot easier with a pointing device. Also drawing diagrams, unsurprisingly!
Also, if you haven't in a while, replace your cap. I replace mine every year or so and it makes it perform much better. You don't have to push as hard, which also reduces the drift problem.
I don't replace the cap often, but I do take it out and clean it, blow out the dust, etc. I'll give a fresh cap a try soon, thanks. It's just ridiculous this highly disruptive behavior is still a thing after all these years...