If humans did not have any facilities for abstraction, sure. But then "knowledge work" would be impossible.
You need to remember some set of concrete facts for knowledge work, sure, but it's just one—necessary but small—component. More important than specific factual knowledge, you need two things: strong conceptual models for whatever you're doing and tacit knowledge.
You need to know some facts to build up strong conceptual models but you don't need to remember them all at once and, once you've built up that strong conceptual understanding, you'll need specifics even less.
Tacit knowledge—which, in knowledge work, manifests as intuition and taste—can only be built up through experience and feedback. Again, you need some specific knowledge to get started but, once you have some real experience, factual knowledge stops being a bottleneck.
Once you've built up a strong foundation, the way you learn and retain facts changes too. Memorization might be a powerful tool to get you started but, once you've made some real progress, it becomes unnecessary if not counterproductive. You can pick bits of info up as you go along and slot them into your existing mental frameworks.
My theory is that the folks who hate memorization are the ones who were able to force their way through the beginner stages of whatever they were doing without dull rote memorization, and then, once there, really do not need it any more. Which would at least partly explain why there are such vehement disagreements about whether memorization is crucial or not.