My personal take is that it's largely a matter of intellectual values. I think that if I considered memorization to be important when I was younger, I would have gotten better at it, for instance. Instead my values tend to be around understanding the reason something is one way or another. I remember a friend being very surprised when I gave a satisfactory definition of 'common law' (because I tend not to know 'random' things like that)—but the only reason I remembered it clearly is because I understood the role it served. That tends to be the only thing that makes things stick for me.
I think the slowness may in part be due to that too. There are a lot of things that folks will have remembered whole, but I just know how to reason my way back to the result from a smaller set of more fundamental things I've learned.