What it seems to come down to is that some people are just naturally better at some things than others (for whatever reason). People have often told me that I have a pretty good memory (I disagree, but that's what they say), mostly due to the fact that I've been able to, on occasion, drudge up meaningless trivia with little exposure to it. I don't know why or how I can do it, I just can.
Though I'm sometimes "pretty good" at remembering things, problems would likely arise if I were to, say, try to skate through a course or six on organic chemistry based on this "ability" alone. Because past expectations have led me to believe that I don't need to work hard to commit things to memory, it may be a nasty shock if I one day find out that I'm wrong. If that happens and I haven't made good use of time that others have spent studying, I'm suddenly behind.
I hope that makes sense.
Off-topic because I'm curious: what makes you think that you're dumb?