It's like math in school. I wasn't good at math in school because I thought that understanding the concepts was enough. But without actually putting pen to paper and drilling through math problems you quickly get lost. It's a skill issue.
I am skeptical that you were able to get any good at Maths or Physics without drilling problems.