Where I live, physics and math are a part of obligatory general education, so you'll definitely have to study them until you're 15. After that, if you stay in the general education system (which is true for 95% of people who plan to go to college), you'll have to learn them for 4 more years. Sure, some people will develop a likeness/talent for it and others will hate it, but everyone needs to pass. Only when you go to college at 19 will people start telling you that perhaps you need to be a physics person to study physics and that it's not for everyone.
Art, on the other hand, is considered something magical and depending entirely on your talents right from the start. Art subjects barely touch on any practice, you have to learn a bit about the history and you pass. But I feel it's absolutely the same. Everyone can (and IMHO should be forced to) learn to read sheet music, play a simple tune on the recorder, draw still life with correct shading and proportions, write a short poem etc.
Personal involvment would give people a much deeper understanding of the topic. I mean, imagine if physics only consisted of everyone learning about Newton, Galileo and Einstein with just the smart kids doing calculations.