That sounds idealistic to the point of foolishness. The end result of your decision to quit art was that you had to get a job as a graphic designer. Doesn't that achieve even less of your vision?
A lot of people encounter this same dilemma: "do I want to conform to an imperfect system in order to stay in my chosen profession, or should I quit?"
By choosing to quit, you are giving up all of your leverage. You will be replaced by the next "yes man" who doesn't share your vision. If you stick with it, you can continue to nudge the system in the direction you want.