Your boss is a doofus. If there really is no dress code then just wear what you feel comfortable in. If someone else comments you could reply with something about how "this makes me feel comfortable and puts me in the mindset to work".