I've been there as a mosque and it was fine, I am agnostic but from a super Catholic family, didn't bother me or the rest of the group at all, it was quite cool.
The smell thought, was super funky, I guess that is what carpets and 1000's of feets do :)
You will be missing out, Turkey is a super cool country, awesome food, super very proud people.
Haia Sofia is not even in the TOP 10 of my list of cool stuff to do in Istambul. I really hope to go back some day and spend a at least 2 weeks in the region next time.