Everything is more spontaneous and loose. I think they would respect anyone in their group who didn't think it was nice to go to that place, as I think you should respect their choice.
You just went 100% for it and I thought it was a bad choice. I work with 2 guys and a girl and all of them are open minded to know what is and what is not going on.
It's the startup world: you can choose not only who works with you, but what works with you too.
Good look on your startup, I understand your goal to be respectful, but don't be too strict... after all, you hate rules.