I have to steelman this position with something now:
If you have a country which accepts broad civil rights for sexual minorities, and you have a large influx of people coming into your country who prefer few or no civil rights for sexual minorities, what happens when you have enough immigrants to force a vote on rights for sexual minorities? Don't tell me that can't happen; ultimately, there's always a threshold beyond which the government loses legitimacy if it refuses to put certain things to a ballot, and by "government" I mean "ruling party or coalition" if you're lucky.
If I were a sexual minority, I'd see border restrictions as being an existential issue.