As phrased its…not, though.
Each of the L, G, B, and T (well, perhaps not the B in all cases) relies pretty centrally on there being a difference between men and women.
(Now, if you were to say, “‘men’ and ‘women’ are defined by gender externally ascribed on the basis of (at least notionally) immutable non-identity characteristics”, then that’s a different story, but that’s a lot different than “there are differences between men and women”.)
"The LGBT community" is in and of itself a part of this weird usage of group terminology to muddy the waters. It's pretty much a political device invented for that purpose - if you don't support all you support none, which is clearly ridiculous.
For example, trans women not being permitted to compete in women's sports has exactly zero bearing on the life of gay men, and is actually in the interest of gay women.
The LGBT community is, in part, built on bonds of similar experiences of ostracism by the larger society.
In recent years, there have been some efforts to tie racial justice into the LGBT identity, but that has been controversial as there is less alignment around issues. Black and brown bands were added to the rainbow flag as part of this, and that was also controversial. The feeling was that that flag was not originally racially focused.