Those people simply don't earn the term "free speech advocates" then. Someone can claim the banner, but if they don't live up to the term itself it doesn't matter what flag they want to fly.
You can claim no true Scotsman, but you need to back it up with reasonable alternatives to alternative definitions of the same term.
Political movements don't care what a single individual thinks.
Once again the optics doesn't match the messaging.