The important thing in rule of law is that it should be possible to comply. If visa holders were told that not engaging in political speech or holding certain views in public was a condition of their visa, then it's somewhat defensible.
What are the speech laws that apply here?
Or are you arguing that visas are pure whimsy? That they're awarded based on whether elected officials like you or not?