I think you should be able to voice any policital opinion. But following the MSM narrative of calling every party that wants to stop mass-immigration as "far right" or "extreme right" has no meaning to me anymore.
I agree with you in that you can't have open borders, but I would definitely use different criteria than religion and skin color, just because they are more visible. Maybe use scholarity and labor requirements instead?
That is just my head. But beside the word used, if you think your WORST problem is "mass immigration", you should pay less attention to politicians.