That's not a counterargument to not needing immigration. Immigration would buffer their population decline and help them to loosen up some of the calcified structures in the society or at least ameliorate their negative consequences.
Of course it's not just about visas. If there is a way and a reason to immigrate, people will do so, even illegally. Japan can't even benefit from illegal immigration because its geographic properties isolate it so well.