But why even allow them to compete under neutral flag, if this isn't political?
Right now it's about a non-state actor (the Olympic committee) doing what amounts to disciplinary action within it's own ranks. From that point of view, it's purely administrative, and keeps politics out of it (to a degree).
Most non-Western aligned countries already see it like that.
Participating under a neutral flag and not representing your country that sponsored you throughout your career is a no-no to a lot of nations.