That's the point of the policy. You drive an ICE car, you pay extra for EV infrastructure you don't use. You drive an EV, you get subsidized infrastructure you don't fully pay for.
Perfectly fine example of a government aligning the incentives towards its goals (EV adoption), rather than ensuring absolute fairness.