The obvious solutions seem to be either:
1. Add enough speedbumps, unsynchronized traffic lights or even unnecessary timed barriers to make the route slow enough that it's no longer optimal
2. Add cameras that fine anyone who doesn't have a permit to pass, which would be given to residents for free and anyone who pays a sufficiently high price