Maybe it went over your head. The point that people live outside the city evaporates when their driving is useful to people inside the city. Only if they were driving through without stopping would there be validity to the idea that outsiders are taking advantage of the city.
Their presence is potentially useful but their driving is not. Each person who takes an empty seat on a bus into the city instead of driving is just as useful to people inside the city without causing traffic.
My reply was about whether residency mattered. You're taking about something different.
There's a massive amount of "driving through" Manhattan. People in Queens / Brooklyn / Long Island heading anywhere west of Manhattan can be a cause of through traffic. The entire point of congestion pricing is to setup incentives so that people drive north or south of midtown rather than through it.