No one; higher fees due to state-enforced limited supply are more than enough to keep the poor from riding taxis.
"Nearly half of [NYC taxi] passengers have a household income of $100,000 or more a year." (2014 Taxicab Fact Book)
"Nearly half of [NYC taxi] passengers have a household income of $100,000 or more a year." (2014 Taxicab Fact Book)
- A highly effective and heavily taxpayer-supported mass transit system (aka the subway) that gets people around marginally slower than taxis, and actually faster than taxis during peak hours. Rides are $2.50, less when purchased in bulk.
- A restricted supply of taxis has meant supply concentrates in the southern portion of Manhattan where fares are frequent and relatively short (maximizing profitability as there is a $3.50 meter drop immediately when taking a new passenger). Taxi drivers strongly avoid areas where fares are less frequent, and rides are longer - and surprise, that's where poor people are.
These have nothing to do with higher fees - enforced supply has resulted in lack of taxi availability to the poor, but not because of high fees. Cab drivers are incentivized to min-max their fares, regardless of how much their medallions cost.
There is actually a program out right now: boro taxis, which are only allowed to operate in the outer boroughs - away from the wealthy parts of Manhattan. This program was specifically started to offset the supply crowding and allow people in more far-flung neighborhoods (read: poorer) accessibility to cabs. They charge the same rates as every other taxi in the city.
Also, before anyone points out that the limited supply is keeping taxis out of the poorer areas, there recently introduced boro cabs[2] are meant to solve this specifically, with limited areas where they can pick up passengers and a much cheaper medallion ($500/year instead of ~$200k/year).
[1] http://factfinder2.census.gov/faces/tableservices/jsf/pages/...
Overall, keeping cabs expensive is probably a net benefit to the overall city transportation/livability ecosystem.