Don't necessarily have to punish poor people in net. The revenue from such a fossil-fuel tax could be either redistributed directly to everyone (the so-called "revenue-neutral" carbon tax) or the revenue could be used by the government to provide services to the poor.

The more important question you comment skips over is: should people (poor or not) be commuting significant distances using fossil fuels in the first place? Such a tax on fossil-fuels will naturally cause employees & employers to be physically closer.