The London Underground, the earliest subway system, was initially built as a patchwork of private railway systems that were later brought under government control.
Most of the subsequent systems (e.g. Paris, Moscow, St. Petersburg) used the London system as a reference.
So, mass transit did in fact arise as a result of private entrepreneurship. The reason subways don't exist in places where it would be 'desirable' is because of government interference, or simply poor economics.
And I'm not even a libertarian.