As for the issue with Wall Street: it would be good to create a market and get Wall Street involved. The politics surrounding Wall Street's involvement in say oil is classic populist dreck: they hate paying more for gas, so they blame "oil speculators" rather than acknowledging the oil traders are just helping the market price-in the fact we're in an era where new oil discoveries come in a firm that's expensive to exploit.
You can look at the EU carbon market's total failure as an example. It's a cap system, where the price collapsed almost immediately, and so to date it has had zero effect of any sort [0]. The cap right now is way above what the market actually wants. It's useless.
The opposite -- price overshoots because of a too tight cap + demand inelasticity -- could also be a really bad thing. Both directly from the pain it causes, and indirectly from the predictable political backlash, which would screw up something else in a new & more creative way.
"I don't see any objective way of setting such taxes,"
How about: estimate the economic damage of a marginal kg of CO2 (external cost), and set the tax equal to that? [0]
As an order-of-magnitude guess: [total climate change damage] / [total CO2 emitted], both discounted to present dollars.
Not an economist.
You're a funny one.
1. They set the cap far too high; and 2. They did not allow banking between phases.
They correct the first error in the second phase of the system. For the second, they still allocated too many permits but the economic collapse in Europe led to far lower output than had been expected, and for a time declining confidence in the future existence of the EU led to a collapse in confidence in the long-term existence of the policy, making "banking" the credits for future abatement less attractive.
Finally, the cap is a "backstop" mechanism, which exists to ensure that the marginal abatement takes place, with inframarginal abatement taking place through, e.g., the Renewables Directive and the Large Combustion Directive as well as very high fuel taxes among many other mechanisms the EU uses to drive renewable energy and consumption efficiency.
The cap worked perfectly from a qualitative perspective - there is a highly visible price on carbon with complete capital market instruments available (forwards, futures, options, etc.) to give long-term investors what they need to make investment decisions. It has been the quantitative parameters that have not been ratcheted high enough. You'd have the same issues with a tax.
Both forms of pricing carbon can work, but both need good design.
One is that a cap-and-trade program acknowledges the status quo. There are a lot of businesses that are currently emitting CO2, and saying, "Surprise! Big tax for you!" will generate enormous political resistance. If I understand cap-and-trade programs rightly, we'd start out by giving those businesses the right to keep emitting, but let them sell the right if they aren't using it. Suddenly they have choices, turning the response from, "Rawr! Tax bad!" to "Hm, can I get a competitive advantage from this?"
The second is that in cap-and-trade, the right price for carbon flows not from government fiat, but from market outcomes. What's the right level for a carbon tax to drive the right amount of reduction 15 years out? Hard to say. Cap and trade lets us fix what we care about, the reductions, and then lets market mechanisms work out how to do that most economically.
The third comes from the combination of the first two. The US right is fervently anti-tax, and has been for decades. They are, however, pro-market, at least nominally. Plus, Wall Street loves anything market-like, as it gives them an opportunity to make a buck. A carbon tax will be fought hard by the people facing it before, during, and after. But markets mean winners and losers, and the opportunity for a lot of capitalist reindeer games.
Lower economic growth, better try your bets elsewhere because unless you can suddenly remove a third of the population from the planet we ain't going anywhere but up.
Energy production will simply shift to what is the most affordable and don't expect it to ever become less than today. If anything our energy expenditures will continue to climb. The only change will be how we create it. When cheaper energy becomes available expect usage to dramatically spike, if anything our biggest threat won't be carbon, it will be heat; waste heat.