I'd like to hear about this from anyone more qualified than me (which would really be anyone).
It takes anywhere from a couple of months to a couple of years to go from "acquired mineral rights" to "pumping oil out of the ground" which you can then sell. It also costs a lot of money. On the order of multiple tens of thousands per day on land and multiple hundreds of thousands per day at sea.
So when prices are up people are expanding and drilling and going gangbusters. When prices are down people are trying to figure out how to cut costs and ride it out. During high prices there tends to be "over" investment and during low prices, "under" investment, both relative to the average investment over say a 5-10 year period. Maybe by a lot.
As you mentioned one of the dynamics is that during low price periods alternative energy is a fool's errand and during high price periods it's genius. During average prices it'll slowly make more and more sense as the average price continues to slowly rise.
But it's far better in some ways to instead of making the average price on an average day to some times make a lot and some times make a little by letting the price vary wildly. Price instability makes alternative energy investing chaotic and unappealing to big institutional investors who might plow substantial money in. Better to wait until the chop settles down and it looks good over a 10+ year timeframe.
So if you get to make the average price on average and destabilize your competitors by letting the price wander all over, what's the downside? As long as you have the money to ride out the low price periods, none!
I don't know when this interesting ride will come to an end, but supply/price management is going to continue to be very important to people with a lot of oil left in the ground.
The US consumes 20 million barrels a day, so at current prices that's $600 million a day at $30/bbl and $2 billion a day at $100/bbl. How much do you think you'd have to buy or sell in the futures market to move the price to smooth it out?
Finally, it's very, very hard to know what's short term volatility and what's long term structural. Everyone "knew" that oil prices were going to be high forever because of peak oil and declining production. Turns out, maybe not.
Prices are set by supply and demand. A worldwide depression has a way of pushing the price down, even with restricted supply. And not that decreasing oil supply is supposed to cause it some of the times, leading to huge price instability, not simple price increases.
TLDR, this price is much more a gauge of our current depression than anything else.