Increases in supply also increase consumption, we use lots of cheap stuff, but not very much of expensive stuff.
Oil hasn't been supply constrained since the 50's, it's price is largely based on what producing countries agree on, as well as geopolitics.
Additionally, governments levy a decent amount of taxes on certain end products such as gasoline. They might very well, as they have in the past, decide to simply up their tax revenue as prices of crude and derivatives go down.