People do consider it.
But one of the issues with the tax system is that it’s not just a revenue generation mechanism. It is also used to encourage ‚good’ behaviours (e.g. saving for pensions) and discourage ‚bad’ (e.g. gambling).
So, whenever you make a change to the tax system, you are not just changing revenue generation, you are also adopting a political position. Are you „hurting frugal savers” by removing their benefits, or „supporting gamblers” by removing their taxes.