The problem is the UK is an oligarchy owned and run by non-dom billionaires. It's essentially a floating labour camp. Virtually all the major industries are foreign-owned, and profits flow out of the country instead of being invested in infrastructure, R&D, and national development.
The public school/Oxbridge educational path is based on a 19th century view of the world, which has its roots in a feudal society that expanded into global piracy.
Rejoining the EU isn't just about undoing the direct, about exposing the population at all levels to a more modern culture with elements of social responsibility and rational state planning.
The EU isn't perfect. But Europe successfully rebuilt after WWII, while politically the UK is still stuck in the 1930s.