The goods that the UK has that other EU nations wanted are being replaced by other EU nations. They are now more competitive than UK businesses that have to deal with EU customs checks and other bureaucracy/taxes on a per-country basis. You can say it is "protectionist", which is what certain politicians in the UK like to cast the single market as now, but this is how every economy in the world is run. The US market is protected too.
> As a US citizen, I cannot freely just travel across the EU either.
True, but as a UK citizen i COULD before 2019. As a US citizen you can work/study/retire in any US state. Would you be happy if that was taken away?
The member states of the EU are still independent nations, they are not controlled by a federal government like the US. Getting into EU lawmaking is a rabbit hole, but my feeling is there is a give and take that overall benefits the member states. At the end of they day each country still controls it's own destiny, as the UK felt it needed to prove.