I am an expat that has been living in Oxford & Cambridge for many years, after living in many other countries.
Upon arrival, I was shocked at how bad the public transport was in Cambridge. Different bus companies, incompatible tickets, unreliable schedules and extortionate prices. Trains are a bit better, but nothing to be proud of.
Second shock comes from car commutes. South & East England is pretty flat. But the road network really needs an upgrade. For example, highway interchanges are often roundabouts (!) leading to absolutely massive jams.
Thus, what would be a regular commute to work in mainland Europe by either public transport or car (say Buckingham to Oxford) is a bit of a nightmare and thus something nobody does.
Third thing is housing. Bad quality, and land controlled by a few leads to really high prices. Plus, low density and the aforementioned transport issues make it really hard to buy in cheap places that are a bit far away.
Lastly, education is expensive and there is a lot of elitism. In the US, I think it's not so difficult to get into e.g. Stanford if you come from a poor background as long as you work hard and get really nice SAT scores. However, in Oxbridge admissions are heavily biased towards high end schools. The only students from poor backgrounds I have personally met had exceptional achievements in their CV (e.g. IMO medals).