So long as you can push the pedals safely all is fine. The Highway Code does have a few words about wearing appropriate footwear, the biggest thing is insurance though. If you are driving in high heels and it all goes wrong then your insurance isn't going to necessarily help.
EDIT: Whoops, no, it's actually legal [1][2] (in fact, it's perfectly legal to drive a car naked in Germany in public - though you shouldn't leave your car then, because being naked in public is not actually legal, the fine-print is that your car's interior is considered your private space), but you may get into trouble with insurance. While searching, I think I may have read that it actually is illegal in Switzerland, but don't take my word for it...
[1]: https://www.focus.de/auto/praxistipps/barfuss-autofahren-ist...
[2]: https://anwaltauskunft.de/magazin/mobilitaet/auto/ist-es-erl...
Tangentially, I once had German autobahn police laugh at my 'gummistiefel' when they pulled me over... The opposite thing of driving barefooted, but hardly proper shoework for driving a car
The real DANGEROUS is with flip-fop, slippers or sandals and I can confirm from my experience.
You can get the front tip of your flip-flop stuck under the break pedal and, besides not being able to break quickly, you panic very fast.
If you're on flip-flops put them aside and drive barefoot.
“It is perfectly legal to drive while barefoot in all 50 states in the United States, in Canada, and in the U.K.”