Hmm, can't agree, having run Xfce for years on HiDPI screens.
Go to Settings, Appearance, Advanced, set the font DPI to something like 144 or what works for your screen.
The only thing that does not obey this is Firefox; the way to adjust it is to set layout.css.devPixelsPerPx to something like 1.5 in about:config.