Yep. USB disks also are generally much more robust to errors than SD cards. My mail spool is on USB, the OS is on the SD card.
Though in my testing, it's still not a night-and-day performance difference, if judged by the Pi 3B+. I am not sure if the Pi 4's USB 3.0 port bandwidth will give that much of an improvement.
It would be quite nice to have any form of drive supported more directly, NVMe, m.2, whatever.
Maybe even a lower end nvme ssd + usb enclosure would be the way to go to get the best performance.
Unless there's something about the USB interface itself that kills random 4k r/w performance?