The only really clunky use case for me was internet access - keeping phone and laptop positioned and aligned for 30 minutes was limiting.
And yes there IS plenty of bandwidth at those frequencies. In fact latest IR standards reach 1Gbps, but it's pretty much extinct. There was an attempt called Li-Fi to use it for as a wireless networking but I don't think it went far.
What I really miss is OBEX (Object Exchange), which worked also over Bluetooth, and which Apple sadly chose not to implement: simplest protocol to just ship a file or a contact vCard over, no setup, just worked - and it's been a standard for 20+ years. Early Android had it too, it was since dropped I think. Sigh.