Fot at home, I tend to stick with 2.4 GHz. It is slower, but with a <100 Mbit uplink to the internet, local speed does not matter. 2.4 does just work better with less APs and thicker walls.
Yes, this happened a bunch more over the Christmas holiday when we had an extra 3 or 4 younger family members all listening to music and videos over their bluetooth ear buds and headphones, which made it much easier to track down as it was quite a rare intermittent failure with only a single bluetooth device being active.