It’s not just the Xbox controller I’ve had issues with. PS4 controller and even wireless headphones have failed too.
What I suspect is happening is that the devices are changing their names between versions of SteamOS/Linux Bluetooth stack.
Once I remove and reconnect the devices they work again for a short while. I also have had a number of issues with the wifi connection on the device just being slow so I liked the idea of running it on a dock so I can use Ethernet.