The hardest part of CUDA is getting it installed and recognised by the linker without bricking the machine or throwing strange version conflict errors. Many machine learning engineers can attest to this.
Yes. For some operating systems like NixOS there are not even official packages.
I've been happiest with PopOS! and their preconfigured NVIDIA package. They've done quite well for an out of the box minimal configuration distro that you can jump into ML without all the normal headaches.