I'm really not up on how this all works, but isn't https://github.com/NVIDIA/nvidia-docker/blob/master/ubuntu-1... hardcoding driver versions in a different way?
Edit: in other words, your Docker image doesn't depend on a specific driver version and can be ran on any machine with sufficient drivers. Driver files are mounted as a volume when starting the container.