Thank you!
For docker-compose, do
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Docs: https://docs.docker.com/compose/gpu-support/