Handing an unprivileged user access to Docker is, functionally, the same as handing them access to sudo. Once you are allowed to run arbitrary Docker containers, you have limitless access to root on the running host.
For this reason, I would vastly prefer requiring sudo to communicate with a local Docker daemon. Sudo was designed for this purpose, has the proper logging and fin(er) grained access control.