Docker performance outside of Linux is always going to be a problem since it will need to rely on VM. There is a big thread of discussion here for docker on Mac: https://forums.docker.com/t/file-access-in-mounted-volumes-e....
Our development environment has been with docker on Mac for a few years. It's definitely slow, but we found it acceptable. Do you happen to use a very large image with tons of IO work?