docker-toolbox you are talking about. The new version (Docker for Mac) with this problem doesn't use a VM anymore, but native virtualization on OS X. The old version doesn't have this problem.
xhyve spins Alpine Linux VM inside which it runs Docker Engine. So basically it don't deviate too much from running Docker prev. to docker-toolbox for macOS.