Transferring the accepted connections this way involves an extra system call to epoll/kqueue to add the event into the particular thread and the accepting thread can become a bottleneck under high load.
A better design would be to share the listening socket across threads and have each thread accept at its own pace, at least this avoids the additional kqueue/epoll system call needed to add the new fd into the thread's poller, but it does cause lock contention in the OS which is still less expensive than a system call. What's even better is if you're on a newer linux version, or bsd, consider using SO_REUSEPORT which allow each thread to bind/accept on the same port and avoids the lock contention issue.
Also you should consider using accept4() to set the non-blocking flag during accept instead of the additional system call to set it to non-blocking.