> When I ran this I gave the server and client 2 cores each. Neither were maxing out the cores.

grep for GOMAXPROCS didn't turn up anything so not sure what you mean by that. This may help with core utilization.

   func init() {
       // 
       runtime.GOMAXPROCS(runtime.NumCPU())
       // …
   }