Instead of supplying a buffer to each read call itself, you can use something like a sync.Pool which will give metered and concurrency-safe access to a buffer: https://golang.org/pkg/sync/#Pool.
(TCPConn implements ReaderFrom, but not WriterTo)
Yes I understand the purpose — hence suggesting WriterTo which is supposed to do that — I simply noted that conn.Read(w Writer) would make Conn not extend io.Reader anymore since it takes a buffer.