While working on a HTTP server in 2007 I discovered removing the "Server" header significantly delayed the render time in Firefox.

So beware of unexpected side-effects!