Polling as an example may be an effective way to lower latency, e.g. Linux NAPI.
Increasing buffer sizes may also decrease latency because data can be processed in bursts, see buffer starvation. But if data comes in at a higher rate than can be processed, buffers fill up and are horrible for latency.