We use multiple GPUs at 100% capacity (all cores) for ML training.
When need be, we use parallelization/concurrency to use every core in computer for our application.
Almost no computer application today buffers while processing (thanks to parallelization / concurrency)
Software can improve a lot faster than hardware can (due to low capital required)