Training is 3x the memory used by inference, and usually run at a much larger batch size