Hacker News Reader

doesn't training require inference? so i guess it would help there too?

Yeah but training requires the larger memory deployment data center infra

Training doesn't require inference. It uses back-propagation, a different algorithm.

Backpropagation happens after some number of inferences. You need to infer to calculate a loss function to then backprop from.