True that, but I think in a very short amount of time, using dedicated general purpose GPUs just for inferencing is going to be mega overkill.
If there's dedicated inferencing silicon (like say the thing created by Groq), all those GPUs will be power sucking liabilities, and then the REAL singularity superintelligence level training can begin.