We use it for inference and it works fine. Now with EFS support it's even better.

We have used it successfully with both Tensorflow and Gorgonia