Take a look at https://tensorflow.github.io/serving/serving_basic and https://tensorflow.github.io/serving/serving_advanced