> As it turns out, TensorFlow's optimizers don't have a way to manually terminate training before convergence.

I don't know how you determined this, but the optimizer minimize op definitely only does one step, equivalent to doing the gradient update yourself.