Your argument only works in a universe with exactly two newtonian point masses, not a single additional mass. In that universe, time isn't the same as in ours. It is perfectly periodic. That you can't figure out the arrow of time in that universe is actually the smaller anomaly.