You could get completely hammered if one of the exchanges goes down temporarily helfway through your arbitrage
When you look at the price history (and open orders history) between some pairs on one exchange you'll find that during these spikes there was no arbitrage being done at all.
One example is seeing the price on ETH-BTC pair being constant despite BTC dropping 10-20%, and then combing back in 5-10 minutes.
Two economists walk down the road.
E1: look, a 100$ bill over there!
E2: if it were one, somebody would have picked it up already!
E1: nods