When you are on the the aero-bars your hands are not on the brakes and the your ability to perform accurate steering adjustments is greatly compromised.

There is a good answer to this question here:

https://bicycles.stackexchange.com/questions/10231/why-dont-...