In the promo pics for the Avengers movie, Hawkeye was wearing double arm guards. As I understand it, they are only worn by n00bs who haven't learned to rotate their elbow to the right instead of down. (Try it. You will see what I mean.)
Google videos of the World Champs or Oylmpics and you'll see what I mean.
Even if this were not the case, why risk it? You don't want to get hit by a bow string on a 44lb bow in the middle of a comp.
The strongest position is when your arms and shoulders form a straight line (with your shoulders low). So the closer to parallel the line of the arrow is to this, the easier it is to hold.
Now, technically you are correct. If your release is perfectly clean there should be no problem. But in reality, there will be some degree of 'plucking' going on. Your fingers can't open fast enough. So the string actually vibrates a bit from side to side. I have seen some weird follow throughs that avoid this problem.
There are very few top recurve archers that shoot without a guard.