You must still be looking where you want to shoot otherwise you could point in any direction since "you're just looking at the iron sights". By that logic I could make any distance shot even if I couldn't see anything past 1m.
Once you raise your arm for real, you don't really need to look at the target, your hand and hence pistol will be in the right spot.
All you need then is to fine-adjust the sight alignment, hence why you focus entirely on the front sight post.
Once you get good you just know how to stand and such, so you'll be mostly bang on right away.
For this reason, if you get glasses for shooting, you'll want them made so your best focus is at the front sight.