Regarding distractions, attention management is extremely important for AR, but has also been researched a lot. As long as we keep advertisers out of AR applications we'll be fine.
Not surprisingly, the military already uses it, and I doubt they'd give figher pilots AR helmets if a screen would be more efficient.
To really make an impact in daily life it has to become usable in public, and the chances of the tech industry getting the tech, the applications and the dorkyness-factor of HMDs solved are pretty slim.
I only see it used in professional contexts for quite some time (e.g. Lockheeds and GEs ideas for airplane maintenance workers seem sensible to me even with relatively high cost attached, if it gets really cheap I could see warehouse workers, building maintenance, ...)
I can think of a lot of scenarios where AR would be a great benefit. For example, imagine an electrician running wires through a commercial building and having the wiring path displayed for him. No digging through documentation, measuring, and marking necessary to get the job done. Or imagine fire fighters with AR heads-up displays getting routing information inside building that is smoky and difficult to traverse.
I'm repairing an automobile engine. There's a certain component I need to remove, but I can't see anything like the photo in the manual because it's hidden below a maze of plumbing and cables.
An AR application that could align itself to reference points on the motor and show me an overlay of exactly where the device is and where I should place a wrench to unbolt it, and perhaps what other parts need to be removed first, would be a godsend in a case like that.
Now extend that use case to something similar, but the technician is working in the cold and dark on an unfamiliar version of the same motor and there's even more value.