That's an excellent point. Add to that a lack of historical diagnosis and you have difficulty in assessing historical trends.
As Aspergers get older, they can more effectively mimic behaviour and are more likely to slip under the radar. My wife (an undiagnosed Aspergers candidate) is often described by my friends as a bit odd. When my child was assessed as an Aspergers, the penny dropped for everyone (ie. My wife's slight eccentricity had a label).