The Lemond quote is in the context of a professional sportsman while racing.
If you're in a running/cycling group that always trains at roughly the same pace it definitely does get easier - indeed I stopped riding with one group partly because my fitness got to the point I found the rides too easy! So yeah, Lemond's quote can/does apply to amateurs taking part in their regular exercise routine but only if they choose to increase their average speed.