The point is that within the boarding process, the slower (elderly, disabled, traveling with children etc.) people are the bottleneck, and letting them board first actually makes the boarding process take less time. The pilot can't even start waiting to get permission to fly until after everyone's boarded.