This is a perfect example of why "average" metrics for such values aren't that great and are often overused as vanity metrics.
A nice chart showing how many users are in each bucket of load time would be far more useful. One that you could easily change the bucketsize from 0.1ms to 1 second and these types of 'digging' wouldn't even be a second thought.