But that doesn't explain false negatives, does it?
Saw reports that bronchoalveolar lavage (introducing and then suctioning out fluid from a branch of the lung) is better at picking up virus if it's present -- but it's much more time consuming, more invasive, and much less pleasant for the patient than sticking a swab up your nose. Not something you're going to do in a parking lot in a drive through testing center for someone only showing mild symptoms.
There are always compromises.