Of course you can average times of day, as well as wind directions. Express them as vectors on a 2-D plane, then separately average the components in each dimension. Although this is somewhat obvious in the case of wind measurements, it works for times of day as well if you think of them as points on a 24-hour clock face.
Your medical friend was not checked out. She was busy stifling impolite laughter at your unfamiliarity with Cartesian coordinate systems.