Most doctors offices are effectively overworked and understaffed small businesses. Shifting all that support to the phone would not be reasonable, from a workload perspective.
Direct patient contacts are a vanishingly small percentage of records requests for a doctor. Doctors could likely handle those via phone, but it doesn't solve the issue of needing an EMR.