If it's the situation that someone needs something from you, I think you're within your rights to limit the "conversation" to whatever medium you're most comfortable with (e.g. I myself prefer e-mail for most communications). But if it's a situation where you need something from them (e.g. sponsorship from a potential PhD advisor) you may have to just suck it up and place the call.

You don't mention why you're averse to phone calls. If talking on the phone (especially with strangers) makes you anxious, maybe it would help to write down notes in advance about potential topics of conversation, and have those in front of you before you make the call.