If it's a call it's a bit out of the blue. Maybe they don't have their headset around/plugged in, maybe they're doing something like eating or doing laundry. You just don't know because all you see is an activity status icon.
Maybe it's just me/my personality.
- I either wouldn't see it because I'm involved in something out and messages are not priority interrupts. (i.e. I don't have notifications turned on.)
- Or, if I see it but now isn't a good time, I'll reply something like sure but right now is a bit crazy. Call my cell at xPM. (Or I'll drop something on your calendar.)
It is nothing like the imposition of requiring them to commute to an office to be interrupted at will.
That's the beauty of async communication. It doesn't matter if they're busy, they'll get back to you when they can, and you're free to do the same. No one gets ripped out of their workflow.