Theoretically, they shouldn't have to. If he's too busy, he probably didn't open it in the first place, leaving only one option.
In practice, however, I find people can usually tell via context that they're being ignored. There are some subjects you'd expect this or see it coming before other, more benign ones.