smtp will eventually give up retrying, and if you're not willing to "always" respond within a few hours, you're going to lose emails.
At least if the the mail get's bounced after a few hours only, they get some sort of feedback, instead of being left wondering.
If you're running this in the cloud and properly automated everything, and its not a billing issue, responding and fixing things could be as simple as a few hours.
If you're running your own physical machine, repair times for hardware issues can be days.
If you're on vacation for a week or longer, and are making a concerted effort to step away from technology and lines of communication, or worse hit by a small bus and in the hospital for any length of time, you may not be aware of issues for some time.
Postfix default is 2d, Exim 3d, and sendmail 5d, last I knew.