Correct. Which is why the client should just error out and stop processing and return the error to the user, who will have more context and knows whether or not a retry is necessary or desirable.
My argument is that your "resilient design around client-server communications" isn't necessary in the majority of cases, and is often unwarranted over-engineering. Poor user experience is fine, if they don't happen very often, and go away upon a retry. Even banks do that. It's fine. No one will be offended if your app shows an internal error message once a month (a five-minute outage in a month is still more than 99.9% availability).