The most frustrating part is explaining technical debt. From a business perspective you have a running software. You can sell it - it has all the features needed. Now you come along and tell me you need to fix something, that no client can see, that does not add any features and it might even lead to bugs. And you need two people and x weeks. I can imagine how "technical debt" is perceived as "snake oil" or "developer fun time" for management.
To avoid frustration it is best to fix technical debt in small steps, mix it in with others open issues. Devoting developer time for a big technical debt rewrite is hard to communicate.