You can address issues that are damaging to productivity, operability and morale, but only way to “fix technical debt” is “rm -rf”.
I think he was saying this ironically, but this is a very real possibility sometimes. There were often times I’ve found (and wrote) code dealing with cross cutting concerns that weren’t core to our business and I ripped it out and used a popular third party (open source) package.
On a higher level, I champion using managed services whenever possible over doing “undifferentiated heavy lifting”.