It's hard to fire them, because to many outsiders, they're workhorses. They know just enough to scrape by on each task, but they also have institutional knowledge and social connections that make them seem invaluable. Often that institutional knowledge is that they're the only person who understands their own shitty code. Their more-capable peers usually know that they're incompetent, but management doesn't.
There are many reasons people like this can survive. Peers don't want to explicitly throw them under the bus. The best of their peers simply move on--to other projects in the same company, or more often, to greener pastures at another company. Their peers cover for them, because it's easier to work around a millstone like this than it is to get them removed. It takes months of concerted effort to get a peer fired for underperforming. If you want to try to get someone like this fired, it starts to look like you're the asshole with a personal vendetta. Especially because they've been there for a decade and you probably just arrived.
These millstones aren't outright incompetent. On paper they've got extensive experience. Socially they're well-regarded by everyone at the company except their immediate peers who know they suck. They've been faking it and faking it well since long before you arrived, and they'll be doing it long after you've departed.
Make no mistake, though. These people are rarely phoning it in. They are busting ass to tread water. Sometimes they believe their own lies, but more often they are terrified that someone will find out just how bad they are. That's why they work so hard.