Two things being different doesn't make one of them irrelevant.

Consider if you made ... something to sell. Something great, that you know people want. But no one bought it. But they want it, they just waited you out until you gave up and threw it away, and then yoinked it from your trash.

It's a violation of social decency, as much as pilfering something from your home -- maybe more so. The mere act of physical appropriation can't be the most important thing, since we don't complain about stealing some dirt or water from your property, and we don't complain about, say, chopping down a tree.

Perhaps a better word for copyright violation is vandalism, not theft. It destroys the value of someone's work, the taking isn't the important part.