If your jurisdiction does not have "adverse possession" or "prescriptive easement" laws, you can probably tear that fence down or do whatever else you want with it. (IANAL and please just talk to your neighbors first!)
None of this is related, either legally or in spirit, with the sentence you cited. The cited statement refers to an unlawful act, over time, attempting to override the law. In most (all in the US?) jurisdictions a law regarding prescriptive easements or adverse possession probably makes your neighbor's actions legal. The entire point of this ruling is that congress never passed a similar law to make this sort of action legal.