Who remembers boundary areas and the kind of fence they have? Access to them is restricted and people don't usually have a reason to go there anyway.

After they were abandoned there's still no reason for them to enter cultural consciousness. They'd still be out of the way shrubberies on the back of fields, with no indication that they were part of the longest shrubbery in the world.

And even if they knew about that piece of trivia, were they expected to pass it along to future generations? (!)