Archaeologists can read worked surfaces with amazing skill. E.g. a bed of pebbles could be a lake bed, a road surface, a floor inside a building, a foundation, and there's ways to distinguish them.
Unless the soil was erased/reformed completely there would be a record of almost any activity, from digging a post hole or trench, to building a wall then removing it again then dumping construction waste into the hole then plowing it under. All are different from 'native' soils deposited by random processes.
Geologic time is not enough to erase evidence of worked soils. Not unless it gets scrubbed to the bedrock and redeposited. And even if that happened some places, it wouldn't happen every place. And its fair to say today, that our civilization has been pretty much every place.