Almost everywhere I drive, people usually speed. As a result, the speed limit is a constant distraction - I have to pay attention to what it is on the current stretch of the road (some of the roads I have to drive on to get to work change speed limits 3-4 times)
and to the average speed of people in each lane, and figure out whether it's in my best interest to drive at the speed limit, or whether I'm going to have people constantly merging around me if I don't speed.
If the speed limit was gone, I'd just pay attention to the velocity of other drivers and make sure I have plenty of room in front of me to stop - which I try to do anyway. I wish some states or counties would try making changes to speed limit policies.