Exactly. There's a train route here in The Netherlands, from Den Bosch to Utrecht, where the train can coast the last 20 km. That's out of a 48 km trip, so almost half!
I would say that's not precisely true. the point of rail systems started out as "big power plant pulls lots of load". Once that was in place, yes, rolling very far unpowered becomes a compelling improvement.
Rails were invented for horse drawn carts. The low rolling resistance was the point. First steam locomotives were pretty weak and needed the efficiency. They were also used on existing tramways that had to be upgraded as trains got bigger.