I don't think anyones concerned that it tracks you with cached data (47 entries in github example, date spread would imply ~ 1 month) -- the concern w/r/t apple is that there seems to be no limit to the amount of logging retained.
I won't get into paranoia mode, or conspiracy theories, _but_ I am fine with either phone being aware of my location, and even my recent movements (subject to disclosure) -- I am NOT fine with that log being undisclosed and perpetual