I believe there's a setting somewhere in your account to turn this on, I think by default it just hides tracks that have been removed from the service. They stay there on the playlist (I guess in theory if the track comes back onto the service it'll reappear, but in practice I've found it's often a slightly different version that doesn't get matched up).