If you properly learn the skills, then refreshing your skills takes way less time than learning them the first time. This means you didn't really lose the skill.

>Sure someone won't fire you because you can't balance a binary tree by hand, but it certainly might exclude you from getting that next job

People that do this need their behavior changed. Testing people on quickly find-able implementations is an absolute circus. Obviously an exception if the job actually involves writing CS algorithms, but most of them do not.