Is randomly deleting stuff from /usr a big part of your use case?
A simple example is if you run 'which vim'. If you have installed vim through pkg on FreeBSD it will show up in /usr/local/bin/vim. Whereas if you run 'which vim' on Ubuntu, it will show up in /usr/bin/vim -- but on Ubuntu Desktop (when I last used it) vim is preinstalled. You can't just look at the /usr directory on a Linux machine in most cases and know what was installed when you installed the machine.