This is actually something I had not fully considered. I had always considered odd modifiers to create a non-central example of the category, rather than creating a new category.
To take your example of a sperm bank, I had always considered that to be a kind of generalized version of a bank. A general "place you can deposit things for future use".
Do you have any other good examples? I'm gonna have to let this roll around in my head for a bit to see what the implications are.
> a supply of something held in reserve
> a place where something is held available (especially : a depot for the collection and storage of a biological product)
Both of which fit food bank, seed bank, and sperm bank.
Which means that a sperm bank probably is properly a bank, although used without qualification, most people probably mean money bank (or are being intentionally ambiguous to avoid the stigma of needing a food bank/going to a sperm bank).