My sense is that "baby" is a term of endearment that is typically addressed to someone in particular. Similarly, you might express affection for someone by calling them "dear" or "honey," but it would be weird to say "I want a dear" or "I want a honey."