If you're American there is a big chance that you're using the word pup for a dog of any age. This in itself is a tell-tale sign of the way many people - especially those who are wont to write or speak about their dogs in public places - think about them, as permanent juveniles who will never "grow up".
As for "pup" think of it like a term of endearment similar to sweetie, babe, honey, etc etc.
The removal of testes/ovaries before puberty has something to do with that.
Go to the third world and look at the dogs in the street. They have teats for nursing puppies and balls for making them, all plain as day. What dogs look like in developed countries isn't natural.
I wonder why there are so many dogs in the street ;-)
There's nowhere where dogs are "natural." They're a species that was bred to the whims and desires of humans, almost always to the detriment of their evolutionary fitness.