It's technically correct, but they tend to come attached to a baby :)
That can't be right; babies are not usually born with teeth.
Well, yes and no. No, in that babies aren't born with exposed teeth in their mouth. Yes, in that babies have two sets of teeth in their skull when born.
Adult teeth start forming at 20 weeks in the womb. Babies are born with two full sets of teeth, they're just not quite finished yet and are buried under the gums and in the jaw bone. There are images of child skulls with the front of the bone removed to show all of the teeth packed into the bone underneath. It's actually pretty unsettling.