There are plenty of other reasons you might have kids but not a wife. Maybe you're a widower. Maybe they're actually your brother's kids, but he and his wife were killed by a drunk driver, and you're the godfather.
Perfectly legitimate cases where you just have no reason to get into it with the freaking cab driver, and probably don't get into a serious conversation, and probably also don't tell him to mind his own damn business.
Wouldn't it also be inappropriate for the cab-driver to follow up, "Do you have kids?" with the question "Are they biologically yours?"