it's the form of sugar that matters.
the sticky (think twinkie) sugar does more damage because it sticks to cavities, fueling bacteria producing acid.
the stickiest, the sweetest fruit i can think of is fried dates. banana comes close.
but fruits don't even compare to twinkie, so don't worry too much about it. chewing fibrous non-sweet veg should be enough to 'brush' teeth naturally. of course brushing your teeth won't hurt