This shouldn't be too surprising, as caffeine itself has a flavor. It is somewhat bitter, which is one reason why it's particularly common in root beer.
Root beer is so sweet that I would think it could mask the bitterness. I wonder if the plants that originally were brewed into it just didn't have any caffeine.
Do the ingredients to coke naturally have caffeine, or was that a substitute for the original alkaloid? I.e., is decaf coke because the removed the caffeine, or is caffeinated coke that way because it was added?
I don't know why caffeine was originally added - maybe for flavour, maybe for the stimulant effects, possibly because of having to not have cocaine in it anymore...