Sideburns aren't called sideburns just because they're on the sides of your face; sideburns were originally called burnsides after the American Civil War general Ambrose Burnside [1].
[1] https://en.wikipedia.org/wiki/Ambrose_Burnside#Sideburns
[1] https://en.wikipedia.org/wiki/Ambrose_Burnside#Sideburns
He's also the godfather of the Big and Little Indian Brothers.