You need to have 2 notations because having 7 notes instead of 12 packs more information in less space. The scale is almost always made of up to 7 notes. Expressing each note as the difference from a default is essentially compression, as it lets you see both the position in the scale (important for harmonic relationships which help you memorize the thing) and the note which you have to play.

Sometimes you need G and the note to its right, so you want to call the latter a variation on A (A-flat). Sometimes you need A and the note to its left, so you want to call the latter a variation on G (G-sharp). They are the same note but they have a different function. It's not an idiosyncrasy, rather, when it works it's more efficient because it describes two things at the same time. It doesn't work perfectly for many cases: Misirlou, the Simpson's theme, anything in whole-tone or octatonic scale, but those are the exceptions in practice, just like some things compress better than others.

It is suboptimal even for the blues scale, but not too much and in fact that shows another benefit of giving two names to a note. The blues scale is mostly a variation on the pentatonic scale; the A#/Bb note in a minor E blues scale is used mostly as an intermediate note. So if you're going up to a B you can write that note A# because it is working like an A, while it you're going down to an A you will write it Bb because it is working like a B.

Likewise for chords. If I see E Ab B, E-A points me at a suspended chord, not a major chord, which would be wrong. Whereas if I see E G# B, it is clearly a triad.