But how would you distinguish all the sounds that don’t require any tongue movement, like (most of?) the vowels?
Vowels change the shape of your mouth's cavity to change which frequencies to emphasize ("formants") and we hear the differences in vowels by analyzing those formant patterns.