I used GitHub actions when building a fin services app, so I absolutely used the hash to specify Action dependencies.
I agree that this should be the default, or even the required, way to pull in Action dependencies, but saying "almost no one does" is a pretty lame excuse when talking about your own risk. What other people do has no bearing on your options here.
Pin to hashes when pulling in Actions - it's much, much safer