There are number of states in the US where this isn’t allowed, and the latch that holds the fill trigger open has been removed from the handle. Sure you can walk away, but the tank won’t keep filling.
It's a jobs program, not a safety program.
Also, people often start the pump and get back into the car while it fills. When it’s full, they get out of the vehicle which generates static electricity (often in winter months where it’s dry and wool or other clothing increases this) and has the risk of discharging on the handle, sparking a fire.
An excellent example of a point with no actual statistics behind it.