Listening on multiple channels might help at busier airports. Ground, ramp, approach, departure, and enroute are all on different channels. Military aircraft have their own system. (That may have contributed to the DCA accident.)
Something like this was proposed in the late 1940s. The FAA was trying to figure out how to do air traffic control, and put out a request for proposals. General Railroad Signal responded.[1] They proposed to adapt railroad block signalling to the sky. Their scheme involved people listening to ATC communications and setting information about plane locations into an interlocking machine. The listeners were not the controllers; they just did data entry. The controllers then had a big board with lights showing which blocks of airspace were occupied, and could then give permission to aircraft to enter another block.
Then came radar for ATC, which was a much better idea.