As a former neuroscientist of visual perception, I always feel compelled to point out that "attention" in the ML sense has very little to do with biological attention.

"Spreading activation" would be a better analogy.

----

It's mostly harmless, though it occasionally results in head-scratchers from the biological POV. I once saw a ML paper talking about "retinal attention", which is funny, because the retina is one of the few places in the central nervous system where afawk, attention is impossible.

There's simply no efferent (brain->retina) non-motor nerves capable of biasing photoreceptor output; the retina is largely ballistic. (Eyeball motion is not attention.)