When I read Transformer I thought this was some sort of electrical circuit based solution. I wonder if you could get some sort of solution by representing the network as an electrical circuit.
You could try it on a quantum computer, but you'll have to wait a decade (at least) the see one with enough bits to solve a problem of practical size ;-)
You can build an analogue simulator for TSP by exploiting the properties of the path integral.
We do not have reasons to believe that a quantum computer would be much better than classical for this class of problems.