I can see that, but the opposite approach seems even more absurd. Most of the physicists (and neuroscientists) who claim their work refutes or could potentially refute free will, seem to be using a definition of "free will" that's inherently magical. Something like: free will means that The Human Mind is an entity outside the universe, and can poke holes into the universe and move matter in a way that cannot be accounted for by the laws of physics.
If you start from that assumption, it seems like a foregone conclusion that free will doesn't exist.