If you can imagine, say, being imprisoned in a 10x10 cell for the rest of your life, and how that would affect your own internal sense of "free will", I think you'll realize it's not arbitrary.
One of the debates around free will centers on an idea that free will means actions must be physically random. So, I think people understand that probability and randomness are really important pieces to understanding free will.
The missing link is understanding that probability is a state of mind. It doesn't matter that the physics is deterministic. (Deterministic physics is indeed also responsible for my current unique perception of my consciousness, so deterministic physics can do a whole heckuva lot.)
So, yes, my moment-by-moment decisions in the sea of the universe's physics and other conscious agents are determined by physics, but to all possible computational agents my moment-by-moment decisions and their affect on reality are weighted random.