Well put. Serendipitously, Dan Dennett and Sam Harris just discussed this last week and released the recording of their conversation[0].
The conversation was interesting and disappointing.
Harris argues that the commonly-held notion of 'free will' is nonsensical. Dennett doesn't disagree but worries that people may construe this to mean that 'all bets are off' and the world will descend into chaos. Harris attempts to explain why this would not be the case; that we'd still have good reasons to imprison people who want to do harm. Dennett agrees but then restates the same worry differently. They never manage to get past this.
Still, a good listen.
[0]https://www.samharris.org/podcast/item/free-will-revisited