The main lynchpin in my thinking is Alice Corp v CLS Bank Intl - that an abstract idea in and of itself is not patentable. I can't imagine a case where mere speech could transcend being an abstract idea ... I suppose there is some slight room on the edges where your source code could be a program that directly interacts with some specific piece of hardware that is covered by a patent ... if that was the case though it seems to me that any program written against a patented processor or architecture would need to be licensed (and maybe that is indeed the case, I'm not a lawyer obviously)
Fair use was a judicially-created copyright law doctrine founded on the First Amendment freedom of speech and press before it was codified, in a very loose form closely mirroring the pre-existing judicially-created factors, into the copyright statutes in 1976. So it is an express exemption in statutory copyright law, but it is also closely tied to First Amendment free speech/press.
And, independently of whether the statutory exemption applies, that the First Amendment freedom of speech and press limits the scope of exclusive rights that can be granted in copyright and permits those rights nominally assigned in the statute to be violated when the government grant of those exclusive rights would conflict with speech/press freedoms is the entire foundation of the pre-existing fair use doctrine.