As written, the First Amendment restricts Congress, while the Second restricts at least all parties to the document, both Federal and State levels. Obviously, such plain readings aren't popular.