So, "I think other people should not be free" is not covered by free speech?
Context matters here, I think:
If I said "The president should not be allowed to say this", this would most likely not affect the president's freedom of speech.
If the president said "ls15 should not be allowed to say this", this may more likely affect my ability to express my opinions.