I'm doing so, but sometimes I can still see/reply to the comment despite me flagging it.

Is a certain threshold of users required to flag a comment before it's removed?