Once you "flag" an article the text changes to "unflag" and clicking again changes it back (at least for me).

But downvotes are forever.