I get that the two things have some overlap and aren't cleanly divisible. What we do is a major part of who we are.
But I mean, it's one thing to say "I think it [his architecture] is awful", or even "I think his architecture had a negative impact on people/society/cherished values/whatever" but quite another to say "I think he sought the ruination of everything good and decent because he was a demented and feeble mind." Because, yes, I do think the last one would only be appropriate if you actually knew something about the guy...
Edit: And yeah, of course, I'm not trying to censor anyone's opinions. Of course you can _have_ the opinion, you can even express it. I just think that it's not what engaging in productive/civil discourse looks like and, depending on the venue, people may call that out or whatever.
I suspect that, in many instances, people who do that are not trying to convince anybody of anything. If they expected to be greeted everywhere with agreement then they would not go around saying things which they know perfectly well are not likely to result in a vigorous and healthy debate if they were said to strangers in the street :)