I think you mentally inserted 'only' into that sentence.
Yes, certainly populists can be dangerous to all of those groups, as can elites. Political violence is certainly not limited to populists: indeed their contribution is small in historical terms.