I've never interpreted diversity programs as telling minorities we are stupider. Universities are a great example, wealthier people with more money can afford better educations and test prep for their children. If the "merit" you refer to is manufactured by circumstance, and the circumstances are weighted in favor of the already powerful, why should we accept the status quo & pretend the admissions standard are an objective measurement?
Quotas are a crude tool but I'd rather see something like that than simply accept a world with a cycle of the powerful remaining powerful in large part due to the less powerful never having opportunity.