> Many of our institutions and cultural practices have evolved around human beings, not ruthless paperclip maximizers.

You just think that because you don’t know how the drug discovery process works.

There’s a step called “lead optimization” where human chemists literally add atoms to drug-like molecules (“lead”) and tests its various properties (toxicity, potency, permeability, …) and iterate until they find a molecule with desired properties (literally “hill-climbing”).

The whole idea of drug trials is to validate those properties in actual humans, in a way that makes it very hard for pharma companies to game the process.