This is because the great majority of DNS hosts do not provide sufficiently granular permissions to only allow changes to _acme-challenge RRs.
e.g. Cloudflare, as far as I can tell, only gives you one API key which grants all access to all zones.
e.g. Most domain registrars who offer DNS hosting who provide an API grant access to all sorts of management functions, not just DNS zone changes.
e.g. Route53 IAM doesn't let you restrict to a single RR, you expose modifications to the entire zone.
I am really not comfortable giving my web application these kinds of powers.
TLS-SNI was useful because it was relatively protocol agnostic, so some of that flexibility is now gone.