Copilot is bad at accessibility because web engineers are bad at accessibility. All of the bad habits in this post were learned from its training data.

That's not to say this can't be fixed: a recurring lesson of LLMs is that the quality of the training data is /everything/. OpenAI made their models better at chess by feeding in higher quality chess data - they could absolutely make it better at accessible frontend code by curating and boosting better code examples.

I doubt they'll do that any time soon, purely because there are so many other training data projects they could take on.

Thankfully we aren't nearly as dependent on a few closed research labs as we used to be.

It would be very exciting to see fine-tuned openly licensed models that target exactly this kind of improvement.