> I want to consider and build both those things, not expect all the keyboard stuff to happen automatically because I used a library to manage mouse events.

I understand what you're saying and commend you for it, but other developers might not want to implement all that themselves, or more often than not, might not even be aware that they have to. In other words, a JavaScript library with zero accessibility features gets added to one hundred websites over night, and in an instant, you have one hundred inaccessible websites. It is an all too common pattern in recent years, as most projects, even the big ones, don't even include accessible examples.