The point of the ADA and similar laws is to improve access for people with disabilities.
Does it increase costs? Yeah, it does. Are the costs "vast"? Generally not; it depends a lot on the situation.
Does it make life worse for people without disabilities? That's mixed. Those who bear the compliance costs get some disadvantages. However, lots of people who aren't strictly speaking disabled benefit from the accessibility and usability improvements of captioning, wheelchair ramps, improved light switches, grab rails, etc. Making your business/website/rental-apartment more accessible, even where it's not required, may end up making you more money as your clientele grows and pay for itself. And of course we all end up with disabilities if we live long enough; let's hope we all last to the point that we appreciate the ramps, larger text, captions, etc.
It should be noted that it was an insignificant quality of life change for the vast majority of people, and that if laws didn't inconvenience people we wouldn't need any. It's too bad that we can't help the disabled by giving everybody ice cream, but that's the situation we're in.