1) the autopilot is actually quite dumb ("keep altitude", "fly on course X", "maintain set speed", "change altitude at rate X / to level Y"), doesn't handle exceptions (in fact, autopilots are designed to disengage in case of exception), and is generally totally blind to environment other than heading, speed, altitude and optionally position (but not, for example, detailed environment on the ground)

2) The latency issues are literally deadly.

My gut feeling is that when companies cut costs by cutting safety features, it doesn't play out favorably for consumers.

Airlines are fighting tooth and nail to not spend money to implement video feeds on the black boxes, a rather doable upgrade with huge safety benefits, and you want to mandate expenditures on converting all air ships to full remote drones? That is never going to happen.

Better to have the final say come from someone at the controls rather than a desk in Silicon Valley. Humans aren't going anywhere.

Main reason pilot error is #1 cause is that hardware problems can be fixed once and for all, while pilots take care of all the exceptions.

Also, doing a CFIT while flying on autopilot is still a pilot error, because they gave the autopilot's orders.

Maybe this will be fine when the technology is there, but it's very obvious that doing this change today -- especially when it's being lobbied to be applied to cargo planes first (historically much older former passenger planes w/o modern creature comforts) -- is going only going to increase deaths by pilot error.

After pilots are eliminated, the #1 cause of accidents will be autopilot error.

We should really just petition the FAA to eliminate all pilots entirely right now: it'll eliminate the #1 cause of commercial aircraft accidents.