The intention is that a pilot should be able to fully understand the complete autopilot algorithm and understand exactly how it should act in any situation.
Today's autopilot can't handle any emergencies at all, they are designed to disconnect when they detect something weird. They don't even understand aerodynamics.
I don't know where Airbus' new system lies, but if we actually designed a fully autonomous plane using modern traditional AI techniques (so no machine learning or neural networks; Just decision trees, adaptive aerodynamic models and predictions) then it could handle a wide range of emergencies.
It should be able to handle bad weather and engine out emergencies just fine, and should even be able to land in the Hudson River or anywhere else it has detailed maps.
These are all documented flight emergencies that any programmer of a fully autonomous plane would know about and test their software against in simulations.
The main areas where such fully autonomous planes would have issues are avionics failures, misleading sensors, emergency landings in areas with substandard mapping and unexpected aerodynamic failures (beyond what their adaptive aerodynamic model can handle)
A more likely cause of automation failure would be failure or disagreement in the various sensors on the plane. These are typically triple redundant, but there have been cases where, for example, all the pitot tubes ice over and airspeed indications are lost. There are ways to deal with this but they're not currently programmed into the autopilot.
Other possible causes include failure of major flight controls requiring workarounds - eg, dealing with a stuck rudder with deliberate asymmetric thrust.
That being said flight control issues can probably be dealt with, if necessary. The real problem is with dealing with ambiguous situations where you have to weigh risks.
Consider: your radios are out. Per standard procedure you should proceed on your flight plan to the final fix and hold until your scheduled arrival time. However, fuel consumption is high - you're not sure, but you might have a slow fuel leak, and your destination has dicey weather. Do you divert without clearance or risk that you might be marginal on fuel at your scheduled arrival time?