Let's say that a little drone could reliably carry a package to doorsteps 10 miles away. Let's assume that it can navigate past trees and power lines and kite strings and so forth.
So Amazon builds up these service centers with hundreds of drones delivering packages in a metro area to meet their delivery time commitments.
Then you get a few weeks of bad weather: regular daily thunderstorms, snow storms, a great deal of gusty weather, etc.
Do the delivery time commitments have caveats for consistently bad weather? Unless you live in California, you have to deal with days and days of rain or other foul weather that will eliminate a drone's ability to fly safely, much less be able to deliver a small package without damaging the contents.
When that happens, Amazon will need to maintain a fleet of trucks with drivers to take over. Maintaining that capacity will deeply cut into the profit of their business model.