it's a convoluted path to get there, but you can put in delivery instructions into your profile that get surfaced to AMZN delivery folk. I had a similar issue where the AMZN maps would send drivers to a non-existent intersection.
After putting in step-by-step instructions, it's been much less of an issue.