There are a million titles, so no physical store will be able to compete with a warehouse. There are no refrigeration or unusual logistics with books. They are fungible: generally speaking, one book is as good as another, identical book.
Something other folks haven’t mentioned yet but it also means direct access to middle class households which are more likely to be buying books.
Profit margin and ability to ship/store easily.
It’s quite simple: generally standardized product sizes and media mail. Don’t think it’s farfetched to say Amazon benefited greatly from USPS in its early days.