It seems that in the late 19th century in the U.S., the annual death rate from horses was approximately 5 per 100,000 inhabitants. That surprised me, as it's ~1/3rd the current U.S. road traffic fatality rate, and as horses were used for more than just transport.
I couldn't find any figures, but obviously the distance travelled per capita has exploded, so it's likely that horse riding is at least several orders of magnitude more dangerous than driving.