1. The $5 Raspberry Pi Zero isn't really $5.
2. The fact that the $5 advertised price isn't the real price, and the inventory games they play raises ethical problems for some people.
3. You can get an Arduino clone for less than $2 (for real, and in bulk).
4. The Pi and other SBCs have higher power requirements, so they can't reliably run off AA batteries.
5. The Pi also requires an SD card, increasing the cost.
6. The Arduino comes in many form factors which are considerably smaller than SBCs.
7. The two aren't drop in replacements, the Pi runs a "real" opperating system that does a lot of things in the background. So communicating with other devices which have specific timing requirements is significantly more difficult on the Pi.
8. The market for Arduino "shields" is much more competitive than the Pi "hats".
9. There are just a lot more shields and sensors available designed to interface with the Arduino
10. The Pi's IO pins are 3.3v, where the Arduino's are (on most boards) 5v. So using sensors/boards designed for the Arduino requires a level shifter