This part would be a reason why: very low GPIO throughput since you're limited to what can be shoved through serial and it all has to be must be manually processed by the extremely slow RP2040. The main upside with something like a Pi 4 is that it has zero overhead access to it via kernel api, so you can pipe hilarious amounts of data through its pins or address i2c devices directly from python with existing libraries for almost every sensor, etc.
The Pi 5 using it through a PCIe southbridge is quite a lot worse, especially since it broke all compatibility and it still can't even drive even a simple WS2812 led strip yet a year after release. They've thrown away the last 10 years of community support and development since the layout was standardized with the Pi 2.