Arguably, it's because you tried it "only for a few hours". Rust has a relatively steep learning curve and it usually takes several months to become comfortable with it. But once you get over it, it often becomes quite hard to return to your previous languages, be it Python or C/C++.
Yes, there are tasks which are ill-fit for Rust and your IO-bound backend example is a good one. Not only Rust ecosystem is somewhat underdeveloped in this area (and don't get me started on Rust async and the heap of troubles it causes...), but also you have business requirements which change unpredictably and Rust's strictness introduces a certain friction in such environment.
But I think your mobile library can be a good project to properly learn Rust. If it's properly encapsulated, I don't think it will cause much trouble to your team outside of build system changes and risks associated with you being its sole developer.