In the end, the deposit is a simpler and low-cost low-effort solution. It's probably the best example of a policy where a small change to law can impact behavior greatly.
If you go to a city with a plastic bag tax, you'll notice that far fewer people are using plastic bags. Cashiers won't automatically drop a single item into a bag, they'll ask if you need them and even when they do, they'll use fewer bags.
The funny thing is, a policy like a 7 cent bag tax is incredibly negligible. That means your big-ass shopping trip with 20 plastic bags is costing you $1.40 if you choose to go that route. You could absolutely choose to do that every time you shop for groceries and it wouldn't affect your finances. But that's the thing, people see a financial incentive, and no matter how small, they take it. Similarly, that's why the can/bottle deposit is so effective.
I read this article recently about giving people a tiny monetary incentive to go to the gym got people to go back:
https://www.nytimes.com/2021/12/08/well/move/exercise-motiva...
Surely, these gym members spent more money on gas to just get themselves to the gym in the first place!
Prosecution of littering is impractical, "it doesn't scale," as one might say. It's nearly impossible to catch that crime as it occurs.
I'm not even sure that "raiding garbage bins" is an issue. I guess sometimes people digging through garbage make a mess, but every time I see this happen in real life the people doing it are relatively respectful and make sure the trash stays in the bin.
Local governments already do pay for the cleanup of litter, but the problem is that litter is just everywhere. It's a losing numbers game. You'd have to hire a literal army to keep it in check.
By the way, can/bottle deposits don't cost the taxpayer money! It's a tax you pay on the drinks when you buy them, and you get it back when you return them. That's why it's called a deposit.