People want the homeless to simply go away and die quietly somewhere, but no, people need to survive, and if need be petty theft is the way to continue living. People need food and people need to fed their chemical addictions. With the lack of services theft is an answer.
If people don't want to experience petty theft well then we need to ensure that every person has food and housing. Create a safe supply of free prescribed drugs so that people don't need to steal to get cash to buy drugs from the street.
These are simple solutions but of course they cost money, so if your top priority is keeping taxes low, well they're impossible and so the status quo of systemic poverty, misery and crime continues.