There are a lot of people in the world who weren’t part of that “we”, and they were often the ones who ended up lending money to the people who did think it was evil.
I don’t think there is any rational argument that ALL lending is evil (there are certainly many categories of lending that have lots of arguments for its evilness)
If we don’t have credit, then the only people capable of starting a business, for example, are those who are already wealthy.