We trust people all the time for many reasons. Authority, past experience, logic (why would they lie about something in context), etc. and we get by alright. Knowing it's another conscious, mortal human existing in society makes it much easier to know when someone is likely being truthful.
Obviously there are all sorts of caveats to that. The main difference is that LLMs don't lie. They don't tell the truth either. They just generate stuff with no meaning. There's no way to ever put any trust in that, the same way I wouldn't trust my ice maker to make sure my dog gets enough water while I'm on vacation.