DNA evidence is also trusted by default and yet:
https://www.science.org/content/article/forensics-gone-wrong...
https://www.nbcnews.com/news/us-news/investigation-finds-col...
With software, generally the only people with the means to demonstrate the software is flawed are the people in control of the software and associated data.