Hacker News Reader

Exactly this.

Decisions based on statistical models can be fully audited and understood, and the results are reproducible because the algorithm is well documented.

The same can't be said of AI models.

View on HN

I believe it’s possible with decision trees and forests, even though they’re very complex inside, to calculate the contribution that each independent variable made to the final result. Of course, I suppose those haven’t been deemed “AI” for a few years.

Sure, you can step through it and see what's occurring, but there's zero explanation to accompany it. To fully reverse engineer and explain a reasonably complex AI model would be quite the undertaking. Instead, we have prompt engineering. It's easier to figure out how to ask the black box nicely/properly to do what you want than to open it up and optimize/tune it proper. If we assume that we get that far, I fully expect the resulting explanation of the black box internals wouldn't be considered a reasonable/reliable basis for making medical decisions.

The explanation is unbelievably straightforward for tree models, even if you don’t fully understand entropy/information gain? What?

Reply on news.ycombinator.com