Hacker News Reader

Just noting that your MNIST demo on https://www.gradio.app/ seems to have a bug.

If you create a small zero it is classified as a 9. Or perhaps it is overly sensitive to line width? Drawing large circles counterclockwise with the small pen seems to be misclassified too.

View on HN

Yeah, the MNIST model seems to have issues. I can get it to make mistakes for all line widths and most digits. Maybe the issue with small digits is that there is no cropping happening and the model was trained on MNIST where all the digits are uniform size.

Actually this is probably a good demo for a ML visualization tool as we were pretty easily able to find these issues. If cropping is the issue, this is also something that a test set might not catch.

Yup that's a great point! So cool to see the model begin to break with crowd testing

Yes we're updating this model, you're right its pretty bad...

Reply on news.ycombinator.com