Hacker News Reader

Yes, it shocked me too! In this project I didn't get a chance to dive deep into some of the decisions the Fast.ai Library did, but I'm hoping to see if there's some inherent gain based on training.

I'm also very curious what the performance of some of the newer architectures, i.e. capsulenets would look like.

View on HN

I know this is a simplistic comment, but do you suppose it was overfitting? I know the tools try to avoid this, but with so many parameters in ResNet50 maybe it's harder to avoid.

Deep neural nets have a natural tendency to overfit.

Ideally, you have a held-out test which the model hasn't seen and only used after model has trained and tuned on the dev and validation sets. Often bad experimental models will repeatedly use the test set in fine-tuning an existing model which may result in your model learning about the test set rendering the test set useless.

In the practice real world results may vary as your training and test data may not represent the actual distribution of the real world data.

fastai uses held-out data for reporting accuracy by default, which was done in this article.

Reply on news.ycombinator.com