Hacker News Reader

A few caveats here:

- It works (that well) only for vision (for language it sort-of-works only at the word level: http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html)

- "Do Better ImageNet Models Transfer Better?" https://arxiv.org/abs/1805.08974

And if you want to play with transfer learning, here is a tutorial with a working notebook: https://deepsense.ai/keras-vs-pytorch-avp-transfer-learning/

View on HN

There's actually been a lot of really good work recently around textual transfer learning. Google's BERT paper does sentence-level pretraining and transfer to get state of the art results on a bunch of problems: https://arxiv.org/pdf/1810.04805.pdf

Thanks for this reference, I will look it up. Though, from my experience people in NLP still (be default) train from scratch, with some exceptions for tasks on the same dataset:

- https://blog.openai.com/unsupervised-sentiment-neuron/

- http://ruder.io/nlp-imagenet/

This is true, but rapidly changing. In addition to fine tuneable language models, you can do deep feature extraction with something like bert-as-service [0] ... You can even fine tune Bert on your days, then use the fine tuned model as a feature extractor.

[0] https://github.com/hanxiao/bert-as-service

Reply on news.ycombinator.com