Hacker News Reader

Comment by lowdose | Hacker News Reader

Meh, I thought Google was always on the cutting edge of ML. They just tried to pimp up a two year old dysfunctional show pony and sell it as a new racehorse. I would like to see a trained GPT3 that's is currently up for rent from the AI shop delivering actual innovation.

View on HN

GPT-3 costs $4.6 million dollars to train currently[0]. Not only is your comment unnecessarily negative, it's just generally a pretty bad take.

[0]: https://lambdalabs.com/blog/demystifying-gpt-3/

What value is added by training a model yourself? Why don't they provide that model?

Isn't the real innovation here to give people a $4.3 million model for free?

Instead of letting people burn unnecessary compute power to train a model 2 years behind state of the art.

The main value is that you can choose your own training data. For example, if you wanted to train a text-generation model on purely C++ code, this would be a good way to do it.

They don't provide that model because then it can't be rented out to people with less upfront cash for insane profit.

This isn't a Google research project, it's just hosted on Colab. Also this is from April, pre-GPT3. Also given that there's tons of cutting-edge pre-trained models available for free but relatively few resources on how to actually train such large models on a hobbyist's budget, this type of work is really important.

Disclosure: I work on Google Cloud.

The link here is just to someone’s personal Colab notebook (which Google hosts for free at colab.research.google.com). This doesn’t appear to be a Google project. It’s more like a link to a GitHub repo.

Reply on news.ycombinator.com