Hacker News Reader

Comment by mpoteat | Hacker News Reader

After skimming through this, it seems fair to say a lot of the research to come out of this group has been relatively "soft", i.e. ethics and interpretability rather than e.g. the newest most powerful models or innovations like gauge CNNs.

Not bad, just surprising.

View on HN

As a sibling said, I think this is a focus of the article.

At the very least Google has release multiple variations of one SOTA model this year:

* original BERT paper:https://arxiv.org/abs/1810.04805

* ALBERT (smaller model): https://arxiv.org/abs/1909.11942

T5 is another architecture from Google that set the SOTA on GLUE and SuperGLUE this year: https://arxiv.org/abs/1910.10683

It uses a Transformer and a BERT-style masked language model loss, but is fairly different in other regards. For instance, it formulates every problem as text-to-text, allowing it to work for just about any NLP problem.

Very interesting, thank you! This may actually be useful for a project I had in mind given that it can be used on more problems :)

It's more like that is what they are choosing to focus on in their annual report. If you scroll through the content in the same blog over the past year you'll see that the posts are very technical in nature.

ethics and interpretability were the major trends of 2019, across the field

Not surprising, important.

Why is it surprising?

ethics

Something that Google definitely needs to do more research on.

Reply on news.ycombinator.com