Transformers: The Google scientists who pioneered an AI revolution | Hacker News Reader

Transformers: The Google scientists who pioneered an AI revolution

ft.com

Strangely (or not) there's no reference made to their AI ethics team as being in any way contributing towards this "AI revolution". Maybe because that team was sacked or moved around a few years back - https://www.wired.com/story/google-timnit-gebru-ai-what-real...

There's no comparison in importance. The "revolution" as described is technological and cultural, both of which are underpinned by transformers and represented by gpt3+ (particularly ChatGPT). The ethical questions have become more culturally prominent as a result of that revolution.

It's also misleading to describe them as having been sacked - the political machinations that some of the ethics team involved themselves in to the point of employer conflict were heavily discussed here at the time.

Eg.

https://news.ycombinator.com/item?id=25292386

but plenty more threads I think.

> the political machinations that some of the ethics team involved themselves in to the point of employer conflict were heavily discussed here at the time.

It certainly seems like some of that ethics team appear, at best, as extremely bad faith actors in that drama.

Well they were hired to study the bias and energy waste of large language models and that’s exactly what they found: https://www.technologyreview.com/2020/12/04/1013294/google-a...

The article notes how other Googlers have faced factual review in paper releases and what Jeff Dean did here was clearly something else. Google leadership moved against the Ethics team at a time of high uncertainty: COVID rates were spiking, the stock market had become extremely distorted, there had been a summer of massive protests, Bezos was leaving Amazon.. this was Google leadership being greedy.

Since then “ethics of LLMs at Google” has pivoted to overt lobbying and attempts at controlling regulation: https://www.reuters.com/technology/openai-google-others-pled...

> It's also misleading to describe them as having been sacked

Officially they were fired without cause.

It’s important to compare how Google handled the employment of their Ethics team with that of Anthony Levandowski, who was allowed to instigate an accident on the freeway on the job, left with a over-$100m bonus, and has thus far thwarted jail time and most of Google’s attempts to recoup what they paid him. Levandowski had lots of approval directly from Larry Page, and Page was aware of the risks. In contrast, Jeff Dean never respected the Ethics team in a similar capacity.

I'm sure there's all sorts of comparisons you could make; the point remains the ethics team - whatever their merits as individuals and importance generally - were relatively unimportant compared to the engineers and scientists in this particular revolution, and describing them as having been sacked was misleading.

Whatever many many bad things Anthony Levandowski may have done, he picked up his own check in Google. The guy developed street view for example.

Their AI ethics team appears, at least to a Google outsider, to engage almost exclusively in rent seeking behavior. I'm surprised many of them continue to pick up a paycheck.

Street View started before Levandowski ever joined Google, and his role in Street View was as an investor not an IC.

https://en.m.wikipedia.org/wiki/Google_Street_View

If there’s any rent-seeking behavior, it’s Levandowski’s church of AI

https://www.wired.com/story/anthony-levandowski-artificial-i...

Perhaps the church of Al can throw a few bucks the way of the recently not-fired-but-no-longer-employed AI ethics crew, similar levels of grift going on in each.

Automated neural architecture search would have eventually found the transformer, and better architectures. Reading the paper, the names of the vectors (QKV) seem dressed up to imply that the authors know how exactly a transformer actually works, but I am not convinced. It just seems to me these folks where the first to stumble upon it. They didn’t invent self-attention, either. Right place, right time, tons of free TPU to experiment with.

https://archive.is/KHsKh

"two Google research scientists, Ashish Vaswani and Jakob Uszkoreit, were in a hallway of the search giant’s Mountain View campus"

"During their chat in the hallway, Uszkoreit and Vaswani were overheard by Noam Shazeer, a Google veteran..."

etc

Is this just thinly-veiled propaganda for the back-to-the-office movement?

Reply on news.ycombinator.com