What does a leaked Google memo reveal about the future of AI?
economist.com
economist.com
It is a huge issue for Google. Not because they're behind, but because chat engines don't seem to map well to an ad-supported model. Unless they talk like the Youtubers who start with "But first, let me tell you about our sponsor, FBX Crypto".
They do map well to a surveillance model. That's already getting regulator attention. Look what happened to TikTok, which is really good at focusing on a user's interests. Chat engines focus like that. They have to, to work well.
Related:
Google “We have no moat, and neither does OpenAI” - https://news.ycombinator.com/item?id=35813322 (May 2023)
Google has a good, two part strategy available.
First is to neutralize the threat: support the open source efforts covertly and/or overtly, as those efforts undercut the revenue of OpenAI and other potential competitors for the base technology. This is what they did with Android: at a minimum all they had to do was ensure that Apple did not become a monopoly, which they accomplished (the secondary effect was to cut the underbrush of any other competitors). A name for this is to “commoditize your competition”.
The second half is to use their internal experience with LLMs and their other ecosystem to use supercharge their strengths. Side inventions can either be expanded in the hope of a (desperately needed, unrelated to LLMs) new revenue stream or given away to starve potential new entrants.
The big guys have good experience in weaponizing open source. But google has become flabby and seemingly lost any culture of execution so I don’t know if they can pull it off. But the path is there.
They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.
Google may already be terminal
Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.
Loads of internal position papers read exactly like this. I've written docs that write like this.
1) algorithmic improvements beat processing capacity. One of the examples in the paper was that $100 in training compute budget was enough to get within 2% of Google Bard performance
2) data quality (even for "tiny" data sizes) beats dataset size in improvements
Hence both aspects of the moat of large companies (dataset size and available compute) matter less than you'd expect.
In contrast the advantage of "open source"/academics/"the market", the many warm bodies trying out new strategies, yields improvements at a rapid pace that Google, even with a huge team, cannot match.
"Leaked Google memo" is true in a sense, but the doc was written by one person who isn't the primary leading force within relevant orgs at Google. The headline is more accurately described as "what does the opinion of one Googler reveal..."
On the other hand, if Google is perceived as weak, then its share price will be negatively affected. So maybe not.
Maybe it's just what some guy wrote.
But then how did the leak happen? Somebody chose to publicize this particular bit of text. Why?
It'd be weird not to integrate the comments first if this was being done by PR.
> But then how did the leak happen?
Somebody thought the doc was interesting and shared it with a news outlet.
I've read 100+ docs that read like this. Some sort of position paper designed to spark discussion, much of which happens in the comments alongside the doc. All of them are just the opinion of a person, not the company.
And as to not being the leader at Google, this hardly matters. Being politically adept and leadership skills don’t necessarily correlate with having better feel and insights into the future technology.
And not being the leader reduces filters, doesn’t have a pull of a contract to protect specific people you lead and your own project, etc.
It’d be better to hear that from a founder, for certain, but a honest opinion of a well seasoned engineer often enough is a better reflection of the reality, than an official statement by a “leading force”.
The opinion is as relevant or not as any opinion expressed on Hacker News, the fact that the person who expressed it works for Google is irrelevant.
It is important that the engineer is an insider, as from outside of Google access to its internal knowledge is limited.
Also, the document we're discussing was not written as a serious research paper. It's a more like one of Steve Yegge's rants, written to be provocative.
When something Yegge wrote while at Google leaked, people knew who he was, so it was interpreted as "here's what Yegge thinks." It's the same in this case, but it's someone else who isn't famous.
Citation needed! ;-)
I suppose you might be referring to diversity training, but it's not the same thing.
Even if it doesn’t represent a monolithic “googles thoughts on AI” it is interesting and newsworthy to consider how at least some folks who work at Google think about the recent developments.
If Google goes on a AI-everything spree and forces it awkwardly into all their products a la Google+ — this may indicate how and where AI will be useful to people (or not).
Imagine this was written by Blake Lemoine, who believed that one of the earlier models was sentient. Would it be reasonable to present this as a company-wide belief?
And what matters is how informed and honest that particular employee is. And if ultimately that employee is right or wrong. If there is some objective truth in his or her statement.
(And for a reference, I don’t think that these 200,000 Googlers or Lemoine were right or wrong, it feels like Lemoine was asking/answering a wrong question, which needs to be asked differently to have a well defined answer).
It does matter a great deal whether a particular opinion is held by someone with the power to act on it or by someone who can only try to influence those who can act on it.
Their perpetual disadvantage of course is comically poor product vision, which puts any of their actual advantages in jeopardy.
I've found the Gartner AI Hype Cycle to be a useful tool for making this point with people who are under the impression that "AI" is mostly generative AI, and LLMs specifically: https://www.gartner.com/en/articles/what-s-new-in-artificial...
In particular, the chart conveys this really quickly.
Deepmind is still the leader in AI, and they have much better models - it's just going to take a while for people who don't study ML to realize this because Google hasn't released any of their good models for public use yet (and yeah, I'm not counting Bard, etc because the real model was probably put together by one guy in an afternoon at Google, and the rest of the time working was marketing).
Lots of companies are evaluating fine-tuning strategies for <60B LLMs. Such models can be efficient, private, cheaper and adapted to the use case. They may not be generalists like GPT-4, but you need them for scaling up usage on high volume calls on the same task if you don't want to have to pay a fortune to OpenAI, or for processing confidential information such as in medical, B2B and other cases. Not to mention the fear that OpenAI might snoop on their private data and use it in their corpus.
Because 70 years of named work based on centuries of interest and efforts can remain unseen if one really tries. /R
> But easier access to AI also means bad actors will be able to fine-tune systems for nefarious purposes, such as generating disinformation. It means Western attempts to prevent hostile regimes from gaining access to powerful AI technology will fail. And it makes AI harder to regulate, because the genie is out of the bottle
And the fact that after centuries of steep increase of technical possibilities in spite of low spiritual growth, with clear consequences every time met with bewilderment - you know, e.g., WWI and the surprise (historically noted) post-facto after the heightened possible devastation -, should not suggest that what are to be tackled are the root issues? The «genie is out of the bottle» at least after the industrial revolution.
You’re really trying to be uncharitable eh?
We’ve been talking about time travel for millennia. But I can guarantee you that if tomorrow we figure out a way for most people to actually reliably perform it and play with it, it will be defined as “bursting into the public consciousness”.
But yes, public perception is a different thing. Most people are totally baffled when I tell them that the first spam email was sent in 1978, because for a lot of people the internet (and with it email) popped into existence somewhere around 2004.
Really not the same. Time travel is hardly achievable and has a crisp goal. Artificial Intelligence, on the contrary, has been producing results since the start, and there are (there have been) entire businesses based on it.
> uncharitable
I was being «uncharitable» in front of the depiction of "masses discovering with surprise things that would have been noted by exploitation of the information available (and never had been as available as it is today)", also because on practical terms I find this unawareness of the world socially dangerous.