Hacker News Reader

Comment by kyleyeats | Hacker News Reader

Bing announced their ChatGPT deal the next day: https://www.theregister.com/2023/01/04/microsoft_openai_chat...

They havent "announced" anything. It would make cost-per-query and latency go through the roof, so its HIGHLY unlikely.

Making a gpt query on every search I'd say that's indeed unlikely. Luckily, most queries are repeated, so caching can work very well.

There are many opportunities there, gpt could potentially be used for common queries to expand the results and even as a way to disambiguate queries. For instance, if I ask it:

"If I make a search query for "go", what are the possible different things I may be looking for?"

I get

> There are many possible things that someone might be looking for when they make a search query for "go". Some possible interpretations of the term "go" include:

> The board game "Go"

> The programming language "Go"

> The command "go" or "Go!" as a signal to start or proceed with something

> The verb "go", as in to move or travel from one place to another

> The website "GO", which is a popular search engine

> It's also possible that the person making the search query is simply looking for information about the word "go" itself, such as its definition, pronunciation, or usage in different contexts.

there may be better prompts, of course.

Also, they may identify some queries as being gpt friendly and get those through gpt, which they may also augment with a suitable prompt. The thing is, giving the query as is to the GPT model is not the only option. They can certainly be creative with how they ask gpt and interpret the results. They don't have to necessarily even display the gpt response, they can use it to improve the results.

Exactly. I keep reading all this hype and folks don't realize how much compute is spent on each ChatGPT query (I'm sure it'll be optimized over time). There's no free lunch

They announced it to The Information, anonymously, in a leak.

allegedly

Thanks. This will help if Microsoft asks it who to sue.

MS would run their own instance. Can you imagine how much money they're willing to spend to dethrone Google?

Why would it make latency go up? It's incredibly parallelizable just copy the weights on more machines.

>It's incredibly parallelizable

No, it isn't. It generates 1 token at a time in a loop until the response is finished. It's a highly serialized task. Parallelization increases the throughput of how many queries can be processed simultaneously, but you wouldn't be able to speed up a single query.

Ah the ol’ have a baby in a month by impregnating 9 women strategy.

The additional cpu to gpu to cpu round trip time I'm guessing?

I really wonder how the economics of running chatgpt for Bing's request will work. GPT is anything but cheap to run, IIRC, it requires more than four A100 GPU cards.

The bill is already largely footed by MS probably. OpenAI and MS are a partnership and they run on azure cloud.

Of course- it makes sense to run ChatGPT free now (reminded that it's an experiment) for training & betatesting. But running it even larger, and on a search engine ? It's a magnitude of difference imo, and totally different than a short-time bill for an experiment.

They could go after the really expensive keywords and do stuff there. I'm just not sure how they'd make more money from that stuff.

Who gives a shit how much it costs. The value is massive and 10x better than Google results. Are you comprehending what that means? After competing with Google for over 10 years with nothing to show for it, Microsoft is drooling over this technology.

There’s a very old story about Microsoft and Win 3.x pen computing. One of the leads was explaining that they worked on it, and someone else opined that it was too bad that they didn’t provide anything useful for MS. The lead replied that the goal wasn’t actually pen computing, it was a special teams “block that kick” play that kneecapped a competitor and so achieved its goal.

If MS can reduce the absolute firehose of money that ads provide to Google, it will totally be worth it even if it takes lighting money on fire to do it.

I asked ChatGPT to emulate various OS's, including Linux and DOS. On a linux emulation, I had it curl whatsmyip.com. This bit was in the response:

    <span id="ipv4">35.230.98.61</span>

That's a google property. I wonder if it was trained on google's cloud.

Or, more likely, it made up a plausible IP address.

The curls have returned two distinct IPs. Both are google's. How probable is that for a plausible guess? Run the experiment yourself. There was an article a while back that shows the prompt preamble to get you going. See what you get.

ChatGPT does not actually execute commands. The IP is in some text dataset and it returned that.

“ .. trained on .. “ ^

["Overall, the specific details of the dataset used to train GPT-3 are not publicly disclosed, so it is not possible to say for certain whether interactive sessions by trainers were used in its creation." - the horse's mouth]

As I just said elsewhere it's not going to "run curl commands" in training time. it's trained on a corpus of public data, it's not running commands at all

Noticed this as well. Just like you can ask it for showing `uname -a` and others. The results tend to vary given the temperature of the result and just seem to invent realistic bits of data from the one it noticed in training (e.g., Stack Overflow ?).

The training technique doesn't require internet access as well, it's not going to give you the IP it used during training.

Very funny tho.

Reply on news.ycombinator.com