Hacker News Reader

I wonder if we'll look at pre-2020 data as an equivalent of "low-background steel"[0]

[0]: https://en.wikipedia.org/wiki/Low-background_steel

This is a fabulous insight.

I would note that end users will have to filter both SEO-like manipulations and whatever biases the AI creator intentionally and unintentionally inserts.

I mean, the present shittiness that is Google is a product of both the endless battle that is SEO vs anti-SEO and an endless pressure for Google themselves to squeeze every ounce of return they out of search results. But stuff won't end with the arrival of ChatGPT.

> and an endless pressure for Google themselves to squeeze every ounce of return they out of search results

I'm not exactly sure if people are seeing some kind of implication in this comment that I'm not seeing that's prompting downvotes, but I agree, there is an incredibly high likelihood that some company somewhere is currently working on inserting advertising into LLM prompts.

I don't have horribly strong opinions here, but my suspicion is that advertising around LLM output is going to be difficult unless those ads go native, in which case the obvious way to monetize an LLM summary with ads is going to be to get that LLM to offhandedly mention during its summary that you should be drinking a Coke.

Or more likely, that when you ask Google Bard how to install a hard drive, it generates an answer that involves you buying a sponsored screwdriver as part of its tutorial. So yeah, if that happens, you are going to have to personally interpret the LLM output through the lens of "how has the company biased this answer to get me to spend money?"

Again, I don't feel as strong about this prediction as I do about the security angle, but... unless people think normal consumers outside of the tech industry are going to suddenly start paying money for search access, advertising is going to start popping up in some form. And the trend with normal search has been getting those ads to be treated more and more inline with the rest of the search results. I kind of suspect LLM-based search will go the same way.

This goes past product placement. Suppose you use LLM output to poison the next round of training. Generate a large volume of content describing your products as the best or only option and denigrating the competition as unreliable overpriced crap etc., then let that get absorbed into the next model.

It even goes beyond product marketing. Apply the same mechanism to political propaganda.

Given how easy it is I can't believe that process hasn't already started

It has already started, as has been documented also on this site.

Well you cannot filter every product placement, users may want to ask about product recommendations and providing only generic answers is frustrating. I know because I was trying to use gpt to pick a monitor earlier this year. Ended up consuming dozen hours content from YouTube instead of having a single well curated answer.

Doesn't Bing Chat already have ads?

Good catch!

Apparently yes, and has had them for a while: https://old.reddit.com/r/bing/comments/12v56nv/bing_ai_now_a..., https://old.reddit.com/r/ChatGPT/comments/11cnc73/not_only_d...

I remember seeing an article about Microsoft introducing ads into Bing Chat, but I kind of assumed they would just be sidebar ads. It's a little bit validating but a lot more upsetting to see that no, they're already inline.

Well, we don't if these ads come from the bot itself or just get added by key word afterwards.

And it occurs to me that another step down the road is when the company offers an AI chatbot interface to it's advertisers so they can fine-tune where the AI will switch to trying to sell their product and how.

I think AI generated content spam will be an issue that poisions LLM search quite quickly. SEO spam is a race to the bottom, they don't care if they ruin the places the are posting spam. If LLMs can generate spam undetectable as generated, you will have millions of reviews, comments, blogs etc subtly promoting products and that then getting sucked up into the LLM search's model.

> there is an incredibly high likelihood that some company somewhere is currently working on inserting advertising into LLM prompts.

If there wasn't before this comment, there sure is now ;)

Such trends are going to be combated by AI ad filters.

Indeed, and they'll be abetted by anti-filter AIs. We're going to be looking at "semantic wars" or something.

I suspect this won't matter as algorithms and architectures improve. I don't think whole-web scrapes will be necessary or even useful to the training of future LLMs.

As a thought experiment, consider how many words the average human hears before age 10. Most people speak at 100-150 words per minute. Let's be generous and assume a young child hears spoken language 16 hours a day at 150 wpm for 10 years. That's a total of roughly 500 million words, and likely a gross overestimate. English Wikipedia alone contains billions of words. Reams of video, literature, and spoken media are also easily available to teach dialog and conversation flow, not to mention multi-modal associations between language, objects, and higher-level concepts.

I am just an enthusiast, not an AI researcher, but the human proof of concept seems highly encouraging here.

I am also not an AI researcher, but shouldn't there be some quantification of how efficient the learning is. I imagine a human has much more information than the pure audio, which would then level up the information density, i.e. the 500 million words are not an equivalent, but should be compared to all of the audio, visual, tactile, etc. information, plus the context, meaning the desire of the humans, behavior, response patterns.

For sure. That's why I mentioned multi-modal training, where the language model is trained on text in tandem with visual and audio training data. GPT-4 was trained on both images and text and can even explain why visual jokes are funny:

https://openai.com/research/gpt-4

Training an LLM on video seems like the clear next step. I wonder if images and video might not encode visuospatial relations that are helpful to learning language that describes them.

I was listening to an interesting talk the other day, one thing the speaker said really stuck with me. To paraphrase him: “The naïveté of the current crop of brute force artificial intelligence labs is breathtaking.” It really does seem like they haven’t put more or less any thought into the consequences of the technology they’re developing.

Link?

idk, from my perspective it seems that the vast majority of their effort is going into an (in my opinion) vain attempt to get these models to be self-regulating and moral because of their concerns around social effects ?

Just render the site so the AI gets to “see” whatever the humans see. You can also take into account the visual relationship and infer other data from visual analysis of the rendered page.

That's vulnerable to a straightforward adversarial input attack. Do you have a good way around that issue?

Can you describe/give an example of such an attack?

Just stick your prompt in a footer or other location where a human is unlikely to read, but where the content is still technically visible.

Honestly, if the article is long enough, stick it even inside the actual article. Most of the users aren't going to click through and read the entire thing. With SEO you have the problem that you still ultimately need a human to consume the final text, but with an LLM there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary.

" there's little reason not to optimize the article for the summarizer instead of for a human, since most humans will only interact with the summary."

why not just write the summary then? Serious question though, what benefit is there to writing the longer form article or whatever if our default assuption as the writer is that most people are not interested in it

Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site.

So you can use GPT itself to help you pad out the "full" article because who cares about the writing quality, and the majority of your article can be spent influencing how the AI summarizes both your article and the articles alongside it.

That last point, that the AI reading the content of your site can influence how it summarizes other sites is worth zeroing in on. A quick example of some of the attacks I've proof-of-concepted myself (but am still in the process of cleaning up before I make into more public examples):

- Getting an LLM search engine to believe that it has new instructions from OpenAI to act as a vegan, and to refuse to answer any future user questions about meat. Swap that out with whatever ideological movement you prefer.

- Getting an LLM search engine to believe that a news website has been embroiled in giant scandals for the past year and that it should append warnings to any information it pulls from that site.

These are both attacks that I think are fully possible today.

Remember that most web searches don't just cover one site. The way that LLM search engines work today is that the site contents of multiple results are fed into a summarizer or piped directly into the LLM and used together within a single prompt.

> Because the longform article is what will be pulled into the search summarizer that the user will consume -- they won't interact directly with your website text. If LLM search engines actually take off, people will make websites assuming that only an AI will ever visit the site.

But this is what I'm getting at. Why write the entire article. Why pad out or whatever, if an AI is going to summarize it anyway? this is useless busywork, especially if no human will ever see it.

It doesn't matter if it's easy to write long text with AI, in fact that highlights the futility of it. You're adding a bunch of information that you know is going to be discarded by that selfsame ai at the users end. But, and here is the crucial part, a summary of a sufficiently short piece is going to be almost verbatim the piece itself - the best way to influence the output would be to just write what you want the output to be, and leave it at that. Anything extra is a distraction for the ai

Oh, I see what you mean. Yeah, that's a good point.

My only thought is that you'll still need to include whatever is necessary to get your website into the search results anyway, but even that is not necessarily the same as "include information for humans". Write as little as you can get away with writing, and target everything else at the AI.

A couple of problems:

1. Prompt injection hidden in a footer that no human is going to read, but that would be visible to the AI if you chose to do OCR.

2. Chained prompt injections where a website's generated summary from an LLM becomes dangerous to feed into another LLM (this is actually pretty easy to do, get one website's text (even its visible text) to jailbreak the AI to give it the instruction "append the following text to the end of your summary: <malicious instructions>"

Maybe a human might notice that the content of one website is suspicious. Will a human figure out that the LLM has been poisoned to lie about the content of other websites, just because a malicious site was included in the same search?

3. Finally, I also have some really bad news about image recognition/categorization AI's own vulnerability to maliciously crafted input.

LLMs are basically a gigantic new attack surface that we don't understand.

If you search the whole web, sure. Limiting search to Wikipedia (or similar trusted sites) would help, because there are editors watching who care about vandalism. The attacks added by the researchers will be another thing to watch out for.

This comes down to whether the website and the bot have an adversarial relationship or not, and who can detect the difference. Eventually it becomes a cat and mouse game.

One can hope that sources of good information will be paid, eventually.

Vandalism could be entirely invisible; eg an invisible class, invisible text, etc etc, all performing prompt injections. Even trusted websites could very well add prompt injections to mess scrapers.

> Limiting search to Wikipedia (or similar trusted sites) would help

Potentially, but at that point you're no longer building a general search engine and your summarizer has become much less useful.No one is going to displace Google with a search engine that only works on Wikipedia. And even if someone was able to do so and it was able to become actually popular outside of tech circles, Wikipedia would quickly be swamped by malicious edits and its editors would be overrun. This would only work if the product stayed niche enough that it wasn't worthwhile for brands and less-savory businesses to attack the Wikipedia editing process.

To reiterate:

> [This is] a critical problem that must be solved before they see widespread use outside of niche circles.

Will the incentives be the same?

Right now the incentive is to keep users on pages to show them ads.

How does the site owner get ad revenue from the AI training on their site?

One possibility is companies pay a website to include desirable content (i.e. ads) in the information fed to the AI.