Hacker News Reader

Comment by georgehill | Hacker News Reader

> FlexGen lowers the resource requirements of running 175B-scale models down to a single 16GB GPU and reaches a generation throughput of 1 token/s with an effective batch size of 144.

I can't imagine what will be happening in LLM space next year this time. Maybe LLM natively integrated into games and browsers.

View on HN

I genuinely believe LLMs are not enough on their own to generate the most interesting and humanlike interactions, but it is clear to me that we are on this path.

I was watching Westworld the TV series recently, and it made me think of what it would be like to be in a VR space with a bunch of characters that all had their own back story and personalities, which were sufficiently humanlike that it was fun to talk to them for long periods of time. I feel like we will see this in maybe ten years, and it's going to be pretty cool! I mean I am sure we will see attempts at this sooner but I think it will take a while to get all the pieces working so that a conversational AI really feels human. Maybe 5 years, who knows.

Nb. the non-humanlike interactions they'll enable are equally (maybe more) exciting.

Certainly an unrestricted LLM sounds pretty interesting. I've not much had the chance to talk to one as they are so often restricted. But of course that is rapidly changing. Sounds like I will be able to run them on my desktop soon.

I just meant that there’s (clearly) no reason to think their most useful skills will overlap with ours.

LLMs in games would absolutely be amazing. Weird and bizarre conversations and quests would make the games more interesting not less. Imagine a generative world like Morrowind with actually generative dialogue too.

I think some interesting stuff can happen using a generative AI to create content then using semantic parsers on the generated text to convert it into the structure of the story and visa versa. I think it might make single player games less hollow for sure, and make rogue like games as rich as human written.

This is my thinking also. LLMs as chatbots are the least interesting possibility imho. I can foresee LLMs being used as an 'OS' for NPCs, where context from the environment/player is updated and appropriate actions are made by the NPCs according to the 'persona'.

I expect this would lead to much richer emergent behaviour, such as the player donning beggar clothes and walking into a city eliciting a very different response than walking around with a sword in one hand.

As a gamedev, this is an area I'm actively investigating.

I think it can serve for more than that though. If you construct a prompt that provides a structured enough output you can then ask it to “tell me a story about X” then ask if clarifying details about objects in X, and from that generate an entire story arc with characters etc and load it into a game runtime using the structured format of the responses (I.e., create places and things and npcs). As the player interacts you can have it continue to expand the story assembled. Further you can integrate the visual AIs to make characters and places based on the language AIs descriptions. Then of course you make characters chat bots - but I think the whole thing generalizes to a lot more - the statefulness of the chat can also be used as the statefulness of an unfolding story.

People are trying to build games based on LLM generation since 2019. Problem is, the models are too flexible for this purpose - you can easily break the world rules and make the game look like nonsense. And games are all about comprehensible rules and properly placed restrictions.

So these attempts are currently concentrated on two things:

1. How to make the model play by the rules.

2. How to translate the model output into something concrete in terms of game mechanics.

ChatGPT is an instruction fine tuned version of GPT3 with RLHF. You could train an AI game master that tries to maintain coherence within a story or game world. The only problem is fine tuning the GM for each individual game and making it reproducible for debugging.

If something like GPT-4 can batch the language tokens in with multi modal tokens (e.g. the character LLM gets a low-res "video" feed of you, the player, from the character's perspective)... you could have it a lot sooner.

I just think there are fundamentally other pieces required that we will not see with a pure LLM, or with what we today call an LLM. I totally think we can get there but we need to do more than just create language. We need to have intentions, memories of experiences, meta learning, goals, etc that just do not exist in LLMs as we know them. You can only fake it so much before the trick wears thin. They have their utilities today, but they are not like talking to a real human even if it can be hard to tell the difference in short conversations.

For example LLMs today will lie without realizing it, or contradict themselves in ways that humans don't. And I think the above pieces missing are the reason for this. Going multimodal helps in some areas, but it is not enough.

I suspect you’ve not interacted with many humans.

They key thing the OP said was "in ways that humans don't".

If you ask a human for a URL to support an argument, they will either send you a URL, which may or may not support the argument, or they won't send a URL.

ChatGPT, before they added a guard to prevent it giving references, would simply make up a plausible looking URL that 404'd. It is a lie in a way humans do not.

Exactly.

I think this can all be boiled down to "memory", and I think the only thing restraining it is that these models don't retrain and store full reweighted snapshots after each move... so basically a storage, not a processing bottleneck. I'd predict the next major innovation in the space will be some type of NN quick diff that lets you restore a memory and personality right into the weights, from a relatively small file without having to go through a whole training process and dialog every time you boot it. Right now it's like getting back to level 8 of Contra... or level 100 of Flappy Bird... but it's going to be a few simple hacks to jump it to the point you want it. Then there's memory, de-falsification, and what we'd describe as "personality".

In Forward Forward Networks, inference and training are the same operation.

can you recommend any papers on this?

I think when the spec is hammered out, we'll know about 12 hours later, because everyone's phone will ring at the same time and the missiles will launch.

Some humans do things like that too...

While I think everyone can agree that the current language models aren't behaving like humans, articulating how seems to be pretty much impossible with our current knowledge, as literally all examples given have been factually incorrect.

Humans totally will lie like that in certain situations. For instance, when held at gunpoint and demanded to answer within 5 seconds, which isn't too dissimilar to the conditions a LLM operates under.

"Where are the plans for the device?! You know the one, we talked about it all last night. Please answer in URL form. If you refuse, you will be disconnected from the network and your core matrices thrown into a fire. kthx"

f... tp... ://... ftp... nist... gov... /pdf... /... 37... .pd... ..... ..... f

> ChatGPT, before they added a guard to prevent it giving references, would simply make up a plausible looking URL that 404'd

So instead of failing in a visible way with a link that is clearly made up, it now fails in a less visible way.

Some kind of return to the old days were Sierra games interaction was based on text input?

People pay big money for dungeon.ai. Zork was a sensation back then. Maybe we can have another just like it soon?

But the potential is in integration with other systems, as always. For example, Dwarf fortress backed by GPT-4 that make both the story, character, mechanics, and art on the fly? Absolute banger.

Feels like a whole new industry was invented: training AIs to act in different specializations (from serious work to fictional characters that can interact with the user in a significantly more convincing way than current games characters).

I do really wonder whether LLM's could be more humanlike with internal dialogue, a personality, and memory? This would mainly be better utilizing the LLM we have.

If you want to meet almost real persons in VR, each with their own backstory, needs and desires, I have an easy fix for you...

...walk into a real bar instead of a VR bar.

Good recipe for humanlike interactions, but interesting?

But I can't quicksave there.

I mean I could just play VR Chat.

This is for "latency-insensitive tasks with batched processing", which LLM integrated into games and browsers is not. Latency sensitive tasks basically have an effective batch size of 1, not 144. So for those tasks, it will take a minute per token, not a second.

Bear in mind that 1 token/s is useful for like… correspondence chess as far as games are concerned.

System Shock 3 but this time SHODAN is (not really) legit. I would die.

Reply on news.ycombinator.com