TerrifiedMouse

I don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.

So the watermark is audible to listeners.

I wonder how said watermarks will survive lossy audio compression since a big part of lossy audio compression is to remove parts of the signal humans can't hear and won't notice is gone.

> when I speak am I just spitting out the next word that makes sense or is there more to it.

There is more to it. Specifically you are doing so to advance towards a specific goal. LLMs don't have goals. They just pick from a list of likely tokens - based on their training data - at random to generate the next token.

> forcing them to spend all of their funding until they are forced to sell assets to cover costs

It will be hilarious if they sold all their IP to Google or Amazon.

But it's more likely in such a case, they will just open source / toss into public domain all their IP, shut their doors, and start a new non-profit which MS has no good reason to sue.

I think we are already mostly there.

https://in.ign.com/playstation-4-neo-ps4/148831/news/a-fully...

With more universal machines (powered by AGI), with human-like hands and maybe legs for outdoors, I think cost will come down due to economics of scale and the ability to repurpose machines easily.

> If the board wants to turn the company into some fantasy AGI foundation and cease work on their LLMs

Depends on the contract. If the contract says nothing about OpenAI having to work on LLMs ... then there is nothing MS can do if they stopped all LLM work and worked on something else.

> Lawyers for Microsoft and other investors will be knocking on the boards' door Monday morning and some of these board members who have never had a real job in their lives will get steamrolled.

Just because MS sues, doesn’t mean they will win. If their contract with OpenAI say they don’t have control over the for-profit part of OpenAI (and they definitely have no control over non-profit part) then they don’t. The courts will rule against them and it will be just a waste of everyone’s time.

> outweigh the niceties that other lisps provide.

But don’t people only use Emacs Lisp to program Emacs?

One difference between ChatGPT and people though is that when they don’t know something, the latter usually just tell you they don’t know while the former makes up BS.

So what's the downside to more tires?

Any reason why we don’t put as many tires on (existing) trucks as possible? Seem like a good deal reducing tire wear.

> I started showing people ChatGPT when it first came out, they shrugged, they didn't get it. Most people still don't get how important generative AI is and will be. Eventually, they'll have that moment too.

I have seen ChatGPT. Until they fix the error rate, I see it as a novelty. A toy you can’t count on nor offload responsibility to.

If you use a small clip of Brad Pitt in a 10 sec ad without his permission, he will probably sue you and win. So I don't think fair use is going to protect you.

Right of publicity.

https://www.inta.org/topics/right-of-publicity/

> Not all iMessage users can send/receive RCS or SMS.

What do you mean? If Apple includes RCS like they did SMS then it would operate as follows:

Is target an iPhone with iMessage enabled? Yes? Use iMessage. No? Fallback to RCS. RCS doesn’t work? Fallback to SMS.

> since Apple absolutely refuses to let people actually send an SMS to a number it considers to be participating in iMessage

Are you sure? I remember iMessage falling back to SMS if for some reason it can’t get through on iMessage.

RCS for iOS. 6 different ways it’s fragmented. :p

Or maybe Apple will just implement it exactly as the spec says with no frills.

Frankly, the color distinction was never about shame. iOS needs a way to let the user know that they are talking to someone without iMessage and that they are using SMS to do so because many carriers charge for SMS use.

If carriers can charge per RCS message like they do with SMS/MMS, usually after X amount used a month, Apple needs to make it a different color from iMessages which they provide for free.

People have to know if they are using free iMessages when talking to other people or if they are using up their SMS/MMS/RCS quota.

Edit: Maybe charging for SMSs is not a thing in your country but it is in mine. If I see a green bubble I would be mindful of the number of messages I send because after 200 SMSs I going to get charged per SMS.

Some carriers charge per SMS - after X SMSs per month. iMessage is completely free.

Kind of important to be able to tell if you are burning up your SMS quota.

SMS can also cost money. There are plans that charge per SMS after X amount per month.

It’s important users can tell if they are using SMS so they don’t get a giant phone bill at the end of the month.

> As far as I can tell, it is correct: It does not know the contents of the paper. It barely understands the high-level concepts involved

LLMs work by predicting the next bunch of words - it's advanced auto complete. If most of the training data replies "I don't" to the question 'Do you, or do you not know the contents of Dr. Franz 1994 paper "Code Generation on the Fly: A key to portable software"?' then that's what the LLM will say.

It's half useful for answering frequently asked questions but don't expect it to evaluate its current state and give you an accurate answer.

> We don't know enough about what "knowing" something or "processing" a concept means in terms of human thought processes to know whether there's a meaningful distinction between the level at which an LLM processes these concepts vs. humans or whether there is a meaningful distinction between reasoning and intelligence vs. "modelling the statistical properties of human languages".

But we do know what LLMs do, model language. Not knowledge. Not “thought”. Language.

And the way we get them to spit out output that’s satisfactory to us is just absurd. If you read the link I posted earlier, you will know that if you set the “temperature” of an LLM to zero, it just repeats itself talking in circles. It’s only by adding randomness to its “next token” search that we get output that possibly satisfactory.

> I don't know if "do everything manually and slowly so that you never have to set realistic expectations" is a great way to do business.

It’s called Expectation Management. It’s how successful business is done.

Marketing 101: Under promise, over deliver.

P.S. Apple is quite good at this IMHO. I notice they often very quickly go from product announcement to product availability. My theory for why this is good is that there less time in between for people’s imagination to run wild and fill in the information gaps with their own ideas that the actual product might not satisfy. Expectations remain ground when their customers receive the product vs a product getting hyped to the moon such that it’s practically impossible for it to ever live up to the hype.

Except we can’t get LLMs to say it doesn’t know - it doesn’t process the concept of “know” and “don’t know”, heck it doesn’t even process the concept of true and false.

It’s a language model. It models the statistical properties of human languages.

> What I actually suggested is that a willingness to fill in details is necessary, not that filling it in no matter what or without having something reasonable to fill in is necessary.

Except that's exactly what LLMs do due to the way they work.

It's statistically guessing the next token. Unfortunately, just because statistically the next token is among the most likely to appear doesn't mean the sentence it forms is correct or even makes any sense.

Frankly, the whole way LLMs work is kind of absurd in the light of what we are trying to do.

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

The darn algorithm actually has randomness built into it - and it's an absolutely necessary component.

Hallucination in this context is when LLMs make up stuff.

The OP's assertion is "make up stuff" is all LLMs really do and sometimes the stuff they made up matches reality - and sometimes it doesn't.

Maybe hallucination isn't the right word. "Statistical guess" is a better term IMHO.

Tried it in Bard and got:

BBBB... (until the program is stopped)

> you'll usually find people answering different questions to the ones they were asked, for example.

Most of those people are doing it intentionally though - e.g. politicians when you ask them a question they don't want to answer.

> I don't think you can have functional intelligence without a willingness to fill in details you believe are missing

I disagree. A big part of intelligence is realising/admitting you don't know. That you are missing information. And you don't just fill it up with BS. You going find that information.

> Consider how much effort we spend reinforcing in children how to recognize and suppress what is fantasy vs. reality, and to make them favour telling the truth

Do we? I don't think children have problems telling the difference between fantasy and reality. If a child isn't telling the truth, most of the time it's not because they can't tell the difference between truth and falsehood, but because they are deliberately lying to you for one reason or another - e.g. they don't want to get punished.