Hacker News Reader

> I had always been under the impression that chat gpt just learned from sources, and then gave you a new result based roughly on its sources. I think some of the confounding variables here were, 1.

If I’ve understood the transformer paper correctly, these things probabilistically guess words based on what they’ve been trained on, with the probabilities weighted dynamically by the prompt, by what they’ve already generated, and by what they “think” they might generate for the next few tokens (they look ahead somewhat), with another set of probability-weight adjustments applied to all that by a statistical guess at which tokens or words are most-significant or important.

None of that would prevent them from spitting out exactly what they’ve seen in training data. Keeping them from doing that a lot requires introducing “noise” to all the statistics stuff above, and maybe a gate after generation that tries to check if what’s been generated is too similar to training data and forces another run (maybe with more noise) if it is, similar to how they prevent them from saying racist stuff or whatever.

You have understood correctly. What LLMs are, at least in their current state, is not fundamentally different from a simple markov chain generator.

Technically speaking, it is of course, far more complex. There is some incredible vector math and token rerouting going on; But in terms of how you get output from input - it's still "how often have I seen x in relation to y" at the core level.

They do not learn, they do not think, they do not reason. They are probability engines. If anyone tells you their LLM is not, it has just been painted over in snake oil to appear otherwise.

For all that I agree there's a big pile of snake oil in this area, I disagree with you overall.

Having played with Markov models since I was a kid*, LLMs are really not just that.

All that stuff you acknowledge but then gloss over, that is the actual learning, which tells it which previous tokens are relevant and how much attention to pay to them. This learning creates a world model which is functionally (barely but functionally) performing something approximately reasoning.

Statistics and probability is the mechanism it uses to do this, but that alone doesn't make it a Markov chain, those are a very specific thing that's a lot more limited.

For example: consider a context window of 128k tokens where each token has 64k possible discrete values. If implemented as a Markov chain, this would need a transition matrix of (2^16)^(2^17) by (2^16) entries (unless I've gotten one of the numbers backwards, but you get the idea regardless). This is too many, and because it is too many you have to create a function to approximate those transitions. But even then, that only works as a Markov chain if it's a deterministic function, and the actual behaviour is not deterministic due to the temperature setting not (usually) being zero.

* Commodore 64 user guide aged 6 or so, so I didn't really understand it at the time but that's what it was

I have the original guide here. Do you remember where you spotted this? Before the bouncing ball sprite demo?

Off the top of my head and 34 years later, sadly not.

Best I can do is describe this code/listing:

It created a few lists of words (IIRC these lists were adjectives, nouns, verbs?), and in the main loop it kept track of which list it had just taken a word from in order to decide what to pick next — e.g. if it had just picked an adjective then was allowed to pick either another adjective or go onto a noun, if it had just picked a noun then it could end the sentence or it could go on to a verb — and it would pick a random word from whichever list.

This either fit entirely on one screen, or very close to that — I was little, I would have made a syntax error if it had been much longer.

(Perhaps it will turn out to be my family's user manual wasn't even the official one, though I do remember it being a thick blue thing which matches the pictures I've seen online).

You can't write software code without the ability to think. You can't tactfully respond to emotional expression without the ability to think. If that is snake oil then we are all walking, talking, snake oil.

The way we do it is not the same way they do it. They literally only predict the next probable tokens. The way they do it is amazing, and the fact that they can do it as well as they do is amazing, but human thinking is a lot more nuanced than just predicting.

The fact that AI seems to be so reasoned is not that they are doing reasoning, but because there is a phenomenal amount of reasoning inherently embedded in their training data.

AI actually thinks in the same way that the figures on a movie screen actually move. It's a trick, and the difference may be pedantic, but it's very important in order to have a real discussion about the ramifications of it.

As far as I know we don't know how we do it. We have very little clue how our higher level behaviours emerge. So you can't claim we don't do it the same way.

Of course I can, humans learn faster from far less data and don't hallucinate to the same extent. What they do is very likely similar to a part of what we do, but they're missing critical components and my feeling is not all of them (empathy and creativity for example) are even possible to replicate outside of a human experience.

You are extrapolating from a result to the implementation and making a judgment call it's thus not the same. That is not valid to do. You can come up with countless examples of the same tech underlying principle beeing used but the results are dramatically better now. Lithography for example.

You could also look at a koala and make the argument they function totally differently from us since they almost can't learn anything and are extremely stupid.

I would definitely make the argument that Koalas function differently from us, also from observation, which is a perfectly fine thing to do.

You're so wound up in your theories that you're not making much sense to me.

We learn our behaviours by immitating others.

You can clearly see behaviourol pattern in people and in their parents.

For example the boy who brushes his teeth the same way his father does.

I'm really lost on what you think your brain is doing? Have you never thought through things but acted differently? Like procrastination? Spouting out something and thinking after "ah man i should have just done x instead of y'?

> You can't write software code without the ability to think.

Counterexample: using genetic algorithms to write corewars programs: http://www.corewars.org/docs/evolving_warriors.html

> You can't tactfully respond to emotional expression without the ability to think.

Counterexample: ELIZA.

If there are 8 blue beads and 2 red beads in a jar, and I ask the computer to draw a bead out of the jar and its a blue one that it has drawn, did it really think about giving me the bead?

> You can't write software code without the ability to think. You can't tactfully respond to emotional expression without the ability to think.

Of course you can.

https://en.wikipedia.org/wiki/Chinese_room

They’re not responding “tactfully”, you’re projecting emotion to a bunch of words written coldly.

It’s like writing a program that has a number of fixed strings like “I feel sad” or “I’m depressed” and when it sees those it outputs “I’m sorry to hear that. I’m here for you and love you”. The words may be comforting and come at the right time, but there’s no feeling or thought put into them.

>but there’s no feeling or thought put into them.

How do you empirically measure this?

Humans can measure feelings, computers can't. Therefore I can say if ChatGPT doesn't have enough feeling but it can never do the inverse to me.

That feels simplistic, but we're dealing with fundamentally human concepts. I see absolutely no reason to work under the assumption that computer programs are somehow in the same domain as human thought, which is what a lot of people (you) are saying.

The goal should not be to demonstrate ChatGPT and Humans are different, because to me that is obvious and should be the starting point. Rather we should do the inverse, show that ChatGPT is indistinguishable from a person, as measured by Humans. And then, maybe, we can consider granting this computer program human rights like the right to use copyrightable media in a transformative way.

Ah, but that is really hard to do. So the AI tech bros don't do it, and instead work in the opposite direction.

They learned already at the beginning. Its called training.

Its the same thing we humans do, just a lot faster and focused on the content we give it.

'think' what is thinking? Recalling what you learned?

Wiki says: "Their most paradigmatic forms are judging, reasoning, concept formation, problem solving, and deliberation. But other mental processes, like considering an idea, memory, or imagination, are also often included"

Talk to an LLM, it will reflect these concepts very well.

'reason': even people don't reason. I had plenty of discussions with people who do not act logical. And there have been plenty of good examples of LLMs leanring to reason. Look at Grok2 and just wait for GPT 5.

You put the achievement of LLM down as its nothing despite the fact that it could mean a lot more. How big is the chance that we are also just probaiblity engines?

We as humans are more individual than a LLM and we do have more mechanism in our brains like time components, emotional interactions, social systems.

And not even your mentioning of "Markov chain" is correct: A LLM Architecture is not how a markov chain works otherwise we wouldn't have the scaling issues we have with the LLMs...

> What LLMs are, at least in their current state, is not fundamentally different from a simple markov chain generator.

The output can contain words that are only in the prompt and not the training set. (I asked the Bing chatbot about a "word" I got from /dev/urandom)

I don't think a proper Markov chain can handle that sort of thing?

I remember similar news about ML services that generate mnusic: they are able to reproduce melodies and lyric from copyrighted songs (if you find a way around filters on song or artist title) and even producer tags in Hip-Hop tracks.

All this latest ML growth is built on massive copyright violations.

It’s not a copyright violation.

Maybe not myself, but many averagely-talented artists can draw Mickey Mouse.

They might even draw one for me if I ask! Or I can just find it on Google… (technically my computer is producing it on the screen…)

That in itself is not a copyright violation. But if I use it, in a commercial manner, then it becomes a copyright violation.

Producing copyrighted things isn’t illegal. It’s on the user to not use copyrighted things, in a way that’s illegal (not fair use or licenced).

I wanted to note that you cannot compare learning by a human and "ML learning" which basically is calculating coefficients from copyrighted materials. Don't those coefficients fall under definition of a "derivative work" by the way?

ML models are not "learning" in the same way as human do, and while they use the misleading word "learning" is has completely different meaning; also, ML models are not humans and therefore they are not subject of laws; the engineers who perform calculations are.

So comparing calculation of ML model parameters to a person studying art is incorrect; you should compare engineers performing calculations with data from copyrighted material to a person studying art. It is immediately obvious that there cases are not equivalent. And those engineers are not learning anything in the process so the cannot use the analogy as an excuse.

The fact that those service can reproduce copyrighted content proves that it was used during training. And was it legally obtained? How do you think, services like Udio bought millions of CDs? Or they got the training material somewhere else? You cannot legally download content from streaming services for example.

> Producing copyrighted things isn’t illegal.

There's no difference between producing and copying. Let's take a real world example: music samples. There's an entire clearinghouse process for music sampling, more or less forged after the 80s/90s when sampling blew up. Record companies and artist were like, "hey that's my song", courts agreed, a market was created.

This is pretty analogous to what's happening now, which is code samples. Developers are like, "hey that's my code". But that's where we're diverging, and this is probably because big companies aren't involved. People were sampling Atlantic Records' stuff. People aren't sampling Microsoft's stuff, they're sampling random GitHub OSS project guy's stuff.

But to your point, you're basically arguing that it's fine as long as no one listens to "Bitter Sweet Symphony". Most people think it's not the end user (listener) who's infringing copyright, but the party doing the copying (The Verve). Even if we accept your principle here, you're putting way too heavy a burden on people who use services like Copilot. Am I supposed to check that everything I autocomplete is properly licensed? You more or less said "shut these services down" in so many words.

> Producing copyrighted things isn’t illegal

That isn't true, strictly speaking. The right to reproduce work is covered by copyright law, irrespective of whether the reproduction is commercial or not.

So what? Most uses when we're talking about code or artwork are going to involve someone taking the generated result and publishing it somewhere.

> But if I use it, in a commercial manner, then it becomes a copyright violation.

No, that's incorrect. Commercial use has nothing to do with it. Any act of distribution, regardless of whether or not it's for commercial or personal use, regardless of whether you charge $100,000 or $0, falls under copyright law.

You can usually dismiss anyone who talks about the law in such absolute terms, especially when it comes to copyright.

The U.S. Copyright Office has some guidelines about fair use, and non-commercial as well as personal use are listed as considerations that courts take into account when judging whether an unlicensed copy constitutes an infringement:

https://www.copyright.gov/fair-use/

You're technically correct, but this ignores some unfortunate realities of DMCA for the smaller fry.

The bigger fry are and will keep fighting to change the ideas of copryright, now that it's inconvinient for them after spending decades strenghening it.

No one who draws Mickey Mouse for personal use is getting prosecuted or litigated, big or small.

I pretty much said that, yes. But the argument of "personal use" ends when you post on the internet. Which is what most "big fries" are doing as some endpoint.

But those are more "medium fries" anyway. The "big fries" aren't gonna take any risk whatsoever unless they are going the edgy parody route (someone like Adult Swim). There's little upside to Bungie or Microsoft or Laika posting Mickey mouse to begin with

You are making it seem like people aren't getting sued for drawing Mickey Mouse for personal use because Disney would rather go after the big guys.

My point is that it is perfectly legal to draw Mickey Mouse for your own personal use because it does not violate the law.

I see. We're in two threads and looping. I replied to this here: https://news.ycombinator.com/item?id=41383588

In summary, you probably can and Disney won't care, but it is technically not allowed outside of fair use constraints. It's not legal in the same realm that it's illegal to ride a bike in a swimming pool in Baldwin Park,CA. Key point: don't make money, don't be stupid, and don't be unlikeable.

Even then, most platforms control the content and they probably won't defend you and would take it down. That's not a legal matter so much as a platform policy.

>people aren't getting sued for drawing Mickey Mouse for personal use because Disney would rather go after the big guys.

It's even simpler than that. If a lawyer costs $10k to run a small claims case, and they have a shaky chance (fair use) or a low payout, it's not profitable to go after you.

That's why other factors need to come into play, like potential brand damage, scaring off imitators, or simply pissing off the wrong lawyer somehow.

Technically torrent sites do not host copyrighted content but you can go to jail for this.

But the AI is a commercial product, that is able to produce copyrighted material. How is that not using copyrighted material for commercial purposes?

A cassette recorder is a commercial product that is able to produce copyrighted material. If the user directs it to.

There was no media used as the material for the cassete recorder.

yes, and no. Stricly speaking you need a licesnse to allow you to use copyright information. Even if it's non-commercial (obvious example, if you post, say, Jasmine making out with Hitler on some non-monetized account, Disney can take that down. Or try. It really comes down to if the platform wants to argue fair use or parody or whatnot. Most won't defend you, though).

But enforcement-wise, Disney won't bother going after every copyright potential. They will focus on the biggest money makers or the biggest potential to brand damage. So it's not a worry for most people who will just draw some mickey mouse for a friend, or even private client as long as they aren't stupid.

What were you asking it to do? If it's less than 15 lines, tell me and I'll write the solution myself. You can check how close I get.

My point being that lots of OpenGL is practically boilerplate.

> there was no license provided for the code or the repo

Interesting - I assume any code that's not licensed as "free to use for whatever purpose I want"

Not at all: unless a license is provided, the code is fully protected under copyright and you have _no_ rights to copy it or use it in _any_ way you want (unless falling under "fair use" clauses for the jurisdiction you're in/the author is in).

The opposite

no license[0] is the default fallback when nothing is provided. Realistically, it's "use at your own risk", because someone who doesn't license may not even be aware of what others do with it (or you fallback to whatever rules of the platform you posted on).

https://choosealicense.com/no-permission/

Uh, no, this is not how copyright law works.

If there's only one way to do it, or a developer familiar with the field would independently come up with the same way of doing it, then the "copyrightability" of the result comes into question.

This doesn't stop you getting yourself a legal headache though, of course.

I do not understand your problem.

Do you wanna life in fear as a software developer because you did the same thing as others? Even if the problem has basically not an unlimited way of doing it?

Do you want a certain amout of code being (c) and patent?

I personally don't. I see an advantage of limited patents for complex magical algorithms were someone was really sitting there and solving a hard problem to reap the benefits for a short period of time, but otherwise no.

I do not want to check every code block for some (c) or patent.

This has happened quite a few times with me as well, both with chatgpt and phind (phind in particular is often basically stackoverflow with a few variable names changed).

One way that can happen is if your prompt and context are so specific, that this copied code is the only thing that matches.

This would also imply that this specific question is a rare one, with few examples online for it to train on.

>I think some of the confounding variables here were, 1. this was a very specific use case and not many examples existed, and 2. all opengl code looks similar, to a point.

Yeah, that's why I wouldn't trust AI right now with much except the most basic renderin boilerplate. I'd be so brazen to wager that 90% of the most vauable rendering recipes are prorietary code within some real time rendering studio. Of the remaining, half of that is in some text book and may or may not even be available to scrape online.

LLM's still needs a training set, and I'm not convinced the information even exists to be scraped on the public internet for this kind of stuff (if years of googling has taught me anything).