hellohello2

Touchscreens are so passé, the future is talking to your computer and having it do things for you.

I though 4 bit (mixed) quant is pretty standard, at least for local models that basically what everyone runs? Unless I'm mistaken it can really be a negligible difference in perf.

By (morbid) curiosity did you program as a hobby before? Sorry for being a jerk, just actually wondering how different people are taking it.

IDK, I thought programming was great fun before and am having a blast making fun stuff with AI.

Yes it is, and imagination is how we got to the moon, or got rockets that land, or why I can talk to my computer now.

Why use apps that you dislike, just ask Claude to click through them for you. And no I'm not being ironic, this is the way things are headed, you just abstract over all the crap by having an agent do it for you.

"Slop is its own cure"

Social roles get created at the frontier of what isn't technologically feasible without a human, as culture develops around some form of human activity. Role A does B, role X does Y; I don't know, maybe its just time for new roles that didn't exist before.

Publicly-accessible AI started working 1 year ago, 2 years tops. What did you expect? It to solve war driving prices up?

The main difficulty with fast (low-latency) inference, is not actually computation but loading parameters from memory. The problem with generating 1 token at a time isn't that that its expensive computationally (it is, but so is training), but that you need to stream your entire model from memory for every single token (and also the KV cache but that's besides the point). So the strategy usually taken is to process big batches of user requests, letting you share the memory loads across users. This means individual answers aren't that fast, but you can do lots at once. This is why local inference isn't cost-effective; its because the model weren't designed for it in the first place.

My hypothesis for Jev is that they simply generate many answers independently in parallel from your prompt, and then discard the duplicates (or train to avoid duplication with attention between the ). In that way the entire batch is 1 user's prompt.

I was thinking about this some more, and perhaps the best solution for subscription plans would be to charge more for real privacy. In which case breaking that privacy would be committing fraud. Just a thought.

Option 2 feel reassuring enough, but is out of reach of particulars.

Option 1 is not. In part because it is opt-out (will it turn back on on its own like my Facebook privacy settings?), and not always respected (sending feedback can mean your chat is used?). Also because disabling "Improve model for everyone" is very vague.

There simply needs to be a setting like "my data is confidential", in which case there clear guarantees like there are for ZDR.

As an example, I've seen people speculate that while input prompts and output tokens are discarded, thinking traces are retained for training, which could leak information. I doubt this is true, but it shows that the policy is not unambiguous and reassuring enough to remove all doubt.

Thanks for asking.

You guys should seriously offering a clear way of working with (semi-)confidential data for particulars. Regardless of what is actually done internally, toggling off an opt-in isn't reassuring enough, which is why people are having these worries.

Generative models copy training data verbatim and also generalize, the two are not mutually exclusive.

You can have a look at the literature on exact copying in image models if it interests you, but just online we often see online examples of agents outputting code that already exists, even if its not the common case.

I very much doubt OpenAI points the model towards a specific conversation, but these trillion parameter models can very much "remember" their training data. For instance I can ask GPT to summarize my papers from their title alone, without looking them up, and it works decently.

Of previous, not concurrent work. Science is friendly competition, and spying on others is unfriendly.

To offer a counterpoint, Airpods Pro are a great product that significantly increased my quality of life. Depending on how sensitive you are to noise then having easily accessible/socially acceptable noise cancelling is incredible. Perhaps you are too much of a normie to get it ;)

I simply do not see how you can interpret "if they do not deny training on them, they can't deny plagiarism" as "all outputs are necessarily plagiarizations of the training data".

There is a difference between claiming an action is a transgression, and claiming it could be one.

I'm aware cluster sizes vary, I was mostly wondering how long much compute you need to serve a frontier model (since you seem knowledgeable on this topic). For instance back of the enveloppe Kimi K3 fits in ~24 H100, so my naive first impression is thats its not out of reach for a university-sized cluster to serve a few instances. I agree it may not make much economic sense I'm asking out of curiosity.

Perhaps I am underestimating the size of these frontier models then, how big of a cluster do you think would be required to serve a university?

Every university I know has access to clusters with fresh GPUs. Not sure when you graduated but you'd be surprised how much money is getting poured in I think!

I was too slow to edit this post but I'm not sure why I said this. This is too assertive about a situation I don't know much about.

Of malfeasance no, but they could have easily plagiarized unintentionally. If you commit mansalughter, you still need to explain yourself, even if it was a complete unlucky accident.

What I meant to say is that, in many cases, a generative model's output is not in fact steered by minor amounts by lots of training samples, but instead steered by a just few samples. Some outputs are influenced by many inputs, and some by very few, it really depends.

In answer to a post suggesting that training on a datapoint could mean plagiarism, you said that this would imply that all outputs are plagiarized. This is not the case, no, because generative models do not "copy" or "create", they do both at different times.

I did not agree or disagree with the original poster, I was explaining to you why I thought you disagreed with them. If you understand what I said above, then why do you disagree with them?

EDIT: I just saw your other post on "general inspiration" and I believe I read the situation exactly; you appear to believe that inputs used to train generative models get "lost in the parameter soup", but it is not always the case.

Between companies, direct malevolent competition is OK. Between academics, there are other rules to the game. When you go into a boxing match, you agree to get punched in the face.

All this to say, trust is important, and grounded in social convention. So I do agree with you, but also disagree.

Whenever this is OK or not really depends on how the breakthrough is contextualized, and how there people at play, here, agree to contextualize it.

In my view, in the blog post, there is much discussion about who will be publishing the paper. If instead it was just a blog post that said "oops, we beat you to it, our model is the best", it would have been different.

Hm. Looks like its possible everyone behaved terribly here unfortunately. :/

I remained impressed by ChatGPT however!

Are you certain of this? I would be inclined to believe you but it would be nice to know decisively.

But there is evidence, the blog post says: "While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models ."

In other words, yes, they had been using ChatGPT, and yes, ChatGPT could very well have trained on their data. Now that there is evidence, we need an investigation: yes or no, was it the case?

I don't believe people are denying that the model is impressive. The problem is that learning someone else is making progress on a topic using method X and then rushing to scoop them borders on academic misconduct. If, on top of this, their private conversations about X were used in the proof, I really don't see how its defensible...

This is a common misconception, so its understandable that you have it. Generative models can both plagiarize and generalize. The question here is which of the two happened.

When someone authors a paper, is it on others to proove the author did not use their work as inspiration? No, it is on the author to give credit where it is due. You guys are acting as if it its legal issue, when it is not.