_0ffh

They can only pay so much because of the expectations about the generated value. Those might turn out wrong, as any expectation, but value is very much in the picture.

You can try "language model", that's what I use when talking (when writing I use LLM when I expect the abbreviation to be understood).

I find just juxtaposing these two normal words flows much more nicely from the tongue than the comparatively awkward "LLM".

Not the same person here, but a data point nonetheless. Before supplementing D3 I had a cold basically every year, sometimes twice a year. Since I brushed up my levels I average 1 cold in 6 years.

To me it appears as if the study using the constructed dataset was a completely different one than the one that was concerned with AI.

For the AI study real data from "3.4 million people who submit 4 million job applications to 1,700 job postings across 150 employers and 11 industry sectors" was used.

I wonder how personality forming it is, being a curious kid growing up hacking on computers. If you don't get what you expected, it's almost never the computer's fault - it means you did it wrong, and need to reconsider. There's no excuses and no dumping responsibility on anyone or anything else.

I have the feeling it probably teaches you something, or at least it should. Something not too unlike epistemic humility, maybe.

It would still degrade it's effectiveness, which is what they claim to want. Exaggeratedly: If it wasn't so, you'd just need fundamental math in the training data, as everything else can be derived.

The question is: If biological, computer security, and ML research are so bad, why do they even train on the relevant data?

The only answer that makes sense is they wanted the model to be competent and usable in these fields, just not by you, which is why they had to bolt on a badly functioning crippling device after the fact.

No, it's not about reverse engineering. It targets ML research.

No at least we know why they spent all that money on "safety research".

I blame it on the art world being full of pseudo-intellectual little shits, so of course they get off on the Marxian esthetic of throwing around signalling phrases like "late capitalism". They live and die by pretense, there just isn't anything more than that to these "people". All facade, nothing inside. The average LLM has more soul than these types, I fear.

There is actually a way to get really amazing sample efficiency out of a learning setup, and that's engineering in a load of appropriate inductive biases, which personally I am convinced evolution has done for us. Explains a big chunk of the "how are brains so sample efficient" problem really easy, but unfortunately without handing us an easy way to replicate it, which makes it unpopular. Also, it's something that we don't really want to do in the same way evolution has, as all those biases do even further reduce sample efficiency for all the things for which they are not appropriate.

Thoughtful comment, and I'd like to add another angle: However meaningful it is to say the community is divided, I also think that individuals are "divided" on the question as well.

I can speak from myself as an example (although n=1): I am incredibly open to machine learning and the advances it brings. On the other hand I am extremely conscious of the fact that the current LLMs do often write bad code, which becomes especially obvious once projects go much beyond "private toy" size. For me personally the consequence is that I try to make my projects even more modular, and the modules even more clearly delineated. With proper guidance, current LLMs work mostly fine when working on isolated modules. That doesn't mean they don't sometimes fail even there, but they also on rare occasions come up with surprisingly clever solutions when you let them loose on a code base, as long as the problem you want them to fix is mostly isolated in a couple of modules.

So long story short, you can be all for LLMs and still be conscious of their shortcomings, and that just vibe coding applications you intend to let loose on unsuspecting customers is probably a bad idea, and possibly outright immoral. We have already seen more than enough examples of vibe coded applications dumping sensitive user data into the lap of anyone who is inclined to pick up a stick and prod them a little bit.

Good news! On the current trajectory, I am very hopeful nonlinear RNNs could make a comeback. Which would incidentally help to ease the memory pressure for inference tasks.

Didn't have one. I was convinced I would experience this since I was a teenager. Blame science fiction if you will.

You're right to be sceptical. I have trained reasonably good SLMs for the TinyStories dataset on my 4060Ti (16GB) with no problems. You'll only encounter problems if you want to try if your ideas scale up to models any bigger than "arguably tiny".

That's a valid concern, given the paper makes clear that the effect over the polite/impolite scale seems to be model dependent (it finds the reverse correlation of earlier studies on even older models).

Maybe coupled with a nod to manus-mania?

What problem would that be that has not already been addressed further up the chain?

And it is an ad hominem - it's nothing more than an allegation impugning the character of libertarians in order to dismiss their arguments. The allegation alone does neither prove anything about the actual character of these people, nor what their view on reality and empathy actually is, nor if that view is actually wrong, nor who is doing the actual projecting here.

I prefer to judge such advice by the available facts, not by hearsay about the moral character of the advisor - especially not hearsay spread by his enemies. Your ad hominem has no bearing on the argument.

So, how is trusting politicians and bureaucrats to be selfless and focused on their duty to society working out for you?

Buying out maybe, but that only exacerbates the problem for the company in the long run. Regulatory capture is what actually works, but not within the libertarian framework, because regulation again is not a market mechanism, but government intervention into the market - exactly what libertarians say we should have less of in the first place.

Mind you, not different, or "better" intervention, but less, or even none at all. One could argue the point about libertarianism is that you can't trust the government to do a good job because it is based on force, and not voluntary market interactions, and hence lacks the proper incentives. It's just a bunch of guys on a spending spree with other peoples money, and their incentive is to make as much of it as humanly possible land in their own pockets.

That's infuriating to me, and I wasn't even the victim!

I'm so lucky such a thing never happened to me. The closest thing was a math teacher in middle school asking me about a problem solution of mine in a test, that he had marked "mysterious" (I had to come up with my own solution path during the test because I hadn't memorized the canonical one we were taught). He asked me to explain my reasoning, and when he was satisfied that it was sound he gave me full marks for the question.

The corporation (that runs internally as a planned economy) will get more and more inefficient the larger it gets, because that is what planning an economy does. Which in turn means it will loose market share and be forced to lean up until it is competitive again.

Personally, I'm not bothered very much by LLM confabulation, as long as it's the result of missing context. In most practical tasks, we either give context to the model, or tell it to find it itself using the internet. What I am concerned with is confabulation that contradicts available in-context information, but that doesn't seem to be what is measured here.

* In favour of our genes.

Wouldn't the higher density of liquid water be an advantage?

Well, the famous Turing test was evidently insufficient. All that happened is that the test is dead and nobody ever mentions it anymore. I'm not sure that any other test would fare any better once solved.

Still, attributing that progress to "years of research at Google" alone is simplifying the facts to the point of being just plain wrong. That kind of research was always very much in the open and cooperative, with deep levels of standing-on-shoulders.

Attention e.g. was developed by Dzmitry Bahdanau et al. (those being Kyunghyun Cho and Yoshua Bengio) in 2014 while interning at the University of Montreal.

The insight of the paper you point to was that with attention you could dispense of the RNN that attention was initially developed to support.

Openreview link is not working, was split apparently.

https://openreview.net/forum?id=tO3ASKZlok

Also nobody in his right mind uses lookup tables where the table value is actually the float approximation of the true f(x) - you choose the support values to minimize an error (e.g. mse) of a dense sampling of your interpolated value over x (or, in the limit, the integral of the chosen error function between the true curve and the interpolation of your supports). If you want to e.g. approximate a convex function using linear interpolation, all the tabulated values f'(x) would be <= the true f(x).

I took a look and it's probably a color isoluminance effect.