kmod

You should check out the new nogil project by Sam Gross, which is what's being talked about these days -- he actually successfully removed the gil, but yes with the tradeoffs that you mention. The other projects were, by comparison, "attempts" to remove the gil, and didn't address core issues such as ownership races (which are far harder than making refcount operations atomic).

Assuming those numbers are realized it would mean bankruptcy, essentially, and questions like this are pretty standard and well-thought-about there. IANAL but I think this is why Chapter 11 bankruptcy exists (where you keep the company going because that's valuable) vs Chapter 7 (where you liquidate it). I think the Purdue bankruptcy is similar where the company is somewhat being handed over to the people that were harmed, because that's more valuable to them than selling the company piecemeal and then distributing the proceeds.

I think it's safe to say that if people were getting value out of this then we would be hearing about it a lot

My understanding is that most of the safeguards are inside the model, but there are some safeguards that are outside the model. In particular if you ask the API to generate copyrighted data it will, but then the connection will mysteriously break after the first few words which I assume is a separate system watching the responses.

Not disagreeing, but a fascinating thing they did (as a one-off fine-tune?) was teach ChatGPT about the openai python client library, including the features that were added after the cutoff date.

"copying" != "copyright infringement": I'm just saying that the LLMs are copying, and I'm not getting into the legal/societal question of whether we want that to be illegal or not.

We as a society have determined that certain sorts of non-consensual copying are allowed: "fair use" broadly, and maybe you can consider "mental copying" in this category. Maybe we'll add LLM training to the list? It's not like copyright rules are a law of nature: we created them to try to produce the society that we want, and this is an ongoing process.

Again, I think there are fascinating questions 1) does LLM training violate existing copyright law + case law or does it maybe fall under a fair use exemption, and 2) is that what we want. But I think "do LLMs make copies" is dull and trivial and I don't know why it comes up.

Not trying to express an opinion on the legal matter, but as a technical matter it's pretty obvious that LLMs create copies of (some of) their training data.

Here's GPT-3.5 reciting the Declaration of Independence: https://chat.openai.com/share/eb30c373-7fec-4280-892d-479567...

Unless you're claiming that GPT-3.5 is deriving the Declaration of Independence (from information about the founding fathers?) I don't see how there's room for debate about whether information has been "copied" into the model.

I have done this test in the past with copyrighted material (harry potter) but they have since added safeguards against it, but my understanding is that the model is still capable of it.

I'm saying `A => B` (something should be illegal if it's primarily for crime) and you're saying `-A` (LLMs are not primarily for crime), which is not really a disagreement. My point is disagreeing with the GGP who argued that `-B regardless of A` (LLMs should not be a crime regardless of whether they facilitate crime, because it is the end-user who is committing the crime).

I happen to mostly believe the conclusion (`-B`) but not the particular argument.

This is the "guns don't kill people, people do" argument. Not saying that that proves things one way or another, just that assigning responsibility is not really a cut and dry question and many people think that it's important to look prior to the final interaction

IANAL but I believe in the US tools that are designed to circumvent copyright are illegal, which makes sense to me inasmuch as one believes that copyright should be protected

Sorry that this is a bit of a pet peeve of mine, but I think it'd be helpful if you described what your product is: I agree it's a good idea to list its features and targeted use cases, but those are descriptive properties that don't say what this is.

Sure at the end of the day the only thing that matters is if my use case is handled, but after decent effort I was unable to determine that for myself. The problem is that there are a lot of different things that have the properties you've described, and several of my key requirements are not entailed by your description.

As an example, I remember seeing a product that takes a command line tool, inspects the available flags, and produces a UI that lets you fill them in visually. I believe this has all the properties that you have used to describe your product (turns backend code into a tool anyone can use), but I suspect that Onu is different in some essential ways.

Though fwiw for my personal use case I'm hoping for something lighter-weight. It's great you offer a self-hosted option (exposing this stuff to a third party is a non starter for me and I suspect many others) but "self-hosting" connotes mental burden to me, vs other options in the space which are not exactly "hosted" in any sense. I think maybe I'm not really your target customer.

Anyway, I wish you luck! I'm happy to see more competition in the space.

fwiw OpenAI uses different models for ChatGPT GPT-4 and API GPT-4 (the latest one, not talking about the pinned 0314 one). In the past I noticed the API model was newer than the ChatGPT model, but in general it seems like they are willing to make different tradeoffs between the two https://twitter.com/kevmod/status/1643993097679020037

I wouldn't be surprised if they are trying to reduce the cost of the ChatGPT GPT-4 model since if you use it heavily they will be losing money on you. They could also be trying to increase the number of people they can serve with the existing amount of compute that they have available.

In my anecdotal experience I noticed that ChatGPT GPT-4 had gotten far faster recently which is consistent with a theory that they are trying to cost-optimize the model, though today it is back to the normal speed. I've also had some frustrating interactions with GPT-4 recently similar to what people are saying, but overall I think the prior is pretty strong that we are seeing normal statistical variation.

"1% false positive rate => 99% confidence about a positive signal" is a really intuitive step to take, but you should be careful because it turns out to be horribly wrong in general: https://en.wikipedia.org/wiki/Base_rate_fallacy

(Perhaps not in this case but in general this can get you into trouble)

Though personally I'd say that it's quite likely that the p=0.01 threshold falsely flagged a non-cheater, which to me says it is too lax.

P(them saying this | you can kind of get negative reviews taken down) = 1

The lines you quoted give them tremendous wiggle room, and nothing from the original article is actually disallowed by them.

They are necessarily non-constructive existence proofs, for sure. You're not alone in some potential unease with that: https://en.wikipedia.org/wiki/Constructivism_(philosophy_of_...

I think the term of interest here is non-computable: irrational numbers can have finite kolmogorov complexity.

You're kind of "begging the question" here, where you're assuming that non-computable numbers exist and then using that to show that some numbers are non-computable. You can definitely show that these things exist, but that relies on "believing" the set of axioms that you used to prove it.

IANAMathematician, but it's "true" in the sense that we primarily use a set of axioms (ZF/ZFC) which produce this result. But in a sense one of the axioms is that these things exist (powerset axiom), so this result is a bit less illuminating than one might hope.

My understanding is that we can reason about these sorts of non-constructible sequences but we can't really know if they "really exist" because it's not clear what that even means if their existence has no implications for physical reality (which is possible but not known). So now we're into the territory of whether there is such a thing as abstract truth independent of reality.

I wrote a blog post in the past about my reading on this topic, and I can't claim that's it's accurate but at least I tried to talk about this subject: https://blog.kevmod.com/2022/04/09/do-the-real-numbers-exist...

The bar on my block (NYC) takes it a step further: they require you to install an app to see the menu. So I guess it can always be worse?

OpenAI's data privacy policy is... worrying. Even when they say they don't look at your data, they leave a carve-out that they will look at it if they are "concerned" about it. I hope things like this help pressure them to change.

The project was originally codenamed "s3box" and then changed for obvious reasons haha. I think Drew picked the name "Magic Pocket"

I think that's right

Funny that you say this, the more common view I've heard is "we made JS way faster why can't we do that for python". I believe I heard Mark Shannon (one of the lead devs on Faster CPython) say this at one point.

I haven't followed things recently but I know that at least initially they purposefully kept the bytecode intact so that existing debuggers and compilers and what not would still work. I believe they put the optimized bytecode in a different field, and accept this duplication as the cost of supporting the feature you requested.

I always dislike the "requests per month" metric because it seems like a transparent attempt to impress with a larger absolute number: 500k requests/month is 5sec per request, which feels pretty underwhelming

I built something that summarizes hacker news discussion pages for me, and I have been very pleasantly surprised how much nicer it is to read "angryuser1 and angryuser2 got into a heated discussion" than to actually read the discussion. Forcing everything into the neutral gpt voice has greatly reduced the emotional valence of my HN consumption

I believe the cuda stuff, via Nvidia licensing restrictions, is forced to live outside of these packaging systems (so that you sign a Nvidia eula). Not saying this is a good thing but I think that none of the systems you mentioned would handle this well either

I think part of the problem is that it's technically correct to say "my product does X" even if it does X extremely poorly. I'm not sure if this can be changed because any line for "does-X vs does-not-adequately-do-X" is going to necessarily be subjective.

So personally I think the problem is that people see "this product does X" and interpret that to mean that it does X well. I don't think it's necessarily bad that we're seeing an explosion of AI tools that are a bit underwhelming if people understood it as such -- we're on, after all, a site with a heavy startup focus and saying "your product doesn't do everything that I want" is a bit antithetical to that.

But yeah specifically for this one there are arguments that "X is not even possible, especially not with this approach" so it's a bit more egregious.

I don't quite understand this. The returns to scale have always been sublinear (ie "diminishing"), but the scale-maximalists didn't worry about that before. Also I heard Sam tout on the Lex Friedman podcast how amazing it is that the scaling laws worked so well for GPT-4. So I wonder what changed?

Sure yeah the cost numbers are getting very large and we can't keep scaling forever. But Google could easily 10x the training cost of GPT-4 if they thought it would protect their search business. I'm still skeptical that scaling is enough to reach the thresholds we want, but I'm surprised that it's being claimed right now when there's a huge rush of new money into the space. I wonder if this is some sort of misdirection by Sam

I think this is amazing that this is written by someone in the field, so it's possible it was motivated by personal experience. I just picture someone saying "ugh my peers are only more successful than me because they're more attractive, and I'll prove it!" Not saying I actually think this is what's happening but wouldn't that be amazing

Do you have any evidence that this is GPT-3.5 level, or are you just repeating what they said? We have an abundance of claimed capabilities already; that's not what's lacking.

Bold of you to assume that people actually run GPT based loops!