Chabsff

Not necessarily. Here's a rough example (it's not what's going on here, just a representative idea):

There are words/tokens that are heavily correlated to the prompt (a yes or a no, for example), and then there are others that are going to be less so (adjectives with a lot of synonyms for example).

Given a text, you can identify what the "load bearing" and auxiliary words/chunks are. Then, looking only at the auxiliary words/chunks, you should, in principle, be able to determine what other wordings could have gone there instead. From this, you can, very roughly, recreate the token probability distribution that was in effect when those tokens were generated. With the probability distribution in hand for enough chunks of text, you can start inferring properties about the RNG process that was used to sample from those distributions.

But then, this notion of "load bearing" vs "auxiliary" can be expressed directly in the probability distributions. A load bearing token just has a very high probability, and thus any RNG bias that may have been in effect will likely be swallowed in the distribution. So the parts of the text that are highly dependant on the prompt will naturally not be contributing much information about he RNG in the first place.

I don't think it's the dream, I think it's the cynical market-driven logic.

Automation has been rolling out for a long while now, and most of the low-hanging fruit of things that can easily be redesigned in a cost-effective way has been dealt with already. The role of these robots is to address all of the stuff that would have been automated by now if they could have.

Humanoid robots can be dropped in wherever human labor is currently used.

That makes them extremely easy to sell to industry. This unlocks, in principle, a large chunk of the potential untapped industrial automation market that still relies on human labor because the ROI of redesigning production lines didn't make sense.

I almost entirely agree with this. However, after coding quite a few large projects with LLM assistance, I can say with certainty that there is some form of "art" at play with regards to structuring requests for LLMs.

However, this "art" is not so much about how to present a given request to the LLM, but rather guestimating what the scope of the next chunk of work should be to balance getting as much out of the model as possible while avoiding the machine going off the rails.

Obviously, this is a moving target and different models perform differently for various chunk/scope of work. I look at my successful sessions with LLMs and I'm not sure I'd be able to articulate a clear set of rules to apply here. You just... gradually build a intuition for how much you can throw at the LLM at once.

That being said, I'm pretty convinced at this point that this is a property of the coding assistants as they exist today, and what "working well with LLM assistance" means will keep on changing.

Hello HN. There's still a long road ahead for the project, but since I've recently closed the loop on bootstrapping the language's core within itself, I thought now's as good a time as any to throw it out there and start getting other people's takes on it.

Chirp is my attempt at engineering a programing language from the ground up that packs together all my favorite bits and pieces into one tight and consistent package. It's got a bit of Rust, a bit of Zig, a LOT of TypeScript, and a lot more LISP than I initially expected.

What sets it apart is its treatment of sets as a trait, and how they are used to express constraints on variables while keeping the code as boring and straightforward as possible.

Here's a taste:

  let fizz = { x:int | x % 3 == 0 };
  let buzz = { x:int | x % 5 == 0 };
  let fb_range = 0 ..= 100;

  let eval(v: fb_range) = match v {
    fizz ∩ buzz => "fizzbuzz",
    fizz => "fizz",
    buzz => "buzz",
    `any => f"{v}"
  };

  for (v ∈ fb_range) `print(eval(v));

To manage expectations: Code emission isn't ready yet. As things stand, it's effectively a scripting language with an intuitive set-centric syntax and runtime constraint checking. But, especially since scripting a compiler is what it's meant to do, I think it's still enough to show around a bit.

TU-level concepts (mostly) dissolve during the linking stage. You need to compile with -c to generate an object file in order to see the distinction.

Also, the difference manifests in the symbols table, not the assembly.

> and to avoid the warning (error) the code is decorated with compiler pacifiers, which makes no sense!

How is that a bad thing, exactly?

Think of it this way: The pacifiers don't just prevent the warnings. They embed the warnings within the code itself in a way where they are acknowledged by the developer.

Sure, just throwing in compiler pacifiers willy-nilly to squelch the warnings is terrible.

However, making developers explicitly write in the code "Yes, this block of code triggers a warning, and yes it's what I want to do because xyz" seems not only perfectly fine, but straight up desirable. Preventing them from pushing the code to the repo before doing so by enabling warnings-as-errors is a great way to get that done.

The only place where I've seen warnings-as-errors become a huge pain is when dealing with multiple platforms and multiple compilers that have different settings. This was a big issue in Gen7 game dev because getting the PS3's gcc, the Wii's CodeWarrior and the XBox360's MSVC to align on warnings was like herding cats, and not every dev had every devkit for obvious reason. And even then, warnings as errors was still very much worth it in the long run.

It's not just a pure matter of law, and looking at it from that perspective is naive.

Legacy publishers in general (and a few big ones in particular, like der Spiegel) have been lobbying hard for legislatures to redirect big tech revenue to their failing businesses.

The focus on AI here is really just the continuation of that ongoing fight that has been raging for over a decade now. If it wasn't that, it would be some other wedge.

I'm not saying Google is squeaky-clean here, far from it. However, it's important to keep in mind that the main drive here is to get publishers paid, not to force Google to be accountable to some specific standards.

In the grand scheme of things, we've only had about a quarter century where you needed a *very* specific kind of problem where prosumer hardware wasn't adequate across computer science as a whole.

It's kind of amazing we got that at all for a while.

There's no confusion here. It's pretty easy to make the argument that the combination of auto play and short form is orders of magnitude more problematic than the sum of their parts.

There is a HUGE difference in that the combined short length with the fact that the video starts playing before you even have a chance to make a decision on whether to watch it or not leads you to a "heh! I'm here already, might as well just watch the thing".

Honestly, I don't mind the format in principle, and the process that goes from YT's homepage to watching a single one of them is not that bad to me. As long as I get to make a decision that I want to watch something, consciously go "I will click on this thing and watch it" and only then proceed to watch it, then it's _fine_.

It's the algorithmic loop that starts the moment you scroll to the next video that starts playing before you even have a chance to decide whether or not it's something that you want to watch that's abhorrent to me.

If your argument is that the guardrails only provide a false sense of security, and removing them would ultimately be a good thing because it would force people to account for that, that's an interesting conversation to have

But it's clearly not the one at play here.

Agreed.

DEATH handing out swords to kids as Santa in the Hogfather is a funny joke, not an example to follow.

> Have you read the literature? Do you have a background in machine learning or statistics?

Very much so. Decades.

> Being trained by predicting one token at a time is also not a criticism??! It is just a factually correct description...

Of course that's the case. The objection I've had from the very first post in this thread is that using this trivially obvious fact as evidence that LLMs are boring/uninteresting/not AI/whatever is missing the forest for the trees.

"We understand [the I/Os and components of] LLMs, and what they are is nothing special" is the topic at hand. This is reductionist naivete. There is a gulf of complexity, in the formal mathematical sense and reductionism's arch-enemy, that is being handwaved away.

People responding to that with "but they ARE predicting one token at a time" are either falling into the very mistake I'm talking about, or are talking about something else entirety.

> They do it by iteratively predicting the next token.

You don't know that. It's how the llm presents, not how it does things. That's what I mean by it being the interface.

There's ever only one word that comes out of your mouth at a time, but we don't conclude that humans only think one word at a time. Who's to say the machine doesn't plan out the full sentence and outputs just the next token?

I don't know either fwiw, and that's my main point. There's a lot to criticize about LLMs and, believe or not, I am a huge detractor of their use in most contexts. But this is a bad criticism of them. And it bugs me a lot because the really important problems with them are broadly ignored by this low-effort, ill-thought-out offhand dismissal.

For the sake of this conversation "understanding" implicitly means "understand enough about it to be unimpressed".

This is what's being challenged: That you can discount LLMs as uninteresting because they are "just" probalistic inference machines. This completely underestimates just how far you can push the concept.

Your pedantic definition of understand might be technically correct. But that's not what's being discussed.

That is, unless you assign metaphysical properties to the notion of intelligence. But the current consensus is that intelligence can be simulated, at least in principle.

Yeah, but that's their interface. That informs surprisingly little about their inner workings.

ANNs are arbitrary function approximators. The training process uses statistical methods to identify a set of parameters that approximate the function as best as possible. That doesn't necessarily mean that the end result is equivalent to a very fancy multi-stage linear regression. It's a possible outcome of the process, but it's not the only possible outcome.

Looking at a LLMs I/O structure and training process is not enough to conclude much of anything. And that's the misconception.

> I feel like LLMs are a fairly boring technology. They are stochastic black boxes. The training is essentially run-of-the-mill statistical inference. There are some more recent innovations on software/hardware-level, but these are not LLM-specific really.

This is pretty ironic, considering the subject matter of that blog post. It's a super-common misconception that's gained very wide popularity due to reactionary (and, imo, rather poor) popular science reporting.

The author parroting that with confidence in a post about Dunner-Krugering gives me a bit of a chuckle.

Not by virtue of that alone.

A choice of tech stack can never be enough to prove anything. It only establishes a lower bound on resource usage, but there is never and upper bound as long as while() and malloc() are available.

The black box I'm referring to is us.

You can't have it both ways. If your test for whether something is intelligent/thinking or not isn't applicable to any known form of intelligence, then what you are testing for is not intelligence/thinking.

Ok, but how do you go about measuring whether a black-box is doing that or not?

We don't apply that criteria when evaluating animal intelligence. We sort of take it for granted that humans at large do that, but not via any test that would satisfy an alien.

Why should we be imposing white-box constraints to machine intelligence when we can't do so for any other?

I'm glad someone picked up on my dumb joke :), I was getting worried.

That being said, round-tripping works just fine, axiomatically so, until you go out of your way to discard the imaginary component.

And phase. People always forget about the phase as if it was purely imaginary.

You might have a point if this was a user problem, but it's not. It's a site operator problem. And from their POV, a CDN provides the exact same functionality.

I know this is tongue in cheek, but I'll give it a serious reply in case someone finds themselves "inspired".

A CDN. What you are describing is a CDN. We have CDNs today and the problem still exists because most of today's websites refuse to operate within the constraints. There is no need for new infrastructure to deploy this solution, we just need website operators to "give up" and operate in a more static way.

The bots flock to where the data is. Moving to a different network is just begging for the bots to tag along.

My experience with SSA is extremely limited, so that might be a stupid question. But does that remain true once memory enters the picture?

The llvm tutorials I played with (admittedly a long time ago) made it seem like "just allocate everything and trust mem2reg" basically abstracted SSA pretty completely from a user pov.

I know that footguns are broadly "on-brand" for the flipper zero ecosystem, but exposing high-voltage leads like that without any warning whatsoever seems a bit much.

Isn't it by far and wide the most common use of taxi services? It certainly is basically the only time I ever use one.

Waymo getting into that space seems like a pretty big step up in market penetration.