Taek

You can effectively achieve the same result with this simple operation:

  hash = sha256(current_time());
  for i := 0; i < n; i++ {
      hash = sha256(hash.append(current_time()))
  }

This is because the number of nanoseconds between hashes is actually itself variable, and this is true for physics reasons that are basically beyond the control of any attacker trying to manipulate your entropy. If your time() function has a resolution of nanoseconds, you only need your loop to iterate about 50 times to get a cryptographically secure amount of entropy. If your time() function has a resolution of milliseconds, you need to let this run for more like 20 milliseconds, and if your time() function has a resolution of seconds you need to let it run for more like 5 seconds.

The reason I like doing it this way is that it happens entirely in userspace, it's genuinely a secure method of generating entropy, and it has no dependencies on potentially buggy firmware or microcode outside of the time() call, which is both fairly narrow, fairly heavily used (meaning a bug is likely to be discovered during testing, as the implementation is likely heavily scrutinized), and also fairly easy to test independently - just look at the number of nanoseconds that elapse at each consecutive call to sha256(current_time()) and verify that there's some statistical variance. The above suggestions are assuming about 2.5 bits of variance between calls, meaning there should be a range of at least 20 nanoseconds between your slowest and fastest hash call. This has been true on every CPU I've ever measured, including microcontrollers.

I wrote this elsewhere but I felt it was worth saying again. "9x smaller" is an idiom, much like "spill the beans" or "it costs an arm and a leg".

Idioms don't have to make literal sense or be linguistically/mathematically correct to be useful. All that matters is that other people know exactly what you mean when you say it.

And, pretty much universally, if I tell someone "the compressed file is 10x smaller than the original", they are going to know what I mean is that the byte size is 10% of the size of the original.

That makes it an idiom that is perfectly okay for everyday use.

It's an idiom, much like "you are pulling my hair". Are you actually pulling my hair? Of course not; taken literally, many English phrases make no sense. But they are common elements of the language and everyone understands them, so they aren't problematic.

This is the same. Taken literally, "9x smaller" is nonsense, but everyone who hears that phrase knows exactly what mathematical operation you are referring to, thus it's a totally acceptable way to express that you mean to say 11.11% as large.

What parts of the United States can you visit which have a >10% chance of being kidnapped or murdered? That means for every 100 visitors, 10 people don't come home. I honestly don't think that applies to a single neighborhood in the entire country. If murder/kidnapping rates get remotely close to that high, the FBI steps in.

I also can't think of any US cities where neighborhoods have military grade armed security. Sure, there are places where every local business has an armed guard, but that's not really the same as hiring a trained private militia. The armed guards are for protecting against petty theft, not for protecting against organized crime.

On point four, I'm not sure if there are places in the US where minorities need to maintain relationships social relationships with cops as a survival mechanic, but it certainly doesn't apply to most cities, and I don't think it applies to anyone who is white.

On point five, I don't think you understand. There are parts of the world where having white skin will get you, quite literally, reminders every 15 minutes "hey it's really not safe for you here, do you want to hang out inside my shop while I call you a taxi?" No part of the US is like that for travelers. I know there are occasionally ICE raids that make the news, but "hey you strictly cannot be outside without a local chaperone" is just not a thing in the US.

I get that the political system could be better, but having been to places that actually are not governed by the rule of law, I can assure you that the US is doing quite well on that front. When there is actually no rule of law, consequences include:

+ entire regions / areas where merely visiting those areas invites a highly non-trivial (think, more than 10% chance) chance of being kidnapped or murdered

+ every neighborhood and business has substantial, often military-grade private security

+ if credit exists at all, it exists outside of any formal banking structure and will have interest rates that are north of 30% APR, sometimes north of 100% APR. I've genuinely seen interest rates on credit as high as 2% *per day*, and these are rates that the local population is willing to pay for certain short term expenses (like food)

+ families that maintain good social relationships with the local police/militants/whoever-has-guns live substantially better lives than people without good social connections to the local authorities.

+ Travelers are told on repeat: "it's really not safe here for non-locals, you should stay inside and also reconsider being in this part of the world at all"

+ If the travelers are there for business reasons, they are probably assigned 24/7 armed guards (as many as 4 guards per traveler, each guard carrying full-auto weapons) by the locals, provided for free.

And while the US maybe has a neighborhood here or there which might be like this, every part of every major city in the country has more rule of law than the above.

Definitely booting into a decoy system beats an automatic wipe.

I do like the idea of forcing due process to access an encryption key. I'm not sure if that idea is compatible with current law, but it seems just at the very least.

I just wrote a utility to rip all comments out of the code. Now the code is fully uncommented and it has saved lots of input tokens and also lots of meandering because the model is no longer getting stuck on bad ideas it told itself about.

A job faire might serve you well!

I have a hard time looking at the world around me and thinking "close to no value"

Some sounds are genetically more intense than others, like shrill screeching sorts of noises. So not quite as bad as the euphemism treadmill, because there is biology that makes certain sounds fundamentally less terrifying than others.

Was on the 6th floor of a tower. Shaking lasted almost 2 minutes. There was no damage as far as I could tell (Medellín) but they still evacuated everyone to inspect the building. City is in a bit of chaos, people are fine but unsettled, and the comms lines are clogged with people presumably calling their families.

Was a bit tense to see my phone alerts going off over and over, continually increasing the estimate of the earthquake strength, followed by the shaking getting worse.

For people who don't know, those phone alerts give you literally like 5 seconds of heads up. By the time I finished reading the first message, the shaking had already started. And I got the message while I was holding my phone and using it.

This title is killing me. "The runtime stays open" is such a heavy, attention grabbing sentence, it's all I can stare at on the front page. It's the type of writing that makes me hate working with LLMs, because it's so powerful at monopolizing my attention, and makes it difficult for me to focus on any of the other words on the page (whether the front page, or the comments page).

That's a different bad UX pattern. If a button has already rendered in a certain location, a new button shouldn't replace it without first giving the user ample warning that a material change is about to happen.

I wish software apps had "tape-out rules" the way that computer chips do. Basically, when you design a computer chip, a program reviews the design and compares it against something like 300 pages of rules with stuff like "wires of X metal and Y metal can't be within Z distance of each other".

We could make something similar for UX. Just a bunch of design pattern constraints that throw flags if you try to ship something with well established UX warts.

I don't think it's that bad, if I recall correctly it's about 8 kilobytes per token, and a token can be 3-4 characters so you're talking ~2 kilobytes per character.

An image token I recall is something like 16x16, so you get 32 bytes of overhead per pixel. And a character is minimally like 20 pixels including the whitespace, so you've jumped from 4 characters per token to maybe 12.

So 3x savings... which actually maps pretty closely to 60% savings.

There's also Chinese models, which aren't trying to self-limit capabilities.

Is anyone in favor of this?

What is Krea's approach to content such as pornography and gore? It's been frustrating to see all of the leading models take a very hard line on excluding vice content, even when it is perfectly legal, in the name of safety.

I hope you don't mind if I get super pedantic here. But what qualifies as "a problem", and what counts as "has not encountered before" and what counts as "adapted solutions"?

Because "a problem" could be as simple as conducting heat through a lattice of atoms and "not encountered before" could be from a specific temperature hotspot that was never seen before, etc.

It's really thorny to get to the bottom of things!

Can you define a clear boundary for me somewhere between a rock and a human brain at which consciousness is definitely not possible on one side of the boundary, but is maybe possible on the other side of the boundary?

Yes I agree. We have such a poor understanding of consciousness that really we can't even rule out whether a simple set of if-else statements is enough to create consciousness.

Common sense says "surely not"... But where is the exact point at which "surely not" becomes "well maybe..."?

I haven't been able to find a satisfying answer to that question, which forces me to assume (at least until a satisfying answer is found) that consciousness is a gradient and in fact does exist in little bits all over the place.

If an LLM, which abstractly is just a stack of transformers, is able to reason about itself, why wouldn't a chess engine (also a stack of transformers) also be able to reason about itself?

That reasoning may not manifest as English that we can read, but that doesn't mean it isn't there.

As information bounces through the many layers of the model, I find it plausible that you could get reflection in there somewhere.

I also find it reasonable that consciousness may not even require reflection / introspection.

That's the main problem with consciousness. We really have no idea how it works, therefore it's really difficult to conclusively state that something lacks consciousness

That's where things get interesting. As soon as you start asking what counts as communication... what about signals passing between cells? What about heat passing between atoms?

As soon as try and draw a firm line between "X counts and Y doesn't", you find that you really can't. There are no obvious boundaries between a deeply complex and fully functioning FL human brain, and a pile of atoms bouncing around arbitrarily.

On what grounds would someone establish than an LLM could be conscious but a sufficiently large/complex transformer model aimed at chess would not be conscious?

"we" is the set of people who believe that machines can potentially be conscious.

Another weird thing that keeps coming up - "people don't think that image models or chess models are conscious"... yes we do, and we have for many years.

Or rather, we aren't *certain* that those things are conscious. But the idea that they might be is not strange.

Your comment is conflating a few things here, the philosophical problem of induction is one I am very familiar with, and it's also unrelated to what we are talking about.

"If every piece of knowledge is created by combining previous ones, then no true grounded knowledge is possible" - this is not true, and has a very simple solution. What you are calling "grounded" knowledge comes simply from observation.

When the very first human saw that an apple falls of a tree, they have acquired what you call "grounded" knowledge. They can then combine that knowledge with the grounded knowledge that other things fall from trees to start reasoning about why things might fall from trees.

The problem of induction is different. It says, we know that things fall from trees because things have always fallen from trees, but how can be certain that what we call the laws of physics will not arbitrarily change on us?

And of course, it's a famous problem because it doesn't have a satisfying answer. The answer is basically "well everything we know is wrong if induction is wrong, so we will pretend induction is not wrong and hope for the best". And at least so far, that approach has seemed to work (heh heh heh).

I think this argument is fragile, especially when you consider agents like Claws. Any LLM that is training on its own previous interactions with users has indeed directly experienced them.

But also, what's the difference between having the memory of a prior experience and actually having gone through that experience?

For humans, reading a book about an experience and living an experience is different, because for a human the actual experience has so many more inputs attached (the sights, the smells, the sounds, etc).

For LLMs, when they train on a memory they can actually truly recapture that memory entirely by replaying the senses exactly. And LLMs are no longer limited to just text, they can do sound and video as tokens too.

Though, it's not clear why that matters to a consciousness. Whether your inputs are of one type or two or two hundred, there's no clear indication that a specific number is required as a catalyst.

It's not intuitive to me at all that the two are connected. Evolution adapted us to suffer because it is a powerful motivator for growth.

It's not clear whether evolution adapted us to be conscious, or if it just happened.