HNHacker News
TopNewBestAskShowJobs

inciampati

1,697 karma · joined June 29, 2017

submissionscomments
inciampati··on O1 isn't a chat model (and that's the point)
o1 appears to not be able to see it's own reasoning traces. Or it's own context is potentially being summarized to deal with the cost of giving access to all those chain of thought traces and the chat history. This breaks the computational expressivity or chain of thought, which supports universal (general) reasoning if you have reliable access to the things you've thought, and is threshold circuit (TC0) or bounded parallel pattern matcher when not.
inciampati··on O1 isn't a chat model (and that's the point)
Looking at o1's behavior, it seems there's a key architectural limitation: while it can see chat history, it doesn't seem able to access its own reasoning steps after outputting them. This is particularly significant because it breaks the computational expressivity that made chain-of-thought prompting work in the first place—the ability to build up complex reasoning through iterative steps.

This will only improve when o1's context windows grow large enough to maintain all its intermediate thinking steps, we're talking orders of magnitude beyond current limits. Until then, this isn't just a UX quirk, it's a fundamental constraint on the model's ability to develop thoughts over time.

inciampati··on The Toyota Prius transformed the auto industry
A regular EV also requires no special infrastructure. Just your regular house electrical system (e.g. 120v wall outlet in the US). Only on long trips do you need fast chargers or does range anxiety kick in. If you do a lot of long drives I can see that being a problem. But if not I'd be willing to bet you'd lose less time charging than taking your vehicle to the mechanic for maintenance and repairs. Because the EV, when built well, requires a frighteningly low amount of maintenance.
inciampati··on French modernists were alarmed, inspired by newspaper's voracious dynamism
Poetry with a heavy dose of truth.
inciampati··on RWKV Language Model
the recurrent model needs a mechanism to replay past context. no need to go quadratic to access all of it. they could replay multiple times to get effects similar to attention.

the hardware lottery, well... imo it's really about leveraging fully parallel training to learn how to use a memory. attention is quadratic but it can be computed in parallel. it's an end to end learned memory. getting that kind of pattern into RNNs won't be easy but it's going to be crucial before we boil the ocean.

inciampati··on Things we learned about LLMs in 2024
> They shouldn't be able to do this

Really? ;) I guess you don't believe in the universal approximation theorem?

UAT makes a strong case that by reading all of our text (aka computational traces) the models have learned a human "state transition function" that understands context and can integrate within it to guess the next token. Basically, by transfer learning from us they have learned to behave like universal reasoners.

inciampati··on Does current AI represent a dead end?
Want reliable AI? Stop approximating memory with attention and build reliable memory into the model directly.
inciampati··on The trap of "I am not an extrovert"
Socializing is always draining. It's a physical fact that it simply takes a huge amount of energy to engage with other people, no matter what you think your personality type is. Dealing with socialization is a big part of why we have these huge energy-intensive brains.

I think the discussion here suggests it's an issue of perspective. Is socialization a fact of life, or is it something you are allowed to opt out of?

It's okay to be tired. A day that leaves you exhausted should be celebrated and followed up with rest to prepare for the next day!

inciampati··on The trap of "I am not an extrovert"
FWIW, I'm "extroverted" in the terms used in this thread, but I'm also all of these: mathemetician, farmer, scientist. We can work on monk or priest but those might be mutually exclusive!
inciampati··on The trap of "I am not an extrovert"
Yes, extroverts do understand this limited capacity. It's like when you live an ultra sedentary lifestyle, and one day you realize you can't catch your breath after walking up four flights of stairs. Like any capacity, socializing requires exercise. We are all descended from a long line of people who lived extremely social lives. It's only in the last decade or two that it became possible to live a productive live by text, alone, without the intense realtime, full mind and full body experience of being with other people. Use it or lose it. And don't assume socialization is easy for some class of "extroverts" who are dominating "introverts" into being full members of the organizations which they work for. It's hard, worthwhile work for everyone. Socialization capacity is like any trait. You have to use it or you'll lose it. I say this because I think modern life is depopulating as people decide they have a condition that's some kind of innate disability. I wish y'all would accept that it's just hard, but worth it.
inciampati··on Scientists Clone Two Black-Footed Ferrets from Frozen Tissues
Telomere length is dynamically regulated. It sounds like SCNT does well at triggering the correct reprogramming to reset epigenetic features that might have led to Dolly's troubles (although it seems these were greatly overstated: https://blog.cirm.ca.gov/2016/07/27/cloning-breakthrough-dol...)
inciampati··on Amdahl's Law
Imo the microecon metaphor is off base. This is about any parallel process being limited by it's serial phases. There are ways of looking at economies that see this process angle where Amdhal's law plays a role. But it's not the rediscovery of microeconomics.
inciampati··on Salmon return to lay eggs in historic habitat after dam removal project
In order to survive, you wouldn't want to be wiped out if your home stream vanished. You'd want at least the likely chance of going to another stream to spawn and breed. Probably the salmon just swim upriver when they want to spawn. And it happened to be that now the Klamath is open.

Yeah, there's clearly tendencies for the fish to return to where they were born. I'm sure that's driven by all kinds of complex genetic memories and probably more importantly selective advantage due to adaptation to the specific characteristics of the given stream, but genetic memories for a specific stream seems a little bit unlikely.

inciampati··on Everything I built with Claude Artifacts this week
Exactly. Use aider to do this.

Tell claude or your favorite LLM to write a full plan to implement what you need in such a way that your coworker can implement it.

Copy the result into aider, and check the results!

inciampati··on C-Motive's electrostatic motors use printed circuit boards instead of magnets
Extremely high value analysis! Probably why we aren't seeing an announcement for a motive product release.
inciampati··on The unknowns surrounding the mysterious rise of cancer in young adults
The repair of the wear and tear seems to provide more opportunities for the body to reject cancers. Also, aerobic activity may encourage selection for oxidative metabolism which is very suppressed in most cancers. It's counterintuitive but the epidemiology is very clear that exercise is protective. It doesn't wear you out faster but rather helps you rebuild and maintain yourself more rapidly (without which, you wear out).
inciampati··on Forget ChatGPT: why researchers now run small AIs on their laptops
You can use it for everything. Just make sure that you have an input method set up on your computer and phone that allow you to use whisper.

That's how I'm writing this message to you.

Learning to use these speech-to-text systems will be a new kind of literacy.

I think pushing the transcription through language models is a fantastic way to deal with the complexity and frankly, disorganization of directly going from speech to text.

By doing this we can all basically type at 150-200 words a minute.

inciampati··on Open research Europe: Towards a collective open access publishing service
Yes, it's not for papers.
inciampati··on OpenAI acknowledges new models increase risk of misuse to create bioweapons
Then you have to boot it up. Bio equivalent of putting the lightning in the rock (semiconductors). The DNA is like the lightning (like the code) the rock is the biological system you drop it into to run it. Booting viruses is not super easy.
inciampati··on OpenAI acknowledges new models increase risk of misuse to create bioweapons
Yes, they absolutely are! If this thing they are working on (or prognosticating about) is world endingly dangerous, it increases their importance.
inciampati··on OpenAI threatens to revoke o1 access for asking it about its chain of thought
OpenAI created a hidden token based money printer and don't want anyone to be able to audit it.
inciampati··on Show HN: Vomitorium – all of your project in 1 text file
find ... | xargs head -n -0
inciampati··on Show HN: An open-source implementation of AlphaFold3
Are you familiar with ColabFold?

https://github.com/sokrypton/ColabFold

inciampati··on FB partner admits phone microphones listen to people talk to serve better ads
But it never quite clicked with everyone. It's nasty to do it on the sly.
inciampati··on They don't make 'em like that any more: the 3.5mm headphone jack socket
Is water resistance a reason why we're seeing the elimination of the 3.5mm jack?
inciampati··on The Water Myth (2018)
If you are working outdoors in hot weather (35°+) you can easily require 8-10L of water a day. That's like 50 "glasses". Then the next day if it's 20° you'll need only a handful of liters. It all depends on context. The article makes a great point that humans are super water inefficient. But we can handle high heat and aerobic output so long as we have water.
inciampati··on ARPA-H announces awards to develop novel technologies for precise tumor removal
This program is incredibly difficult to get access to and has been monopolized by a few elite institutions whose name recognition potential seems to have blinded funders. These institutions basically control everything about the cohort, but they're not innovating well with the data that's available and I haven't seen much of anything interesting coming out of the investment. This is in stark contrast to the UK Biobank which has access control but is relatively egalitarian in access. It's been producing major results since just a few years after it's inception, and continues to.
inciampati··on The Webb Telescope further deepens the Hubble tension controversy in cosmology
Does quanta magazine manage to reach this level of detail in other fields?
inciampati··on Antifragility in complex dynamical systems
Eloquent, realistic, and sad. I'm living my own version of this (in biotech). The journey is the wonder.
inciampati··on Medieval
I am over TE. Just today I was looking at the OP-1 that's had a dead key since it was about a year old and been completely non bootable since a year or so after that. With the EP-133 I made the mistake of believing TE would do a better job of the practical design of their instruments. But it broke with interface problems reported by thousands of people. TE wasn't very supportive of my repair request and I don't have time to chase them for a replacement of something fundamentally broken. I don't want more objects that won't last.
← PreviousPage 4 of 20Next →