Comment by mjburgess | Hacker News Reader

Comment by mjburgess | Hacker News Reader

Parent Full thread

> Only if there is no constraint on the model. But the fixed number of parameters

Nope. cf. the under-determination of evidence by theory.

> So accurately modelling the past is certainly very informative for the future.

Nope. cf. we don't have infinite measures on infinite events in the past.

Consider that any given measure of an event, say M(E) is really a measure of a near infinite number of causes, say C1...n. Now, how many independent measures of these do we have? A handful.

so we really have, M(E|Controlling(Handful C1..n)))

do we want to model that?!? No.. that's insane. That's superstition. That's what all of science stands against.

Here's what we do: we build models not fit to any data. We build models that can generate the data we observe, but we build them by familiarity with reality *NOT* with the measures M.

How do we do that? Many ways, but the body is a key component of the answer. We have practically certain causal models of the body.

   ConceptCup = MySensoryMotorActions(RealCups)
   I_Touch =  MySensoryMotorActions(Me)
   P(I_Touch_Cup|Cup_Exists, ConceptCut) ~= 1
   P(I_Touch_Cup|-Cup_Exists, ConceptCup) ~= 0

by iterating through such sensory-motor actions we can find concepts which produce these discrete splits. We call these concepts 'objects'. And from these we assemble models of reality whose shadow is our measures.

We handle the cup, and by handling the cup can imagine the cup, and by imagine the cup can generate measures of the cup.

You cannot reverse from a shadow of a vase to the clay vase itself: there are an infinite number, given abribtarily infinite parameters. The task is to find the right parameterisation; not to suppose that any given one has one solution (it doesnt, so a fool's hope anyway).

Such a condition is just a formalisation of supersition: my model is decided by the data; my parameterisation is 'free'. My model of the world is cooincidence.

Indeed, the truth is, in a sense, only a single parameter.

>Nope. cf. the under-determination of evidence by theory.

You mean the underdetermination of theory by evidence? This isn't really relevant. Given no prior information and no constraints on the model, the theory is underdetermined. But given some strong constraints, there will be one model that best explains the data. Of course a model can't deal with an arbitrary distribution shift, but no one expects a model to be insensitive to a change in the underlying data. The question is whether the constraints of natural systems and an LLM with a given inductive bias and finite capacity are sufficient constraints. It's not totally clear that it is or isn't, but this isn't decided by the underdetermination principle.

>Nope. cf. we don't have infinite measures on infinite events in the past.

I don't see how this is relevant to the point it is in response to.

>Such a condition is just a formalisation of supersition: my model is decided by the data; my parameterisation is 'free'. My model of the world is cooincidence.

Everything is data in the end; your interactions with the world by touch are just data. Our brains have an inductive bias towards certain interpretations of certain data, and these interpretations tend to promote proliferation in organisms. But LLMs have an inductive bias as well, one that allows them to search for circuits that generate training data. LLMs tend to land on interesting models that result in non-trivial generalization abilities in some contexts. This ability goes beyond just frequency modeling. Of course LLMs are limited to what it can know of the world through its data paradigm. But so are we. Such a limit in itself doesn't imply an in principle limit to modeling/understanding in LLMs.

Our "brains" engage in abduction, never induction. And the actions of our body under our will aren't "data".

P(SensationOfTouchingACup | MyConceptCup causes HandMovement causes Sensation)

is not something with a statistical condition.

You're projecting statistics onto reality. Reality is a place of necessity, not frequency. Our bodies place of causes not consequences. Our interaction with the world is a causal knowledge.

If everything were P(A|B) knowledge would be impossible. Thankfully we're in the world and we know directly and without inference: we move our hands according to a technique and they so-move. Absent this direct, immediate, causal, certain knowledge of our own bodies --- there is no way of knowing anything.

All knowledge is a recursion from the certain causal mechanism of the body: hands to tools, tools to models, models to data; from data to refinement of models.

Otherwise all "knowledge" would be LLM-like, based merely on pre-occurrent patterns of data. There would be no imagination, no possibility, no necessity, no reason.. indeed, no knowledge.

The world painted by P(A|B,| stupid limited dumb measures of the past) is a dark, dangerous and fickle one.

You should not wish to live there. It makes no sense, nor could it. Thankfully, you're able to walk around; imagine things that have never been; actually learn; grow (organically); adapt (physiologicall); develop skills (sensory-motorily). So you arent so severly disabled you're reduced to replies which are maximially consistent with what happens to be in alll ebooks ever written by people who could write them for the first time.

You arent so disabled that the very basis of writing is precluded: direct access to the world.

wow @mjburgess

you need to elaborate on your theory of knowledge/concept formation as I have not seen it explained anywhere else in this manner. particularly: "Absent this direct, immediate, causal, certain knowledge of our own bodies --- there is no way of knowing anything.". and "All knowledge is a recursion from the certain causal mechanism of the body: hands to tools, tools to models, models to data; from data to refinement of models." and this: "Reality is a place of necessity, not frequency. Our bodies place of causes not consequences. Our interaction with the world is a causal knowledge."

may i enquire on the sources for the above ? spinoza ?

let me know if you have any longer writeups on it, and if not i would urge you to write.

a starter https://chat.openai.com/share/f9e6ad4a-5566-4a42-b1f7-e9ff11...

thanks ! that the world could be modal is a revelation to me, as i viewed it as deterministic unfolding interrelated events - the links, and material are nice for me to sit back and reflect.

using chatgpt as the philosopher's assistant is a nice touch.

can i ask a follow up question: what sort of ethics derive naturally out of the above ?

I don't think modality and determinism are in tension -- this is a misunderstanding behind the free will debate.

Imv, it is literally true that "you could have done otherwise" without there being a violation of determinism.

How? Determinism, in this narrow sense at least, is about how events in the physical world relate across time, ie., that necessarily P(later|earlier) = 1. These events are to be thought of as infinitely precise states of the maximally basic stuff of reality, ie., all the info that possibly exists.

But these arent mechanisms, these are states. As soon as you describe relationships between states, ie., causal mechanisms, you're talking about what would happen if the universe were in some state that it may never enter.

I take it to be a basic property of reality that these mechanisms are (at least as) basic as these states. Eg., that the motion "of the most basic stuff" is as fundamental as "where that stuff is".

So, eg., suppose there's a basic atom A and basic atom B and they move this way: A repels A, A attracts B, B repels AB, B attracts BA, etc.....

Now this behaviour is a basic part of their existence: were there to be a universe of AAABBB, then "this would happen"; if BBBBBAAA then "something else would happen".

The "initial conditions" of the universe, ie., it's state prevents its mechanisms from ever entering certain states. But those states are possible given those mechanisms. It's in part what "mechanism" means that it is possible to enter more states than just which ones happen to occur.

So, on free will, what does it mean to say "I could have done otherwise" -- it means that the relevant causal mechanisms make genuinely possible many states. (But the actual initial conditions precluded observing more than one).

Or as a layman would put it: you would have been kind were you a different person; so, your cutely was determined by the kind of person you are. It is because of who you are (state) that what you did (free causal mechanism) was cruel (particular state obtained by operation of causal mechanism).

This may make it clear what people mean when they say, "well your brother wasn't cruel!" as if that mattered. Well: it does matter! It shows that the causal mechanisms we call, "people acting in the world" are so wide open (free) that kindness is possible.

Thus we do have free will. We are free insofar as we are in motion: our possible behaviour is much greater than our actuality. And we are determined: we, by bad luck, arent in a world were our better behaviours are realised.

It is a fundamental property of those particles above (A, B) that were they alone they wouldnt move. You cannot eliminate that property for talk of what they happened to do in the actual world. In the actual world they are really freer than can ever be observed directly.

But it is trivial to observe this indirectly: we can pull A far way from B and see what happens (etc.).

Likewise when people say "You (qua causal agent) could have done otherwise, and you didn't, so you're guilty!" there is no error here at all.

You were guilty precisely because your actions were not accidental, were not indeterminate or random. Your actions were determined by your state. And we judge that state to be one of guilt for a crime: possessing some intent and means to kill, say.

It is in this way determinism is required for free will; and required for a modal universe. If the motions of particles were indeterminate they wouldnt be causal.

thanks for taking the time to elaborate. I have to admit, i dont understand it intuitively, but it is food for reflection.

Reply on news.ycombinator.com