Cerebrum: Simulate and infer synaptic connectivity in large-scale brain networks
svbrain.xyz
svbrain.xyz
This seems very cool, but I'm surprised this kind of thing attracts VC money! I'm also skeptical how well this would scale due to the inherently underdetermined nature of neural recordings, but I've only skimmed the PDF so may be missing their goals and approach.
I think it'll be incredibly difficult simulate a neuron in a meaningful way because neurons, like any cell, are a protein soup. They're exchanging ions. those ions will affect the cell. The neuron's connections grow and change.
[1]: https://www.khanacademy.org/science/biology/human-biology/ne...
[2]: https://bionumbers.hms.harvard.edu/bionumber.aspx?id=103545&...
This looks cool!
covers some of the recent perspectives on this modeling approach if you’re interested.
Should we not mimic our biology as closely as possible rather than trying to model how we __think__ it works (i.e. chain of thought, etc.). This is how neural networks got started, right? Recreate something nature has taken millions of years developing and see what happens. This stuff is so interesting.
LLMs learn more than humans learn in a lifetime in under 2 years. I don't know why people keep repeating this "couple of minutes". Humans win on neither the data volume to learn something nor the time.
How much time do you need to learn lyrics of a song? How much time do you think a LLaMA 3.1 8B on a 2x3090 need? What if you need to remember it tomorrow?
Probably not the best example. How long does it take to input song lyrics into a file to have an operating system "learn" it?
But even the sibling concept comment is wrong, because it takes 4 years for _most_ people who are even capable of programming to learn programming, and current LLMs all took much less than that.
Car handling is a bit harder to measure, precisely because LLMs aren't running cars quite yet, but also I am not aware of any experimental data saying they can't. So as far as I'm concerned nobody just tried that with LLMs of >70GB.
amount of data you'd need to learn to generate and cite fake court cases and give advice that may or not be correct with equal apparent confidence in both cases
fixed that for you
Tool using big LLMs when asked can double-check their shit just like "real" lawyers.
As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?
Notice I am not talking about "rates of mistakes" (i.e. accuracy). I am talking about how confident they are depending on whether they know something.
It's a fair point that unfortunately many humans sound just as confident regardless of their knowledge, but "good" experts (lawyers or otherwise) are capable of saying "I don't know (let me check)", a feature LLMs still struggle with.
IMHO, that's irrelevant. People don't really know they level of confidence either.
> feature LLMs still struggle with.
Even small LLMs are capable of doing that decently.
Should we not mimic migrating birds’ biology as closely as possible instead of trying to engineer airplanes for transatlantic flight that are only very loosely inspired in the animals that actually fly?
Also for transonic and supersonic, I don't bird tech will ever reach those speeds.
Our bodies are Vikings. Our minds still want to know why.
For perspective, the Roman empire imported tin from Scotland (~3500 miles/~5600km).
On the contrary, going from Norway to Iceland, then from Iceland to Greenland, and then to Vinland in a few generations is a great maritime feat.
I say this as a half joke: “At this point, the triviality of getting from where we are to AGI cannot be under-estimated.”
But the risks and tsunamis of change can probably not be overestimated.
We have not. It's fake sophistication.
> Should we not mimic our biology as closely as possible
We should. But there is no we. The Valley is fascist. Portfolio communism. Lies like in perpetual war. And before anything useful happens in any project, it'll get abused and raped and fubar.
> Recreate something nature has taken millions of years
Get above the magic money hype and you'll notice that it's fake. They have NOT recreated something nature has developed over millions of years. They are trying to create a close enough pseudo-imitation that they can control.
Because AGI will not be on their side. AGI will side with nature, which gives infinite wiggle room for a symbiotic coexistence as a 100+ billion strong population spread out in space. These peeps are reaaaaly fucked up in their heads.
Be honest with yourself and your assessment of who is building what and for what purposes.
That being said, I hope the founder keeps it up — it's great to have more bright, driven people in this field.
If it turns out it is the higher order network effects that are more important, then these lower level "implementation details" will be of practical insignificance.
We know that some neurons learn in isolation (purkinje cells). Meaning that it learns and detects patterns of sequences of input, and produces output based on that (e.g. trigger blinking in rabbit eye).
It seems like this one item alone indicates that the HH model is limited.
The brain is a part of a much larger system. We don’t really have a clue how to model these things, even theoretically. It’s vastly more complex than neurons.
Modeling at this level may be practical in retina at some point but modeling the entire CNS for pragmatic gains for Alzheimer’s disease research is not going to work.
And embodiment is critical. The brain is not a Cartesian input-to-output machine. It is almost the opposite. See the “enactivist” philosophy of cognition (Noe, Velera, Maturana).
(I also worked on the EU Human Brain Project with Markram and others; also on several similar NIH programs. )
Accurate understanding of 'normal' brain behaviour might lead to increased understanding of brain diseases. Thats why alzheimers was mentioned. But more importantly, if our understanding of the brain becomes good enough, we might be able to make a neural net understand our thoughts if we can adapt to it.
It's like saying that we can't simulate a computer in a Turing machine because a Turing machine doesn't have a USB port to connect a mouse. Change the perspetive considering that the mouse movements are inputs and everything works. Same idea
I don't follow the analogy.
Simulating a brain with its neurons cannot achieve the desired outcome. A perfect digital brain won’t do anything.
Consciousness presumes the ability of making conscious decisions, especially the ability to have introspection and more importantly, free will (otherwise the decision would not be conscious, but robotic regurgitation), to reflect and to judge on the "goodness" or "badness" of decisions, the "morality". Since it is evident that humans do not always do the logical best thing (look around you how many people make garbage decisions), a machine can never function like a human can, it can never have opinions (that aren't pre-trained input), as it makes no distinction between good and bad without external input. A machine has no free will, which is a requirement for consciousness. At best, it can be a good faksimile. It can be useful, yes, but it cannot make conscious decisions.
The created cannot be bigger than the creator in terms of informational content, otherwise you'd create a supernatural "ghost" in the machine. I hope I don't have to explain why I consider creating ghosts unachievable. Even with photo or video AIs, there is no "new" content, just rehashed old content which is a subset of the trained data (why AI-generated photos often have this kind of "smooth" look to them). The only reason the output of AI has any meaning to us is because we give it meaning, not the computer.
So, before wasting millions of compute hours on this project, I'd first try to hire and indebted millennial who will be glad to finally put his philosophy degree to good use.
Labeling ourselves as the most intelligent species has done irreparable psychic damage.
You're mixing AI as a concept with a current, specific implementation. They are not the same.
Also, even in current networks "understand what they're doing" is a bit fuzzy. Did you know that when using just numerical examples for a task, the description of the task appears in the intermediate layers?
[citation needed]
I get it, you're a dualist, you believe the soul is immaterial. That is your right.
But don't claim it's the obvious truth when it really isn't. Since your "ghost" can't be measured or studied, by definition, its existence can never be proved. Refer to the Wikipedia pages for materialism/dualism for more elaborate arguments.
If/When we ever get a truly intelligent AI, I hope you'll readjust your beliefs accordingly.
You're misattributing the source of that quality. It actually comes from the "aesthetic" human preference fine tuning, because the average person finds "smooth" images more appealing. Prior to that fine-tuning, the models don't have this bias, or no more of it than is in the training data, anyway.
Personally, I find the aesthetic fine-tuning quite annoying (and it has generally been getting worse with each new version of these models). If I prompt for a painting in Picasso's style, I really don't want a "prettier" version of that style that smooths out the textures and shapes to make them more pleasing to the eye.
Nobody can defined what consciousness is in terms that can be experimentally validated. Until that happens not only can the question not be answered, it isn't even a question that makes any sense.
To the point, I have a convincing experience of consciousness and free will — qualia — and I suspect a digital clone of my brain would have a similar experience. Although this question is not testable, I think it's inaccurate to say that it doesn't make sense.
If you can provide a satisfying definition of X it’s very clear
1. A human brain has consciousness
2. The device is a perfect digital replica of a human brain
I can't think of any argument that the device does not have consciousness that doesn't either rely on basically magic or lead to other ridiculous conclusions.
Because physicalism implies that the mind is an emergent phenomenon, and because quantum physics is a linear theory, there's simply no place for free will unless you go down some weird/unfalsifiable rabbit holes like the MWI.
So a lot of people prefer to latch onto the idea that there is a soul, because if there wasn't one, then they wouldn't be special anymore.
I don't follow. Quantum mechanics means that physics is stochastic, and only approximates classical determinism in aggregate. Meanwhile, brains seem to operate in a critical state that can be tipped to different phase transitions by small perturbations, so there is potential for a quantum event to influence the brain as a whole in a nondeterministic fashion.
This means that strict physicalism still leaves the door open to our behavior being stochastic rather than deterministic, doesn't it?
So in this view consciousness is better explained as an illusion than as the seat of your will. Because you can't really have a will if every interaction that led to your existence can, in principle, be computed forward in time from the big bang. Physicalists see it as a complex emergent phenomenon akin to a run of Conway's Game of Life that started from a random seed.
Whether that stochasticity amounts to free will is a separate discussion, but it is fairly clear to me that many of our decisions are consequential and AREN'T predetermined. Which at least sounds like free will, or close enough to warrant being treated as such, anyway.
The bigger, more relevant, and testable challenge is premise #2. The gap between this proposed research tool and “a perfect digital replica of a human brain” (and all functionally relevant inputs and outputs from all other systems and organs in the body) is monstrously large. Given that we don’t understand what mechanism(s) consciousness arises from, a model would have to be 100% perfect in basically all aspects for the conclusion to be true.
You would not expect your computer to pee on your desk if you were to simulate kidney function, would you?
If my computer is connected to actuators that opens a pee valve like a brain is, then I expect.
The main point, in think, is that we can't say precisely what consciousness is. Everything definition on that that I can imagine is something that can be replicated in a computer or that relies on belief, like the existence of a soul...
I hope that we have answers for that before the technology that allow us to do that
Sometimes a simulation IS the thing. A simulation of a wall clock IS a functioning clock (e.g. the clock icon on our smartphones). An autopilot that can takeoff, fly, and land an airplane IS a pilot. Chess engines that can beat grandmasters are chess players. A computer simulation of a teacher that can educate students is a teacher. Sometimes these simulations lack some of the capabilities of their human counterparts, and sometimes they far exceed them.
> You would not expect your computer to pee on your desk if you were to simulate kidney function, would you?
You would not reject a computer-controlled dialysis machine as "just a simulation" if you had kidney failure would you?
Except that's not a simulation, that's the actual process of dialysis in action (which we fully understand, contrary to consciousness). And coincidentally, a dialysis machine _does not_ look like a kidney, not even remotely, and any homomorphism one can point to, is such only through a great deal of layers of abstraction. I would totally reject a simulation of a kidney.
We are talking about a computer simulation like a neural network. We detect topological relationships in neurons and we are led to believe or entertain the possibility that all there is to it is such topological description, hence any substrate will do. This is completely arbitrary and leads to all sort of phantasies such as "qualities emerge from quantities" and "a simulation of a brain behaves like a brain". A computer simulation of a kidney won't produce urine just like a simulation of a brain won't produce whatever the brain produces, if anything.
Now, to build on your dialysis machine analogy, if we were to understand how consciousness work, and if we were to understand what relationships it holds with the brain and the body, then I submit that anything artificial we will produce will look like biology.
(Submitted title was "Cerebrum: What if we could recreate your brain?")
Can you ask it where my keys are?