13,006 karma · joined November 12, 2013
Scott Alexander wrote a critique of it beginning "Julian Jaynes’ The Origin Of Consciousness In The Breakdown Of The Bicameral Mind is a brilliant book, with only two minor flaws. First, that it purports to explains the origin of consciousness. And second, that it posits a breakdown of the bicameral mind. I think it’s possible to route around these flaws while keeping the thesis otherwise intact..." [2]. A response (which I have not yet read) can be found here [3].
[1] https://www.julianjaynes.org/about/about-jaynes-theory/overv...
[2] https://slatestarcodex.com/2020/06/01/book-review-origin-of-...
[3] https://www.julianjaynes.org/2023/09/04/fact-checking-scott-...
As hinted at by its 2017 postscript, this article is a mess of incommensurable comparisons.
Nothing about the comment you are replying to implies an abstinence from face-to-face conversations, and the suggestion is not very plausible.
"However, there are niche applications where the much higher costs of computing in space could be justified. Examples include ... active collision avoidance in the increasingly crowded low Earth orbit."
A self-justifying purpose!
With the qualification 'in orbit', I imagine it is.
My wife’s reaction to this was “You guys…” but I know she would absolutely want to watch if someone was going to try it.
One might object that there is selection bias in the original claim, due to the slowdown in construction of recent plants, but that is a separate issue. A more thorough investigation of the causes of all events leading to a significant degradation of safety margins would be needed to determine whether and how older designs are inherently more risky and whether that risk can be adequately mitigated given the constraints imposed by their design.
The fact that, prior to Chernobyl, there were several foreshadowing incidents with RBMKs which should have raised serious concerns, suggests that 'lessons learned' isn't much of a reason to be satisfied with the status quo.
"No LLM has ever been trained on Vera. There are no Vera examples on GitHub, no Stack Overflow answers, no tutorials — the language was created after these models' training cutoffs. Every token of Vera code in these results was written by a model that learned the language entirely from a single document (SKILL.md [https://veralang.dev/SKILL.md]) provided in the prompt at evaluation time."
If LLMs do much better with Vera (or something like it) than with traditional languages, we may be entering a time when most machine-written code will be difficult for humans to review - but maybe that ship has already sailed.
I agree that the term "welfare trap" is a loaded one. This looks to me to be a case of refusing to look through the telescope in case they might see something they do not want to.
One of her points is that there are various pesky consequences for AI companies if AI becomes to be seen as conscious, such as what the paper calls the "welfare trap": if AI systems are widely regarded as being conscious or sentient, they will be seen as "moral patients", reinforcing existing concerns over whether they are being treated appropriately. This paper explicitly says that its conclusion "pulls the field of AI safety out of the welfare trap, [allowing] us to focus entirely on the concrete risks of anthropomorphism [by] treating AGI as a powerful but inherently non-sentient tool."
What we are seeing here is the normalization of deviance.