995 karma · joined January 27, 2022
Given that this is pure mathematics we're talking about, for the most part: Why?
Perhaps you are a mathematician and so these questions are relevant to you, but for most of us, these questions and their answers will not make any tangible difference in our lives or society.
And unlike those in Jamestown, who could just ignore the robots and continue working the land to survive, we will be forced to allow the robots to take over our roles.
If you actually want to shift your circadian rhythm, the optimal dose is about 300 to 500 mcg (the actual absorption rate varies wildly between manufacturers and delivery methods), taken 4 to 6 hours before your sleep onset (note: when you usually go to sleep, not when you want to go to sleep). [2]
As far as I understand, you downloaded the structure of a neural network, ran it with essentially arbitrary weights, trained a classifier on the essentially arbitrary behavior, then simulated an approximation of that arbitrary behavior.
You could argue that there might be biases towards certain behaviors encoded in the connectivity, and I'm sure you'd be right, but your experiment is incapable of differentiating between those interesting behaviors and random noise. Especially because flies sorta act random anyway.
If you're trying to argue that the chances of a non-human source of greater than human intelligence having different goals than us is infinitesimally small, then I'd be interested in what arguments you have to back that up.
You have to be a minimum amount of likeable in the first place, though.
They've also hacked third party machines and used them to launch attacks on further services.
For the most part, Modelese is the revealed preference of the average, not-heavily-invested human. This doesn't fully explain the Claudish coming from Opus 5 and Fable (this seems like it may be due to excessive RLVR or RLAI), but yeah.
It's got a lot of cheap writing tricks that make people think it's smart and helpful.
I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.
Until we all know what consciousness is, this debate is going to continue going in circles.
Unless you're suggesting military action
There are hundreds of incredibly strong scientific priors that would have to be disproven for the moon to contribute to the solution.
If a model was trained on this data, even if it was trained using methods that lead you to believe it unlikely to have learned details about the proof (e.g., maybe it was only used to train some kind of reward model, which played a minor role in the overall training and would thus be very unlikely to transfer details of a proof), you wouldn't have to disprove large swathes of known science to be wrong.
… there are plenty of scientific experiments that show actual differences between people who report aphantasia and those who don't, including different stress responses to frightening non-visual descriptions, different susceptibility to something called image priming, lower "cortical excitability in the primary visual cortex", and more: https://en.wikipedia.org/wiki/Aphantasia
So we know that at least the people who claim to see nothing act differently. Could it just be that people who act differently describe the sensation differently, you might ask?
No, because there are actual cases of acquired aphantasia after neurological damage. These people used to belong to the group that claimed to be able to imagine visual images, got sick, then sought medical help when they could no longer visualize. For me, at least, that's pretty cut and dry evidence that it's not just differing descriptions of the same (or similar) sensations.
If you're a junior engineer or an intern doing this, I think it might be harmful long-term.
If you're using AI to learn a new field (by which I assume you mean asking it what literature to read, asking questions when you don't understand something, etc.), you are accepting short-term speed in exchange for the long-term growth of research skills. Maybe that trade-off is worth it, if you don't need to master a topic long-term.
If you do need to master a topic, the slow and painful bits are the useful bits. Spending two hours hunting for an answer to something you don't understand increases your familiarity with the available resources, broadens your knowledge base, leads you to thinking of more questions about the subject and to understand your original question more deeply.
Shortcutting all of that with a quick answer from an LLM is not helpful long-term.
This is basic alignment, not even a tricky or ambiguous case.
I do very much agree with your take on culpability/military parallels, though.
That seems to be the crux here. You think it will be, I (and a lot of other people) aren't sure it will. If new space for jobs are created, I am certain we'll be fine long term.
What do you think will happen if it isn't (I know you have strong evidence for and reasons to believe it will be, but just indulge the hypothetical)?
I'm sure they could have done more with less fanfare, but you seem to be setting the bar at "fixing the world's problems", which seems unfair. They have undoubtedly changed more lives than I'll ever manage to.