900 karma · joined September 1, 2025
Ironically, this post could've benefitted from a good "humanizer" itself.
It doesn't, there are many studies on the "placebo effect" (see for example https://www.nejm.org/doi/full/10.1056/NEJM200105243442106 - "We found little evidence in general that placebos had powerful clinical effects. Although placebos had no significant effects on objective or binary outcomes, they had possible small benefits in studies with continuous subjective outcomes and for the treatment of pain. Outside the setting of clinical trials, there is no justification for the use of placebos.") that reliably show that the only thing it can do is reduce the feeling of pain.
This essay is verbose AI moralityslop that doesn't back up any of the points it makes, makes no single coherent argument, and apparently tries to subtly promote quackery. Truly awful.
"With 45 percent of enterprise employees now using generative AI tools, 77 percent of these AI users have been copying and pasting data into their chatbot queries, the LayerX study says. A bit more than a fifth (22 percent) of these copy and paste operations include PII/PCI."
This is the core thesis of the essay. I have two problems with this statement. I think only a small minority of people reads books or original sources in the first place. The general audience relies on simplified and often wrong generalizations aimed at, well, the general audience. I also don't think researchers are at risk of "rewiring themselves" to rely entirely on LLMs. Science is suffering from an influx of AI slop papers, but it's not that different from all the weak or dishonest research that was being published before. Ultimately, our core academic institutions are designed around minimizing this and incentivizing high-quality intellectual output.
I think this 'artistic essence' of Factorio that makes it art and not 'just entertainment' is entirely accidental.
I like Gwern's writing otherwise, but I think that this essay is titanically wrong-headed and unconvincing. I think that Gwern takes the idea of 'art' too much for granted and tries to figure out a way to jam video games into his idea of what art is because games are 'obviously art'.
Unfortunately, it is generally up to the local developers to provide solutions, and they are often not up to the task. For example, Affinity Designer had poor RTL support for the longest time, due to certain assumptions built into their text rendering engine from the start. But making an equally featureful alternative with better support for these scripts would be a monumental task.
> More importantly, your framework cannot account for moral progress!
I don’t think “moral progress” (or any other kind of “progress”, e.g. “technological progress”) is a meaningful category that needs to be “accounted for”.
> Why does "hunting babies" feel similar to "torturing prisoners" but different from "eating chicken"?
I can see “hunting babies” being more acceptable to “torturing prisoners” to many people. Many people don’t consider babies on par with grown-up humans due to their limited neurological development and consciousness. Vice versa, many people find the idea of eating chicken abhorrent and would say that a society of meat-eaters is worse than a thousand Nazi Germanies. This is not a strawman I came up with, I’ve interacted with people who hold this exact opinion, and I think from their perspective it is justified.
> [Without a moral framework you have] no way to reason about novel cases
You can easily reason about novel cases without a moral framework. It just won’t be moral reasoning (which wouldn’t add anything in itself). Is stabbing a robot to death okay? We can think about in terms of how I feel about it. It’s kinda human-shaped, so I’d probably feel a bit weird about it. How would others react to me stabbing it this way? They’d probably feel similarly. Plus, it’s expensive electronics, people don’t like wastefulness. Would it be legal? Probably.
This is factually correct though. However, we have other reasons for positing germ theory. Aside from the fact that it provides a mechanism of action for hand-washing, we have significant evidence that germs do exist and that they do cause disease. However, this doesn’t apply to any moral theory. While germ theory provides us with additional information about why washing hands is good, moral theory fails to provide any kind of e.g. mechanism of action or other knowledge that we wouldn't be able to derive about the statement “hunting babies for sport is bad” without it.
> The fact that you will be punished for murdering babies is BECAUSE it is morally bad, not the other way around! We didn't write down the laws for fun, we wrote the laws to match our moral systems! Or do you believe that we design our moral systems based on our laws of punishment? That is... quite a claim.
You will be punished for murdering babies because it is illegal. That’s just an objective fact about the society that we live in. However, if we are out of reach of the law for whatever reason, people might try to punish us for hunting babies because they were culturally brought up to experience a strong disgust reaction to this activity, as well as because murdering babies marks us as a potentially dangerous individual (in several ways: murdering babies is bad enough, but we are also presumably going against social norms and expectations).
Notably, there were many times in history when baby murder was completely socially acceptable. Child sacrifice is the single most widespread form of human sacrifice in history, and archaeological evidence for it can be found all over the globe. Some scholars interpret some of these instances as simple burials, but there are many cases where sacrifice is the most plausible interpretation. If these people had access to this universal moral axiom that killing babies is bad, why didn’t they derive laws or customs from it that would stop them from sacrificing babies?
Agents were a thing in AI research decades ago. See for example the volume "Designing Autonomous Agents" from 1990 or the mountain of works on agent-based modeling. The phrase "multi-agent systems" goes back to the 1990s or earlier.