D-Machine

All these policies actually sound good, SSRIs should not be first-line, as there is extremely limited evidence of meaningful effectiveness for anti-depressants generally.

Mmeta-analytic assessments of effect sizes show the observed effects are usually below what is required to count as a minimal clinically important difference, and antidepressants only target generally the "positive" symptoms of depression (sadness, anxiety) while doing little for the "negative" ones (anhedonia, lack of meaning, lack of motivation). They also have some serious side-effects, like emotional blunting and severe sexual dysfunction, the latter of which can persist for months even after stopping to take the drugs.

Even the usual excuses people make for these drugs ("Sure, the average effect is small, but they really help some people" and "Eventually the doctor finds the right one for you") are generally not supported by actual studies addressing these claims either.

There absolutely should be more monitoring and less careless prescribing of SSRIs and anti-depressants generally.

Ironic because it is the people who push for just passing everyone that are actually doing it to just "make the problem go away", in reality.

There's no hypothetical here IMO, this is a real-world problem, and also you aren't limited to pass/fail decisions as a teacher, except in exceptional cases of borderline grades. Otherwise, there is a passing grade / requirements, and grades can be determined by objective tests (all students get the same difficulty tests). Also you have something like an average of many courses over many years to make the pass/fail decisions, ultimately, if we are talking about getting a diploma and/or graduating high-school here.

Also, it depends what you see the discussion as. If laws are supposed to do the right thing, then "pass everyone always" is really starting to look like the wrong thing, even where the intentions are seemingly "good" because they avoid "disparate impact" (in the short term on very narrowly-chosen metrics). Then if your argument is "yeah well we can't do the right thing because lawsuits", well, yes, I agree, practically, but then these lawsuits are basically also evil and/or misguided.

I'd prefer more proof / context than GP gave, but I personally find it very useful to see people making judgements about AI-assistance of articles. Almost no such articles are worth my time, and the more HN people saying it, the more I know not to click past the HN headline.

> Could you prove the disparate impact is random and your pass criteria isn’t racist?

Can those in favor of grade inflation and meaningless credentials prove their decisions also don't have disparate impact and aren't racist? Based on some recent US Supreme Court decisions re: affirmative action, it would seem unlikely this case would be any different. The hard questions about long-term harms to students and society are simply not being asked seriously enough.

Requiring years of schooling that is essentially worthless / provides credentials with no information also has disparate impacts, possibly worse than just properly failing people and letting them sort themselves / be sorted into positions that are actually suitable for them and allow real growth. Schooling is a huge percentage of a modern person's life now.

Yes you are citing my link... which is pure speculation. Do you have anything to actually add?

I asked ChatGPT a simple baking question today (https://chatgpt.com/share/6a1a1445-cc10-83ea-bf1f-957c07ce7e...), and got this rather strange preamble before my response:

    "Could hidden user memory materially change what I should recommend? **No.**"

A typical answer then followed. Users on Reddit report the behaviour as well (https://old.reddit.com/r/ChatGPT/comments/1traouv/why_is_it_...), and the linked article suggests this is new.

The thing is, I have memory disabled. So is this an internal prompt just being exposed ("Does the user have any memories that are relevant? No, [because they don't have any].") or does it mean they are in fact keeping hidden memories / context and using these to inform responses, but didn't find anything relevant for this particular question?

While I suspect the former, the possibility of the latter concerns me somewhat.

Yes, sorry about the edits.

IMO the correct framing most supported by the evidence is that a few vulnerable populations (most likely within a certain age bracket, and already dealing with other significant life issues) are definitely at risk of nontrivial harm from social media. Otherwise, for the majority, it is likely a wash. And since the bias in studies is to look for negatives (much like drug studies don't properly measure negative side effects until much later), we also can't say if there aren't subgroups benefiting hugely as well (e.g. autistic people that can connect online, or other people with unusual interests that before would have remained isolated and disconnected, feel crazy, etc).

So, yes, small effects can be important. The observed effects for social media are quite small and consistent with being driven solely by a small, vulnerable population, however, and this make broad social injunctions and moral panic less defensible.

Also, these are effects on self-report or other psychological instruments, where in general you need to find something like a minimal clinically important difference before you can determine if you care practically about these things. E.g. if a standardized regression coefficient of 0.1 translates to a 1-point shift on a 20-point life-satisfaction scale, is this even something I (or anyone) can even notice? Does it even rise above measurement error?

Since no one is doing that kind of research here, we just get these (relatively meaningless) standardized effect sizes, and it is really basically impossible to know if we should care at all about the observed effects. This is mainly what I mean by the evidence being flimsy / weak, it is really too inconclusive to drive decisions at this point.

> That sounds like we'll devolve into wars over resources

Two astronauts meme: "Wait, it's all been over resources?" "Always has been"

You're gish-galloping a bunch of random studies, most of which appear to be cross-sectional. I've linked a broad review.

Even the longitudinal studies are poor here. See, for example, as this Nature article notes:

    "The study has multiple limitations that need to be considered. First, to interpret the parameters from our analyses as estimates of causal effects one would need to adopt the following assumptions: (a) there are no time-varying unobserved confounders that impact the relation between social media use and life satisfaction; (b) the model adequately accounts for unobserved time-invariant confounding through the inclusion of a random intercept; (c) there is no measurement error in the variables; (d) the time interval between studies (one year) is the right length to capture the effects of interest; and (e) the bidirectional links estimated by our longitudinal model are linear in nature. Only if these assumptions are met can this observational study be said to capture the causal effects between social media and life satisfaction. Second, the data are self-report and therefore only allow inferences about the impact of self-estimated time on social media, rather than objectively measured social media use." https://www.nature.com/articles/s41467-022-29296-3#Sec2

I'd also suggest looking at the coefficients (effect sizes) in the above (standardized regression coefficients barely approaching 0.2 - and this is one of the stronger findings), and other articles. The effects here, even if we were to pretend they were clearly established, are incredibly tiny. Examples:

- social media explaining only 0.4% of variance (https://pubmed.ncbi.nlm.nih.gov/30944443/)

- social-media/mental-health effect around β = .061 (https://christopherjferguson.com/Social%20Media%20Meta.pdf)

These are basically nothing, and yet you have such absolute confidence from people that social media is this big harmful thing. The evidence just isn't there.

No, the evidence is quite flimsy and correlational, and this is clearly a moral panic.

EDIT: See e.g. https://www.nationalacademies.org/read/27396/chapter/6#93, and if you think this is what constitutes "clear evidence", well, you have some very questionable epistemological standards.

EDIT2: Also, limit yourself to proper longitudinal studies and then look at the actual effect sizes reported. You will find, yes, there is broad evidence that social media is likely slightly more harmful to adolescents than beneficial / not harmful, but the actual effect size is so tiny broad interventions are unlikely to have practical consequences. I.e. the most plausible explanation is that the vast majority people are not meaningfully affected, and small subgroups benefit and/or are negatively affected.

It is the usual pseudoscientific / social science attempts to launder "statistical significance" (which you get trivially with enough samples) into practical significance, in order to justify sweeping societal changes.

But with "consciousness" we don't have even a stipulative definition, more like 40 different competing definitions (https://philpapers.org/rec/VIMMAT) which are in many cases only loosely related, and generally are talking about very different things. When the situation is this bad, I am skeptical you can make scientific progress.

Except you can't be sure it isn't producing nonsense when you do this, and generally the model(s) will be overconfident. This has been studied, see e.g. https://openreview.net/pdf?id=E6LOh5vz5x

    > An alternative way to obtain uncertainty estimates from LLMs is to prompt them directly. One benefit of this approach is that it requires no access to the internals of the model. However, this approach has produced mixed results: LLMs can sometimes verbalize calibrated confidence levels (Lin et al., 2022a; Tian et al., 2023), but can also be highly overconfident (Xiong et al., 2024). Interestingly, Xiong et al. (2024) found that LLMs typically state confidence values in the range of 80-100%, usually in multiples of 5, potentially in imitation of how humans discuss confidence levels. Nevertheless, prompting strategies remain an important tool for uncertainty quantification, along with measures based on the internal state (such as MSP).

IMO it is you who are moving the goalposts, most likely in an attempt to hide the fact you were unaware of calibration before this discussion.

> It would still be incredibly useful to have a machine that can merely matches the equivalent statement of confidence or uncertainty that a human would assign to their mental model, even if badly calibrated.

If human feelings are badly calibrated, they are useless here too, so no, I don't agree. Things like "confidence" only matter if they are actually tied to real outcomes in a consistent way, and that means calibration.

> But If we artificially constraint outputs to "Yes" and "No", and calculate the softmax for Yes to be 0.7 and No to be 0.3, that does lead to a straightforward probability calculation. [Not the naïve calculation you would expect, because of how softmax is computed. But you can derive an equation to convert it into normalized probabilities.]

There is nothing straightforward about this, and no, there is no such formula.

> I'm not talking about calibrated probabilities at all. Just the notion of "how confident do I feel about this?"

If all you care about is vibes / feels, sure. If you actually need numerical guarantees and quantitative estimates to make your "feelings" about confidence mean something to rigorously justify decisions, you need calibration. If you aren't talking about calibration in these discussions, you are missing probably the most core technical concept that addresses these issues seriously.

> The question is about relative uncertainty, and the softmax values are just fine for that.

They really aren't, especially if you consider the chain of thought / recursive application case, and also that you can't even assume e.g. a difference of 0.1 in softmax values means the same relative difference from input to input, or that e.g. an 0.9 is always "extremely confident", and etc. You really have no idea unless you are testing the calibration explicitly on calibration data.

> But there are alignment tools to extract out these latent-space probabilities for researchers in the frontier labs

You can get embeddings: if you can get calibrated probabilities, you'll need to provide a citation, because this would be a huge deal for all sorts of applications.

GP is obviously wrong, and probably doesn't know about calibration and/or that it isn't even clear how to calibrate frontier models in the manner we need, given how complex and expensive the training is, and how tricky calibration becomes in e.g. mixture-of-experts and chain of thought approaches.

Common misconception. As far we know, LLMs are not calibrated, i.e. their output "probabilities" are not in fact necessarily correlated with the actual error rates, so you can't use e.g. the softmax values to estimate confidence. It is why it is more accurate to talk about e.g. the model "logits", "softmax values", "simplex mapping", "pseudo-probabilities", or even more agnostically, just "output scores", unless you actually have strong evidence of calibration.

To get calibrated probabilities, you actually need to use calibration techniques, and it is extremely unclear if any frontier models are doing this (or even how calibration can be done effectively in fancy chain-of-thought + MoE models, and/or how to do this in RLVR and RLHF based training regimes). I suppose if you get into things like conformal prediction, you could ensure some calibration, but this is likely too computationally expensive and/or has other undesirable side-effects.

EDIT: Oh and also there are anomaly detection approaches, which attempt to identify when we are in outlier space based on various (e.g. distance) metrics based on the embeddings, but even getting actual probabilities here is tricky. This is why it is so hard to get models to say they "don't know" with any kind of statistical certainty, because that information isn't generally actually "there" in the model, in any clean sense.

Except you can reject the very (stupid) question / framing, in which case, the response is to either close the tab, or respond in a particular response style, neither of which makes the data more informative. This kind of clumsy stuff is just dumb with what we know now, edutainment distraction for the HN crowd.

Yes, very annoying, we know from extensive work in psychometrics that single-item, binary / forced-choice items produce junk responses that are heavily contaminated with response styles (answer in most socially-desirable way, select closest response to mouse/finger, select same response as last time, select random response, etc). Just give people an out ("Diagree with the question / premises", "Prefer not to answer", "Unsure / Can't decide", etc) and make sure you have e.g. a 5-7 point Likert-type scale for multiple items, or up to an 11-point scale for single items.

This kind of site / demo does none of the above, and so can't even be trusted for directional effects (the direction of response may simple be due to the type of people responding, etc).

Forced binary choices on single-item, self-report questions produce scientific junk, absolutely. This kind of design / approach encourages not only magnitude errors, but also sign errors (you can't even trust the direction of the observed effect).

IMO, growing up, unless you lived under a rock, it seems obvious to me that you will have experienced different people pointing at the same colour and uttering very different colour labels (pink vs. red, blue vs. green, black vs. deep blue/purple, etc) from the labels you might have applied yourself. Differing/shared colour perception isn't exactly a rare kind of topic (almost is like the canonical stoner topic, also common online), so I'd be a bit surprised if this demo is actually introducing anyone to this concept already. Any excitement is surely from other implications people think the demo has.

But unfortunately there are no interesting implications from what this site shows. Yes, it demonstrates the boring fact that: "it isn't clear how different people assign different color labels to the same physical stimuli" (and yes, this is FALSELY assuming that everyone's monitors/screens are the same too), but if you didn't already know this... I'm not sure exactly what social context you could have possibly grown up in.

This is the wrong way to do it, psychometrically, see here: https://news.ycombinator.com/item?id=47929056. You need to provide people gradations, or you get junk responses / abandonment, and your instrument doesn't measure what you think.

Wrong way to do it. We know from psychometrics that forced binaries like this just create junk (people disagree with the question, so just choose a forced answer based on some heuristic for each such question like "closest to my mouse / finger" or "most socially desirable" or "same as last time"). So you aren't measuring what you think when you force choice like this.

If you're going to go with linguistic self-report and a single item, you really want something like an 11-point Likert scale. A smart design might get e.g. a person's rating of "blue-ness vs. green-ness" on an 11-point scale, then determine the optimal cutpoint via e.g. clustering, logistic regression, or some other method, to really get something meaningful.

Sapir-Worf and its ilk (if we don't have the language/concept, we can't perceive the difference/thing) are widely disproven and debunked, and don't even pass the smell test (learning new concepts and perceiving new things would be impossible). That kind of thinking is so tedious and decades out of date with modern cognitive science, neuroscience, psychology, etc.

Agreed, there is no clear premise. Of course that different people looking at the same object will use different colour words is a triviality that anyone over, say, 10 years old knows. If that's the premise of the site, it is boring. People are getting excited because they think this implies something about differences in vision or perception... but it doesn't, that requires much more cleverness to test.

But that is wrong. This doesn't test colour perception or vision, it tests verbal classification of colour perception into a forced binary. Everyone could be perceiving the colour qualia 100% identically, but simply choosing different linguistic cutpoints, meaning you can't say this is about vision / perception at all (it may just be about language use).

Asinine and meaningless. Forces a classification on something that obviously anyone with fully-functioning colour vision will classify as "aquamarine" or "turquoise" or etc.

This has nothing at all to do with colour perception, or, if actual differences in perception are involved, this test fails to distinguish those from individual differences in assignment to linguistic categories.

EDIT: To actually test something like this, you need to make an assumption that cannot easily be tested or supported by evidence.

E.g. say we could all agree that, generally, blue + orange is a more pleasant pairing than blue + green. One might then imagine a series of images using orange + varying interpolations between blue and green, with the prompt being "is this combination of colours more or less aesthetically pleasing than the last". The average cutpoint could then be interpreted as a subjective judgement of where e.g. teals become "more blue", from an aesthetic / complementary standpoint. But this test does nothing of the sort.

This was sort of my reading as well: I took "clumping" to mean "bump-shaped".

Right, I think (hope) the OP meant not to emphasize the "search" in the sentence, but the "reputable source". Of course a Google search now is much worse than an AI search.

And it is the ultimately the reputable source that matters, and whether the person actually read it and checked that the details matched the summary (be it human abstract, LLM-generated, or otherwise).