HNHacker News
TopNewBestAskShowJobs

nicklecompte

2,008 karma · joined August 9, 2018

submissionscomments
nicklecompte··on Elicit – AI Research Assistant
"Geared for the general public" is an interesting and revealing observation.

You might not have given the "right test" in terms of the actual userbase, but it is absolutely the right test in terms of Elicit's marketing claims. Elicit might be implicitly geared for the general public, but they are explicitly marketing to scientists.

I suspect a lot of Elicit's target users want to use scientific knowledge in their personal/professional lives, but without doing the hard work of gaining scientific understanding. However, they're not going to spend money on a product that says "we use AI to create sciencey bullshit that sounds plausible in conversation." They want a product that Real Scientists would use. (Similar to how purely decorative Damascus steel Bowie knives are gussied up by an outdoorsman pretending to use the knife to gut a fish or whatever.)

nicklecompte··on Elicit – AI Research Assistant
This is true but if the error rate were 1/1000 I could see the risk management argument for using this thing. 1/100 is pushing it. 1/10 seems unconscionably reckless and lazy.
nicklecompte··on Elicit – AI Research Assistant
I am not sure what you mean by this comment. I took the language from the developers. If you mean that commercial AI providers should give more specific information then I agree wholeheartedly.

I assume it is difficult for Elicit to give specific numbers because they lack the data, and confabulations are highly dependent on what research area you are asking about. So the "rule of thumb" is a way of flattening this complexity into a usage guideline.

nicklecompte··on Elicit – AI Research Assistant
Those numbers are arbitrary and fictional, and the more relevant made-up quantity would be the variance rather than the mean. It doesn't really matter if the "average user" saves time over 10,000 queries. I am much more concerned about the numerous edge cases, especially if those cases might be "edge fields" like animal cognition (see below).

In my experience it takes quite a bit longer to falsify GPT-4's incorrect answers than it does to a Google search and get the right answer. It might take 30 seconds to check a correct answer (jump to the relevant paragraph and check), but 30 minutes to determine where an incorrect answer actually went wrong (you have to read the whole paper in close detail, and maybe even relevant citations). More specifically, it is somewhat quick to falsify something if it is directly contradicted by the text. It is much harder to falsify unsupported generalizations or summaries.

As a specific example, I recently asked GPT for information on arithmetic abilities in amphibians. It made up a study - that was easy to check - but it also made up a bunch of results without citing specific studies. That was not easy to check[1]: each paragraph of text GPT generated needed to be cross-checked with Google Scholar to try and find a relevant paper. It turned out that everything GPT said, over 1000 words of output, was contradicted by actual published research. But I had to read three papers to figure that out. I would have been much better off with Google Scholar. But I am concerned that a large minority of cynical, lazy people will say "90% is good enough, I don't want to read all these papers and nobody's gonna check the citations anyway" and further drag down the reliability of published research.

[1] This was a test of GPT. If I were actually using it for work, obviously I would have stopped at the fake citation.

nicklecompte··on Elicit – AI Research Assistant
> A good rule of thumb is to assume that around 90% of the information you see in Elicit is accurate. While we do our best to increase accuracy without skyrocketing costs, it’s very important for you to check the work in Elicit closely. We try to make this easier for you by identifying all of the sources for information generated with language models.

A 90% accuracy rate seems like the sweet spot between "an annoying waste of time" for honest researchers and "good enough to publish" for dishonest careerists.

I don't like disparaging the technology experts who work on these things. But as a business matter, 1/10 answers being wrong just is not good enough for a whole lot of people.

nicklecompte··on Jan Leike Resigns from OpenAI
I believe the issue is "emotion" and "emotional tone" are not the same thing, in the same way that "humor" and "written joke" aren't the same thing. You can convey emotional tones without having the emotion (that's what I meant by "fake emotion"), just like you can tell a joke without understanding the punchline.
nicklecompte··on Jan Leike Resigns from OpenAI
I am criticizing Sam Altman for making an unethical business decision. I didn't say "Sam Altman should go to jail because GPT-4o is creepy" or "I want to away your AI girlfriend." So I am not sure what "belief system" (ugh) you think I need to demonstrate the consistency of. Almost seems like this question is a ad hominem distraction....

All three of the categories of businesses you mentioned can be run ethically in theory. In practice that is rare: they are often run in a way that shamelessly preys on vulnerable people, and these tactics should be more closely investigated by regulators - in fact they are regulated, and AI chatbots should be as well. Sam Altman is certainly much much more ethical than most pornography executives (e.g. OnlyFans is complicit in widespread sex trafficking), but I don't think he's any better than freemium game developers.

This question seems like a bad-faith rhetorical trap, sort of like the false libertarian dilemmas elsewhere in the thread. I believe the real issue is that people want a culture where lucrative business opportunities aren't subject to ethical considerations, even by outside observers.

nicklecompte··on Jan Leike Resigns from OpenAI
> I guarantee you haven't done an in-depth study of the merits of cryogenics. Neither have I yet, but there are many smart people who do think the tradeoff is worth it.

This is literally the strategy of conspiracy theorists! "You didn't do the research, but Emmett did and he's smarter than you." The scientific consensus is that cryonics is pseudoscience. It does not matter if I have personally done a study. It is entirely rational for me to trust the Wikipedia article on cryonics + common sense. And lots of people way smarter than Shear believed in stranger things.

Emmett Shear - and yourself! - uncritically described cryonics as being a rational, techno-optimist thing to support, and proof of Shear's credentials as a reasonable, grounded thinker, not like these Tesla-smashing wackos. That strongly implies that Shear is a bad judge of what counts as reasonable, because cryonics is ridiculous quackery. Hence why he has Dunning-Kruger syndrome. He's operating in an ignorant Silicon Valley bubble where sci-fi and reality are interchangeable as long as you have a thoughtful affectation.

One thing ChatGPT has made very obvious is that the tech community is vulnerable to the tactics of psychics and conspiracy theorists.

nicklecompte··on Jan Leike Resigns from OpenAI
"People who support cryonics are sober-minded, reasonable individuals who would surely never be misled by unscientific techno-mysticism."

I am amazed at the Dunning-Kruger syndrome on display here.

nicklecompte··on Jan Leike Resigns from OpenAI
Sure, I was being sloppy, I meant "suspiciously timed announcement."
nicklecompte··on Jan Leike Resigns from OpenAI
I wasn't making an accusation about why Leike/Sutskever left, though I definitely understand why you read my comment that way.

The actual accusation I am making is that someone at OpenAI knew the risks of GPT-4o and Sam Altman didn't care. I am confident this is true even without spies in the boardroom. My guess is that Leike or Sutskever also knew the risks and actually did care, but that is idle speculation.

nicklecompte··on Jan Leike Resigns from OpenAI
"fundamentally unsafe people" is probably the grossest thing I've read on here in years.
nicklecompte··on Jan Leike Resigns from OpenAI
I have schizophrenia. I have struggled with nicotine addiction since high school. In 2015 I had three heart attacks in a month, even though I was only 28 and seemed physically fit. Two weeks ago I had a minor stroke.

It is not just me and it is not just the smoking: https://www.cambridge.org/core/blog/2020/08/19/physically-he...

  We have known for many years that people who suffer from schizophrenia die younger than expected, as much as 20 years younger than the general population. This appears unfair, and it was the inspiration of this work. Most people thought that this added risk of death was mostly due to the higher prevalence in schizophrenia of smoking, obesity and to other lifestyle differences.

  For this reason, we recruited 40 patients with schizophrenia and an equal number of healthy controls, and scanned their hearts using a state-of-the-art approach, called cardiac magnetic resonance. This was performed at the state-of-the-art Robert Steiner MRI unit.

  [...] Surprisingly, in our study we found that even after matching patients and healthy controls for age, sex, ethnicity and body mass index (BMI, deriving from height and weight); and after excluding any participants with any medical conditions, and other risk factors for heart disease, people with schizophrenia show hearts that are smaller and chunkier than controls. These changes are similar to those found in aging.
I was able to move to the gum and patch, but I am very high-functioning. People sicker than me have less options. Smoking is very bad for everyone, including people for schizophrenia. We do not in any way benefit from terrible heart/lung damage in exchange for minor cognitive clarity - our hearts need all they help they can get. I have no tolerance for this sort of ignorant paternalism, and I'm ignoring your bad-faith question about "actively target them" because that's not what I said.
nicklecompte··on Jan Leike Resigns from OpenAI
Specifically I am supposing the superalignment people were generally more concerned about AI safety and ethics than Altman/etc. I don't think this has anything to do with superalignment itself.
nicklecompte··on Jan Leike Resigns from OpenAI
https://nida.nih.gov/publications/research-reports/tobacco-n...

  A larger proportion of people diagnosed with mental disorders report cigarette smoking compared with people without mental disorders. Among US adults in 2019, the percentage who reported past-month cigarette smoking was 1.8 times higher for those with any past-year mental illness than those without (28.2% vs. 15.8%). Smoking rates are particularly high among people with serious mental illness (those who demonstrate greater functional impairment). While estimates vary, as many as 70-85% of people with schizophrenia and as many as 50-70% of people with bipolar disorder smoke.
I am accusing OpenAI (and Phillip Morris) of knowingly profiting off mental illness by providing unhealthy solutions to loneliness, stress, etc.
nicklecompte··on Jan Leike Resigns from OpenAI
It is easy to point to loopy theories around superalignment, p(doom), etc. But you don't have to be hopped up on sci-fi to oppose something like GPT-4o. Low-latency response time is fine. The faking of emotions and overt references to Her (along with the suspiciously-timed relaxation of pornographic generations) are not fine. I suspect Altman/Brockman/Murati intended for this thing to be dangerous for mentally unwell users, using the exact same logic as tobacco companies.
nicklecompte··on Current AI models are more creative than humans on divergent thinking tasks
This research strikes me as hot garbage :/ In particular there are dozens of studies like this, which compare to AI with "humans" without adequately addressing sample bias:

  Human participants (N = 151) were recruited via Prolific online data collection platform in exchange for monetary compensation of $8.00. Participants were limited to having a reported approval rating above 97%, were proficient English speakers, and were born/resided in the USA. Average total response time for completing the survey was 34.66 minutes.
Asking a bunch of random people to fill out tedious "creative" writing prompts for $15/hr is not a good way to measure humans. Baffling that the discussion didn't even mention this. But you see this all the time in AI research, and I am sure this headline will be repeated uncritically by many people.
nicklecompte··on Intel announces the Aurora supercomputer has broken the exascale barrier
"Computational fluid dynamics is hard" seems like a true bona fide CS invariant.
nicklecompte··on GPT-4o
This is pure astrology, but given the veneer of objectivity with the magic of AI. Grab a few anecdotes to imply specificity, but then the actual judgments are unfalsifiable nothingburgers which probably apply to 95% of HN commenters.

A lot of tech folks seem deeply vulnerable to the psychological methods of psychics / tarot card readers / etc. Simply rejecting the metaphysics isn't enough when "magical energy of Jupiter" becomes "magical judgment abilities of the fancy computer."

nicklecompte··on It’s an age of marvels
Minor nitpick:

> A past what-the-fuck was that we know exactly how many cells there are (959) in a particular little worm, C. elegans, and how each of those cells arises from the division of previous cells, as the grows from a fertilized egg, and we know what each cell does and how they are connected, and we know that 302 of those cells are nerve cells, and how the nerve cells are connected together. (There are 6,720 connections.)

We have no clue what most C elegans neurons actually do, we only know how they are connected. The behavior of most individual C elegans neurons (and by extension the brain) is basically a mystery, and it might take a few decades of experimental advances to figure it out.

Not to downplay the coolness of this. But a lot of people seem to think "we know the C elegans connectome" means "we know how the C elegans brain works." In fact it tells us very little about the brain - an analogy I like is that it's akin to understanding a complex circuit purely by wires and solder, without knowing if you're connecting to a capacitor, an inductor, etc. The information is necessary but far from sufficient.

nicklecompte··on GPT-4o
What part of this makes you think GPT-4 suddenly developed a world model? I find this comment ridiculous and bizarre. Do you seriously think snappy response time + fake emotions is an indicator of intelligence? It seems like you are just getting excited and throwing out a bunch of words without even pretending to explain yourself:

> using language in a way that requires both a shared world model

Where? What example of GPT-4o requires a shared world model? The customer support example?

The reason GPT-4 does not have any meaningful world model (in the sense that rats have meaningful world models) is that it freely believes contradictory facts without being confused, freely confabulates without having brain damage, and it has no real understanding of quantity or causality. Nothing in GPT-4o fixes that, and gpt2-chatbot certainly had the same problems with hallucinations and failing the same pigeon-level math problems that all other GPTs fail.

nicklecompte··on Energy, not compute, will be the #1 bottleneck to AI progress – Mark Zuckerberg [video]
There is a very intelligent system, far more intelligent than GPT-4, which uses a tiny fraction of the energy. It's called a "rat," and despite requiring only 2 watts of power, it can perform the following tasks which are far beyond any multimodal LLM:

- creating and executing complex plans in arbitrary scenarios

- counting to 6 with 99% reliability in arbitrary scenarios

- understanding object permanence with 99% reliability

- understanding simple, intuitive causality with 99% reliability

But rats can't answer Python questions, so according to a truly depressing number of tech folks, of course GPT-4 is smarter than a rat! AI researchers/capitalists seriously need to spend more time reading animal cognition. If Silicon Valley was in charge of animal cognition research, spiders would be considered smarter than crows.

nicklecompte··on Cubic millimetre of brain mapped at nanoscale resolution
Badly: https://www.lesswrong.com/posts/mHqQxwKuzZS69CXX5/whole-brai... (the comments have some updates as of 2023)

Almost every other cell in the worm can be simulated with known biophysics. But we don't have a clue how any individual nematode neuron actually works. I don't have the link but there are a few teams in China working on visualizing brain activity in living C. elegans, but it's difficult to get good measurements without affecting the behavior of the worm (e.g. reacting to the dye).

nicklecompte··on Cubic millimetre of brain mapped at nanoscale resolution
I think you would also need the epigenetic side, which is very poorly understood: https://www.universityofcalifornia.edu/news/biologists-trans...

We have more detail than this about the C. elegans nematode brain, yet we still no clue how nematode intelligence actually works.

nicklecompte··on Cubic millimetre of brain mapped at nanoscale resolution
Crow/parrot brains are tiny but in terms of neuron count they are twice as dense as primate brains (including ours): https://www.sciencedirect.com/science/article/pii/S096098221...

If someone did this experiment with a crow brain I imagine it would look “twice as complex” (whatever that might mean). 250 million years of evolution separates mammals from birds.

nicklecompte··on No "Zero-Shot" Without Exponential Data
The whole point of past AI winters is that genuinely useful technologies were preposterously overhyped as representing "human cognition," despite overwhelming evidence to the contrary. The bubble bursts - and the money evaporates - because expectations come crashing down, not because the usefulness was a mirage. Lisp was hardly the key to human symbolic thought like some hoped it would be, but it helped improve programming languages across the board. Perceptrons really are quite limited and there's absolutely no way you could emulate a human brain with them alone, but they're still essential for more advanced ML.

But you would think after 70 years AI practitioners would learn some humility! It is very obvious that GPT-4 is dumber than a honeybee, let alone a cat, let alone a crow. But for over a year I've heard dozens of tech folks insist it's smarter than most people.

nicklecompte··on TimeGPT: Production Ready Time Series Foundation Model for Forecasting
The website (https://www.nixtla.io/) says all sorts of companies are using this: Ford, Lyft, Nestle, T-Mobile, Wayfair, etc etc.

That's not actually true, right? Surely that's a lie? I find it very unlikely that these huge companies are actually using this, especially considering how sketchy the website is (the About page doesn't list any employees/executives, there are no case studies, no white papers describing what this product even is.)

I get the impression this product only exists to scam investors who have unrealistic expectations of AI.

nicklecompte··on Stack Overflow users deleting answers after OpenAI partnership
If you check the byline, the author is a Microsoft MVP / product evangelist. So I don't think he's biased towards SO so much as he is biased towards anyone doing business with Microsoft (or OpenAI). He also seems very pro-GitHub Copilot.
nicklecompte··on Vets fret as private equity snaps up clinics, pet care companies
> PE firms aren’t running around flipping veterinarian firms for a couple hundred grand. They do that with fortune 500 companies for billions, not vet offices for tens of thousands. They’re buying them because they expect good ROI going forward.

This is a dumb thing to say. My views are almost entirely informed by how PE operates in mental health, aka actual facts and not stupid mental models of actors optimizing utility functions. According to you, Blackstone apparently thought childhood autism services was a good ROI going forward, and having to sell off all the assets was just an unfortunate little boo-boo. Wouldn't want to hurt PE's feelings by suggesting they were only in it for short-term gain. https://bhbusiness.com/2022/11/11/large-autism-provider-card...

Here they openly discuss the opportunities for profitable exits in mental health: https://bhbusiness.com/2022/11/15/private-equity-expected-to...

I don't think you are responding in good faith - it doesn't seem like you've read the article, you are just responding to the headline + your own fact-free misconceptions.

nicklecompte··on Vets fret as private equity snaps up clinics, pet care companies
> if they’re such a great profit source that PE firms are snapping them up

This is where the Econ 101 naiveté of your comment goes completely off the rails. I am wondering if you are trolling.

- A vet certainly does not need to be a "great" profit source for PE. I genuinely can't believe you'd say that with a straight face. If it barely breaks even and the owners end up flipping the assets and real estate, that's a win.

- The obvious reason PE has stepped up their investments in pet care is that COVID led to a huge spike in US pet ownership. In about 5 years this spike in vet demand will fade. When that happens, these firms will wring their investments dry before moving on, leaving debt-saddled clinics behind. You can't just ignore the actual economic context because you found a fact-free explanation that high schoolers find convincing.

- in general it might be barely profitable to run a good vet clinic, but highly profitable to run a terrible clinic. I doubt emergency vet medicine is very profitable. You can't just assume PE runs the clinics the same as a vet-owned clinic. In fact the article makes it clear that this is not the case.

- There is just a total lack of humility from you: being smug and ignorant is a bad combination. Sure, the article is full of people complaining about specific PE-owned vets, and the comments on HN are full of people complaining about specific PE-owned vets.... but maybe if they understood the law of supply and demand, they wouldn't be so upset.

← PreviousPage 3 of 16Next →