HNHacker News
TopNewBestAskShowJobs

jameshart

22,635 karma · joined March 31, 2011

submissionscomments
jameshart··on “Next-token predictor” is the wrong mental model for LLMs
Yep, we’re all just putting one foot in front of the other, hoping we’re doing the right thing to bring about the outcomes we want, trying our best.

But that’s the point: so is an LLM. Putting one token in front of another, hoping it’s doing the right thing to bring about the rewards it’s trained to… trying its best.

So yeah, not ‘next token predictors’. ‘Next token tryers’ maybe.

jameshart··on “Next-token predictor” is the wrong mental model for LLMs
But if you are an NFL coach and I ask you to decide your next action in order to maximize your odds of winning the superbowl, while yes that does involve you having some predictive ability to think about what impact your actions would have on your odds of winning the Super Bowl… I don’t think you would call the process that you use to decide that next action ‘prediction’.
jameshart··on “Next-token predictor” is the wrong mental model for LLMs
I think it’s a disagreement about what ‘predict’ means.

The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on.

Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful assistant’ or ‘sets of changes to a codebase’.

And that isn’t really ‘prediction’, so much as ‘generation’.

It’s not been tuned to ‘guess the next token right’. It’s been tuned to generate the token that leads to it ultimately scoring highest on its reward function.

It’s not predicting the token, it’s predicting the reward.

jameshart··on Formalizing Fermat's Last Theorem
There is no way Fermat could have fit that in the margin. Definitely vindicated.
jameshart··on New York Times and The Athletic workers demand company scrap Kalshi deal
If you and the buyer know about the risks then you’re not making a trade based on material non public information, are you?

If you make a hedging trade while having material knowledge that you withhold from the counterparty… well, that sounds like it could be edging towards fraud. Like taking out life insurance while hiding a medical condition.

But sure, it’s not against the law to do business with a sucker, so there’s degrees here.

jameshart··on New York Times and The Athletic workers demand company scrap Kalshi deal
Same old story, on a whole new scale. Donny Junior has a piece the numbers game on the east side, so the family makes sure the cops don’t come round asking too many questions…
jameshart··on Any Human Ever – One life, drawn at random from all who have ever lived
Managed to roll a guy born in 1988 in the eastern US. He hasn’t married yet and he’s still alive.
jameshart··on It's OK to hardcode feature flags (2025)
Golden rule of large scale software dev: any system you expose that allows other teams to write text to it will eventually be used by another team as an ad hoc JSON database
jameshart··on It's OK to hardcode feature flags (2025)
In my experience these (configuration based feature flags and feature flag services) are actually two complimentary capabilities that solve two completely different problems but that happen to share the same name: feature flags.

The config approach is critical for using feature flags as a software development lifecycle tool. It is how you manage having a codebase which contains the unfinished code for new feature x, but can still be deployed and pass all tests without feature x being turned on.

In this model you need a mechanism which allows a developer who is working on feature x to enable it for local testing, and for your CI system to be able to interact with the flag system to test that the application works in both states - with x turned on and off.

This is ideal for trunk based development models; feature branches are an alternative approach that doesn’t really benefit from this (indeed it adds complexity to working in feature branches).

Meanwhile feature flag services are to solve the problem that different people using the same software need different features turned on. That can be as simple as internal testers or beta users, it can be holding features to roll out in fixed update windows per tenant, or it can be part of a risk management strategy where features are rolled out through progressive exposure.

It can also be tempting to mix up your feature flags system with an A/B testing system - you can use a feature flag service to expose a feature to a test cohort and measure performance changes.

There can be reasons for doing that but it’s really important not to tie all these things together: not every development lifecycle change is an ab test hypothesis. Not every ab test hypothesis is a development lifecycle change (often it is really about testing changes in data, and feature config is just one piece of data you might want to change). Similarly some other data changes than code changes need to roll out progressively to mitigate risk.

So all these things might be feature-flag shaped, but that doesn’t mean you can substitute different feature flag solutions in and solve the same problems.

This post is saying ‘it’s okay not to have an exposure control solution; you can have a config file’ - which is obviously true, if what you have is a config management problem, not an exposure control problem.

jameshart··on I turned my security cameras into an automatic bird identification system
OHHHH! That explains it. Had not figured out why that false detection kept coming up.
jameshart··on Hacking IKEA Furniture
Billy bookcases though are just slabs of laminated engineered wood cut to rectangular dimensions, drilled with precision positioned and sized holes, plus a stack of dowels and camlock fasteners.

That seems parameterizable.

jameshart··on Hacking IKEA Furniture
One of the best furniture life hacks you can recognize is that you can replace the top panel of most self-assembly storage furniture items with a custom work surface suited to your usecase.

Take two drawer units, and instead of putting the tops on them they were supplied with, put a slab of desktop across both of them with a gap between and you’ve got yourself a desk.

jameshart··on GUIs should be fully keyboard-driven
Yes, but this article is about TUIs
jameshart··on GUIs should be fully keyboard-driven
I’m specifically talking about native apps. iOS apps built using UiKit are going to have first class accessibility support.
jameshart··on GUIs should be fully keyboard-driven
Not requiring a mouse isn’t a magic pathway to ‘accessibility’ though. The fact that something can be used with a keyboard does not obviate developers from the need to consider other aspects of accessibility.

Also: when I copy paste out of an application I don’t want a bunch of spurious chrome glyphs and newlines - copying just the text contents of a TUI scrollable text field isn’t trivial and isn’t always guaranteed to work

jameshart··on Glacier Mice
Huh

> they have been observed in Alaska, Chile, Greenland, Iceland, Svalbard, Uganda and Venezuela

Well, today I learned, Venezuela has glaciers.

Then I googled that, and…

https://www.theguardian.com/environment/article/2024/may/08/...

And so today I learned, Venezuela used to have glaciers.

jameshart··on GUIs should be fully keyboard-driven
In the context of discussing TUI vs GUI (as the article does), the accessibility and keyboard control balance probably swings very strongly to native GUIs; in a TUI you’re at the whim of the TUI framework’s stabs at accessibility support, while on a full fledged native UI you have all the bells and whistles of focus management and text to speech and voice control and high contrast and magnification and motion adjustments…

The idea that it must be more accessible because it’s confined to a mono space typeface and communicates state mostly by changing the foreground and background color of the text it renders makes no sense to me.

jameshart··on Just the rumour of a bug is enough to find an exploit these days
I do wonder what the hit rate is in general for Claude finding a successful exploit when prompted with a rumor that leads it to assume the bug is there.

“I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.

jameshart··on Humanity has the debate about AI consciousness backwards
You can't just go from 'look how much trouble we have with race' and then say 'The last time this came up was about 400,000 years ago'.

No, the last time this came up was... well, take your pick. Dred Scott? Wannsee? The Rwandan Crisis Committee?

jameshart··on U.S. State Department pauses immigrant visa applications
That is not actually what that article says.

It’s a common pattern to say ‘company X laid off thousands of employees but then applied for thousands of H1B visas’ which very much gives the impression that the company has just fired a few thousand Americans and hired a few thousand foreigners.

What actually happens is: a company has a global workforce of 100000 or so. 50000 of them are in the US. 5000 of them are in the US on H1Bs, 45000 of them are Americans.

Company does a 10% layoff: 4500 Americans, 500 people who are in the us on visas, and 5000 people around the world.

Headline: “Company X lays off 10000”

Then the H1B renewal numbers come out and what do you know company X is applying for 4500 H1Bs, to retain their existing H1B staff who weren’t affected by the layoff.

Headline: “After laying off 10,000 staff company x applies for 4500 H1B visas”

Unstated in such a headline: 5500 of those layoffs were foreigners; the company still employs 40500 Americans.

To bring it back to your point: you cited this article as evidence for the claim that Microsoft laid off ‘thousands of domestic employees’; the article does not say where the thousands of layoffs were and explicitly contains a quote from an MS spokesperson that says “H-1B employees were also impacted by job eliminations in the U.S.”

jameshart··on U.S. State Department pauses immigrant visa applications
Employment based and immigrant are not opposite categories of visa - these are orthogonal axes.

There are employment-based nonimmigrant visas like H1B and L1. Being a nonimmigrant visa means it doesn’t grant you a right to immigrate but some (including H1B and L1) are ‘dual intent’ visas meaning you can apply for an immigrant visa while in the US as a nonimmigrant.

There are employment-based immigrant visas, which have categories like EB1, EB2. Like all immigrant visas these are also known as ‘legal permanent residency’, LPR, or ‘green cards’.

There are also non-employment-based visas that are both nonimmigrant (eg visitor visas, student visas) and immigrant (eg family based green cards)

jameshart··on U.S. State Department pauses immigrant visa applications
That seems like a bias you should work on.
jameshart··on Nvidia agrees to acquire Hugging Face for $13B
150 million a year? Why, at that rate NVidia will make their money back in just 86 years.
jameshart··on Qwen3.8-Flash-Next
One of the consequences of encountering a lot of LLM generated text which includes things the model vaguely remembers from its training is that honestly I have grown less tolerant even of human comments and documents that are based on mostly ‘I seem to recall that…’ level sourcing.

In a discussion on economic history, say, someone will opine that Alexander Hamilton had some particular opinion about tariff policy… based on their having a vague memory of a blog post where someone quoted a passage in support of some point. But wait - you can search the federalist papers, the text’s right there to be read, before you commit to saying online ‘Hamilton thought tariffs were a great idea’ you could take your internal ‘I seem to recall reading something about hamilton’s opinion on tariffs’ thought and turn it into a little RAG query where you pull up a source and check before you put another factoid out onto the internet.

And so I feel absolutely the same way about LLMs. I don’t care how much factual information was in the training data, when the LLM wants to rely on something it vaguely recalls having been trained on, it owes it to me to dig up a source and vet it.

There are limits to this, of course. I don’t want it to be thinking ‘but wait, maybe my memory of Python syntax is faulty. Is = used for assignment? <web search>…’.

But in general some caution about repeating vaguely recalled easily checked facts is warranted.

jameshart··on Qwen3.8-Flash-Next
I don’t think that’s the right way to think about LLM ‘knowledge’. They don’t have absolute recall of everything in the training set. They have been trained so that they have weights that can predict what those books might say - that is, if they read them they would find the contents unsurprising. That doesn’t mean it wouldn’t be helpful to pull relevant passages of text directly into context for a particular task.
jameshart··on RAG Is Simpler Than You Think
I think people also overestimate the need for full text search when the one doing the querying is an LLM. If your underlying data is structured records, like a customer database, while humans might not have time or skills to figure out that when they want to search by phone number they need to do a join from the contacts table to the users table and normalize the phone number to look up first, making it best to just surface phone numbers as part of the data that is full/text-indexed… an agent is quite happy to handcraft the right SQL to find records that match on a specific field, given the right SKILLS.md and schema information. Turning fuzzy searches into exact DB lookups is a great way LLMs can augment users.

(Obviously this doesn’t apply to searching actual rich document data - for that, go all in on text search, embedding, etc)

jameshart··on RAG Is Simpler Than You Think
Embedding search is effectively machine translation into a single common ‘language’ - embedding space - and then searching that; cleaner and less lossy than translating everything into English for searching, but harder to debug when it goes wrong.
jameshart··on Tooltips need a delay, and then they need to skip it
Feels like the delay also gets implemented into displaying the tooltip on tap on a touch interface which is probably an error.
jameshart··on Apple introduces M6 and M5 Ultra
You could spend 100 a month leasing one of these for three years and get the best of both worlds
jameshart··on Coconut oil jet fuel matches kerosene's efficiency in engine tests
Maybe those cargo cult guys were onto something
← PreviousPage 6 of 34Next →