HNHacker News
TopNewBestAskShowJobs

trjordan

5,143 karma · joined October 8, 2008

Catch the decisions your agent didn't tell you about https://tern.sh

tr at tern dot sh

submissionscomments
trjordan··on Wysiwid: What you see is what it does
I feel like the main missing point of every "coding will go away" articles (which this mostly, but not fully, is) misses that applications of any reasonable complexity are not fully defined anywhere but their code. That drives everything about modern software.

We've always had the ability to tie the code extremely directly to user experiences. The first problem is that it creates massive code duplication. The second (related) problem is that it creates an enormous, unsquashable number of bugs. There's no internal consistency.

Modern apps are COMPLEX. The abstractions and internal, non-user structure is the only reason they're maintainable at all. I'm working an a 6-month-old, 4-person codebase, and LLM-driven refactors, with me in the drivers seat, miss stuff all the time. If I forget that feature exists in this tiny codebase, how do organizations of any size function? Abstraction, interfaces, etc. -- software design.

The future is probably closer to "LLMs help clean up codebases" than "LLMs own the codebase," because the 2nd statement is effectively equal to "I will rewrite my code in English." English is a _bad_ language for describing complex systems. We could do better than Python, sure, but it's already _far_ better than English for the kind of cross-cutting behavior-and-dependency description software requires. It's valuable to have tools that translate intent into compact, high-quality code, but you still end up working with code as the fundamental artifact.

trjordan··on We can’t circumvent the work needed to train our minds
I was talking with somebody about their migration recently [0], and we got to speculating about AI and how it might have helped. There were basically 2 paths:

- Use the AI and ask for answers. It'll generate something! It'll also be pleasant, because it'll replace the thinking you were planning on doing.

- Use the AI to automate away the dumb stuff, like writing a bespoke test suite or new infra to run those tests. It'll almost certainly succeed, and be faster than you. And you'll move onto the next hard problem quickly.

It's funny, because these two things represent wildly different vibes. The first one, work is so much easier. AI is doing the job. In the second one, work is harder. You've compressed all your thinking work, back-to-back, and you're just doing hard thing after hard thing, because all the easy work happens in the background via LLM.

If you're in a position where there's any amount of competition (like at work, typically), it's hard to imagine where the people operating in the 2nd mode don't wildly outpace the people operating in the first, both in quality and volume of output.

But also, it's exhausting. Thinking always is, I guess.

[0] Rijnard, about https://sourcegraph.com/blog/how-not-to-break-a-search-engin...

trjordan··on ChatGPT is NOT a LLM – GPT is
AI sloppiness of this blog post aside, it's a reasonable observation.

If you're thinking about how to integrate AI into your system, it's worth asking the question of why your system isn't just ChatGPT.

- Do you have unique data you can pass as context?

- Do you have APIs or actions that are awkward to teach to other systems via MCP?

- Do you have a unique viewpoint that you are writing into your system prompt?

- Do you have a way to structure stored information that's more valuable than freeform text memories?

- etc.

For instance, we [0] are writing an agent that helps you plan migrations. You can do this with ChatGPT, but it hugely benefits from (in descending order of uniqueness) access to

1) a structured memory that's a cross between Asana and the output of `grep` in a spreadsheet,

2) a bunch of best-practice instructions on how to prep your codebase for a migration, and

3) the AI code assistant-style tools like ls, find, bash, etc.

So yeah, we're writing at agent, not building a model. And I'm not worried about ChatGPT doing this, despite the fact that GPT5 is pretty good at it.

[0] https://tern.sh

trjordan··on Development speed is not a bottleneck
This article is right insofar as "development velocity" has been redefined to be "typing speed."

With LLMs, you can type so much faster! So we should be going faster! It feels faster!

(We are not going faster.)

But your definition, the right one, is spot on. The pace of learning and decisions is exactly what drives development velocity. My one quibble is that if you want to learn whether something is worth doing, implementing it isn't always the answer. Prototyping vs. production-quality implementation is different, even within that. But yeah, broadly, you need to test and validate as many _ideas_ as possible, in order take make as many correct _decisions_ as possible.

That's one place I'm pretty bullish on AI: using it to explore/test ideas, which otherwise would have been too expensive. You can learn a ton by sending the AI off to research stuff (code, web search, your production logs, whatever), which lets you try more stuff. That genuinely tightens the feedback loop, and you go faster.

I wrote a bit more about that here: https://tern.sh/blog/you-have-to-decide/

trjordan··on I'm absolutely right
OK, so I love this, because we all recognize it.

It's not fully just a tic of language, though. Responses that start off with "You're right!" are alignment mechanisms. The LLM, with its single-token prediction approach, follows up with a suggestion that much more closely follows the user's desires, instead of latching onto it's own previous approach.

The other tic I love is "Actually, that's not right." That happens because once agents finish their tool-calling, they'll do a self-reflection step. That generates the "here's what I did response" or, if it sees an error, the "Actually, ..." change in approach. And again, that message contains a stub of how the approach should change, which allows the subsequent tool calls to actually pull that thread instead of stubbornly sticking to its guns.

The people behind the agents are fighting with the LLM just as much as we are, I'm pretty sure!

trjordan··on Google AI Overview made up an elaborate story about me
This is almost certainly what happened. Google's AI answers aren't magic -- they're just summarizing across searches. In this case, "Israel" + "Jordan" pulled back a video with opposite views than the author.

It's somewhat less obvious to debug, because it'll pull more context than Google wants to show in the UI. You can see this happening in AI mode, where it'll fire half a dozen searches and aggregate snippets of 100+ sites before writing its summary.

trjordan··on Astrophysicists find no 'hair' on black holes
Not exactly. Black holes are not lopsided, for instance. There isn't anything on "one side" of them, once the matter has crossed the event horizon.

Rotating black holes are pretty well modeled as a spinning ring: https://en.wikipedia.org/wiki/Ring_singularity

trjordan··on How to Fix Your Context
It only spawns a single sub-agent (called Task iirc), which can do everything Claude Code can, except call Task().

This is different from a lot of the context-preserving sub-agents, which have fully different toolsets and prompts. It's much more general.

trjordan··on How to Fix Your Context
This is all true, and we've prototyped a number of these things at my current startup. You need to be pretty considered about implementing them.

For a counter-example, consider Claude Code:

- 1 long context window, with (at most) 1 sub-agent

- same tools available at all times, and to sub-agent (except: spawning a sub-sub-agent)

- Full context stays in conversation, until you hit the context window limit. Compaction is automatic but extremely expensive. Quality absolutely takes a dive until everything is re-established.

- Deterministic lookup of content. Claude reads files with tools, not includes random chunks from RAG cosine similarity.

I could go on. In my experience, if you're going to use these techniques 1) maybe don't and 2) turn up the determinism to 11. Get really specific about _how_ you're going to use, and why, in a specific case.

For example, we're working on code migrations [0]. We have a tool that reads changelogs, migration guides, and OSS source. Those can be verbose, so they blow the context window on even 200k models. But we're not just randomly deleting things out of the "plan my migration" context, we're exposing a tool that deliberately lets the model pull out the breaking changes. This is "Context Summarization," but before using it, we had to figure out that _those_ bits were breaking the context, _then_ summarizing them. All our attempts at generically pre-summarizing content just resulted in poor performance because we were hiding information from the agent.

[0] https://tern.sh

trjordan··on How can AI ID a cat?
One of the funny things about LLMs and modern AI is that "the ability to recognize a cat" isn't a trained behavior anymore, as described here. It's an emergent property of training it to predict a lot of things, and cats happens to be present enough in the data such that they're one of the things you can ask a larger model and have it work.

My favorite work on digging into the models to explain this is Golden Gate Claude [0]. Basically, the folks at Anthropic went digging into the many-level, many-parameter model and found the neurons associated with the Golden Gate Bridge. Dialing it up to 11 made Claude bring up the bridge in response to literally everything.

I'm super curious to see how much of this "intuitive" model of neural networks can be backed out effectively, and what that does to how we use it.

[0] https://www.anthropic.com/news/golden-gate-claude

trjordan··on Sprinkling self-doubt on ChatGPT
We've been building out our agent [0], and we've found this to be the case.

We actually dialed it back a bunch, because it feels _terrible_. Yes, you get more correct answers, but it's more akin to giving the agent anxiety. Especially with agents that have access to tools, they'll burn enormous amounts of time on tool calls, trying to get enough information to overcome a motivation that's essentially burned into its identity.

(We saw one conversation where it just browsed social media instead of looking at the code for like 5 minutes, which ... you know, I get it.)

It's been much more effective to make uncertainty or further exploration be part of the agents success criteria.

- BAD: "Critique your own thoughts" -> leads to the agent trying really hard to get it right, but still not willing to actually be wrong

- GOOD: "Expose where your thoughts are unsupported or could benefit from further information" -> leads to the agent producing high-quality results, with loose ends that the user can choose to incorporate, ignore, or correct.

That prompt, combined with dialing up the thinking (either with API or prompt tuning) works much better, because it's sidestepping the training and tuning that's implicitly encouraged it to sound correct at all times.

[0] https://tern.sh, code migration AI

trjordan··on Why is D3 so Verbose?
No fun to have to do a panic migration!

Anyway my latest startup is trying to make AI do migrations, in no small part because of this problem 15 years ago.

trjordan··on Why is D3 so Verbose?
Man, my first startup in 2010 used protovis, the charting library Mike Bostock built before deciding d3 was the better approach. It was rough to have an 8 month old startup with a core piece of tech that suddenly stopped improving.

My main takeaway from so much of this is that "just a chart" is one of the biggest sources of hidden complexity in displaying useful information to people. It's right up there with "a simple web form" and "a web page with some simple interactivity."

Everybody has a wildly different idea of what good looks like. Defaults will never be right. Personal and global taste changes annually. We clown react (rightly) for constantly reinventing the same 4 wheels, but customers gleefully use new stuff all the time.

It's kind of amazing that d3 has been so durable in the frontend world. It really is a wrapper over a pretty solid approach. And yeah, that approach is complex, but that's the reality of visualization. It's hard to imagine another one that's that good.

trjordan··on Attention Is the New Big-O: A Systems Design Approach to Prompt Engineering
A few of these points are right, but a lot of it is so far lost from the reality of LLMs that it isn't even wrong.

Yes: structure beats vibes. Primacy/recency bias is real. Treating prompts as engineering artifacts is empirically helpful.

But:

- “Reads all tokens at once.” Not quite. Decoder LLMs do a parallel prefill over the prompt, then sequential token-by-token decode with causal masks. That nuance is why primacy/recency and KV-cache behavior matter, and why instruction position can swing results.

- Embeddings & “labels.” Embeddings are learned via self-supervised next-token prediction, not from a labeled thesaurus. “Feline ≈ cat” emerges statistically, not by annotation.

- "Structure >> content". Content is what actually matters. “Well-scaffolded wrong spec” will give you confidently wrong output.

- Personas are mostly style. Yes, users like words typed in their style better, but it'll actually hide certain information that a "senior software engineer" might not know.

I don't really get the Big-O analogy thing, either. Models are constantly exposing and shifting how they direct attention, which is exactly the opposite of the durably true nature of algorithmic complexity. Memorizing how current models like their system prompts written is hardly the same thing.

trjordan··on We shouldn't have needed lockfiles
Of course there's no 0-risk version of any of this. But in my experience, bugs tend to get introduced with features, then slowly ironed out over patches and minor versions.

I want no security bugs, but as a heuristic, I'd strongly prefer the latest patch version of all libraries, even without perfect guarantees. Code rots, and most versioning schemes are designed with that in mind.

trjordan··on We shouldn't have needed lockfiles
There is absolutely a good reason for version ranges: security updates.

When I, the owner of an application, choose a library (libuseful 2.1.1), I think it's fine that the library author uses other libraries (libinsecure 0.2.0).

But in 3 months, libinsecure is discovered (surprise!) to be insecure. So they release libinsecure 0.2.1, because they're good at semver. The libuseful library authors, meanwhile, are on vacation because it's August.

I would like to update. Turns out libinsecure's vulnerability is kind of a big deal. And with fully hardcoded dependencies, I cannot, without some horrible annoying work like forking/building/repackaging libuseful. I'd much rather libuseful depend on libinsecure 0.2.*, even if libinsecure isn't terribly good at semver.

I would love software to be deterministically built. But as long as we have security bugs, the current state is a reasonable compromise.

trjordan··on 6 weeks of Claude Code
I think the most interesting change Claude enables is letting AI try stuff. We do this all the time.

I have this sense this works best in small teams right now, because Claude wants to produce code changes and PRs. Puzzmo, where OP works, is <5 engineers.

In larger codebases, PRs don't feel like the right medium in every case for provocative AI explorations. If you're going to kick something off before a meeting and see what it might look like to solve it, it might be better to get back a plan, or a pile of regexps, or a list of teams that will care.

Having an AI produce a detailed plan for larger efforts, based on an idea, seems amazing.

trjordan··on Open Source Maintenance Fee
I love the innovation. The basic idea here appears to be:

- Nobody wants this to be closed source. The code is freely available, and you may do with it as you want. The marginal cost to distribute the code is 0, after all.

- The maintainers, as people, don't want to do charity work for companies. Their time is limited, and if they're going to support revenue-generating activities, they want a cut of the revenue.

So even if this doesn't get perfectly enforced (and it won't), that's fine! The maintainers are now free to respond to complaints with "you need to pay us for us to care." Companies that pay get some level of support; hobbyists get the same experience. Only the companies that ignore this warning will see the consequences, and it's particularly effective for reports where the author leans on "but there are a huge number of important users [to me] that are affected." Pay up if it matters!

It strikes me as a pretty clean solution to a pretty common strain of open-source headache, _especially_ as AI-generated code/reports/etc. are on the rise.

trjordan··on Is the doc bot docs, or not?
The core argument here is: LLM docbots are wrong sometimes. Docs are not. That's not acceptable.

But that's not true! Docs are sometimes wrong, and even more so if you could errors of omission. From a users perspective, dense / poorly structured docs are wrong, because they lead users to think the docs don't have the answer. If they're confusing enough, they may even mislead users.

There's always an error rate. DocBots are almost certainly wrong more frequently, but they're also almost certainly much much faster than reading the docs. Given that the standard recommendation is to test your code before jamming it in production, that seems like a reasonable tradeoff.

YMMV!

(One level down: the feedback loop for getting docbots corrected is _far_ worse. You can complain to support that the docs are wrong, and most orgs will at least try to fix it. We, as an industry, are not fully confident in how to fix a wrong LLM response reliably in the same way.)

trjordan··on Why LLMs Can't Write Q/Kdb+: Writing Code Right-to-Left
Seems like it could easily be training data set size as well.

I'd love to see some quantification of errors in q/kdb+ (or hebrew) vs. languages of similar size that are left-to-right.

trjordan··on Google’s two-year frenzy to catch up with OpenAI
ChatGPT has 400m weekly users. X / Twitter has 600m.

It's pretty popular!

trjordan··on Advent of Code 2024 in pure SQL
If you like this kind of degeneracy, I tried AoC in Google Sheets this year.

I only made it to Day 6, and not even both stars every day. I'm pretty confident my Day 7 solution is correct, but I hit per-cell character limits on longer inputs.

Enjoy :) oh but don't open it on mobile, some sheets crash the app

https://docs.google.com/spreadsheets/d/10FY-89y19tnRM_EAAnCd...

trjordan··on Monolith First (2015)
If you don't want to scale the whole thing (why?), you can also deploy the monolith twice and route different API calls to different clusters. Slice and scale as much as you're willing, paying the price of deployment complexity and bin-packing the usage.
trjordan··on Accident Forgiveness
The servers are already bought; the electricity cost is negligible for actual accidents.

As mentioned in the post: the hosting providers don't actually pay marginal costs for transient mistakes, so neither should honest customers.

trjordan··on The Apple Vision Pro’s missing apps
I don't think you're an outlier, but on the other side, "I watch Netflix on my laptop / iPad" is a _huge_ market. Yes, there's plenty of social watching experiences to be figured out, but solo watching is an entire industry unto itself.

I'm a bit like you. I don't watch TV by myself, and the idea of plugging into a VR device to do so seems weird.

But I carry my AirPods everywhere and mostly listen to music solo. It's not unreasonable to think that the same happens with TV/movies.

trjordan··on Which word begins with "y" and looks like an axe in this picture? (2017)
Friends, please, click on the link. There's a long and interesting investigation that answers the question, which is presumably why this was posted.
trjordan··on Corporate legibility for software engineers
Legibility also increases the ability for a large organization to chase singular ambitious goals.

For example: software estimation is hard. It takes time, it's highly inaccurate (frequently by 2x or more), and rewarding good estimation incentives worse software.

But having deadlines helps functions like marketing and sales immensely. Marketing can plan effective launches, and sales can guide customers towards new products with minimal lag in uptake.

Lots of corporate life is like this. It's a net good if it's used to deliver a better experience to customers. It's a constant fight to make sure legibility efforts aren't being used to hoard power for personal gain.

trjordan··on Ask HN: Is there VC appetite for defense related startups?
The article you linked basically answers your question. The standard model for defense companies isn't VC-friendly -- you're either building hardware with a limited ceiling, or you're building a consulting business that bills by the hour. Those can be fine businesses, but they're not amenable to wiring $10m and getting back $100m 3 years later.

Defense buyers aren't inherently irrelevant, though. I'm working on a defense-related project at a VC-backed company, and plenty of our peers are. We're selling a standard SaaS model, and that continues to support our business model.

The real challenge is that most tech companies scale up before going after defense, so they can absorb the costs. If you're small, you're going to have to figure out how to scale quickly in the defense world. If you can crack that, I'm sure you'd find a VC or two.

trjordan··on Ask HN: I've lost faith in myself as a developer, how do I get it back?
As CTO, you probably don't have the time and space, structurally, to be a good developer. You have things to do that aren't a single project or feature! So of course other developers will be better at "being a developer" than you, by some measures.

I've seen two successful models of CTO that are "good developers:"

1. Knows the core tech of the company really well. There's some small bit of tech at your company that's legit an enabler of your growth. You should probably know it better than anybody. Maybe it's moved or changed or grown in the last 8 years, and you've lost touch a bit? It seems reasonable to carve out time to build something on your own, in the same space, using the new stuff. You probably have a better sense of what's important to get right than anybody else, so building a personal project with that tech on company time _is_ actually a great use of time.

2. Knows the system dependencies better than anybody else. This particularly applies if you have reliability woes or other systematic issues. You have a great position to ask other teams to educate you. Get a couple good devs from each team in a room, drop your ego, and have them teach you what they know in their domain. Do this across a bunch of teams. Some folks might find this irritating, but most folks will like the time to show off their work and share their worries.

Everybody is constrained by where they spend their time. If you want to go back to being a great IC ... you need a different job. Which it might be time for! But you can be technically great as a CTO, and I'm sure I've missed some other ways you can do it in the context of you "day job."

trjordan··on Twitter’s new CEO announces major reorganization
Contrast to Apple, which uses a fully functional model: https://hbr.org/2020/11/how-apple-is-organized-for-innovatio...

I find this evolution fascinating. It makes sense to me that the GM model worked for so long. If much of a company's advantages were cultural, you could put together teams that could out-compete startups within a large organization. Procter and Gamble needs to enter new markets with new products to grow -- there's only so big a market for soap or razors or drills.

But lots of tech companies today have a significant scale advantage. Google ads is most of their revenue; Netflix has a single product. Functional expertise that drives a 2% gain across the entire business is possible and valuable.

So ... why does Twitter want to go to a GM model? Are they watching Meta have success diversifying from Facebook? Or is there something else?

← PreviousPage 4 of 20Next →