Zachzhao

> Coding agents are sirens, luring you in with their speed of code generation and jagged intelligence, often completing a simple task with high quality at breakneck velocity. Things start falling apart when you think: "Oh golly, this thing is great. Computer, do my work!".

But the rough edges are temporary. Coding agents are becoming superhuman along certain dimensions; the progress is staggering. As Andrej Karpathy put it, anything measurable or legible can be optimized by AI. The gaps will close fast.

The harder question is HCI. How do you expose this kind of intelligence in interfaces that actually align with human values? That's the design problem worth obsessing over.

You're right - hallucinations aren't limited to citations. We see a few failure modes:

Fabricated citations: Case doesn't exist at all

Wrong citation: Case exists but doesn't say what the model claims

Misattributed holdings: Real case, real holding, but applied incorrectly to the legal question

From our internal testing, proper context engineering significantly reduces hallucination across the board.

Once we ground the model in the relevant source documents, hallucination rates drop substantially.

Thanks for asking! We're using a multi-source approach:

Case law: Google Scholar + CourtListener's bulk data (great coverage of federal and state appellate decisions).

Statutes & regulations: Currently using Justia for state statutes, but working on scraping directly from state legislature sites. U.S. Code from the Office of Law Revision Counsel's XML releases, and eCFR's APIs for federal regulations.