HNHacker News
TopNewBestAskShowJobs

ewoodrich

2,349 karma · joined October 23, 2011

submissionscomments
ewoodrich··on Claude Opus 4.5
But it doesn't evaluate the area that I am most eager to see improvements in LLM agent performance: unattended complex tasks that require adapting to unexpected challenges, problem solving and ambiguity for a long duration without a human steering them back in the right direction before they hit a wall or start causing damage.

If it takes me 8 hours to create a pleasant looking to-do app, and Gemini 3 can one shot that in 5 minutes, that's certainly impressive but doesn't help me evaluate whether I could drop an agent in my complex, messy project and expect it to successfully implement a large feature that may require reading docs, installing a new NPM package, troubleshooting DB configuration, etc for 30 min to 1 hr without going off the rails.

It's a legitimate benchmark, I'm not disputing that, but it unfortunately isn't measuring the area that could be a significant productivity multiplier in my day-to-day work. The METR time horizon score is still susceptible to the same pernicious benchmaxxing while I had previously hoped that it was measuring something much closer to my real world usage of LLM agents.

Improvements in long duration, multi-turn unattended development would save me lot of babysitting and frustrating back and forth with Claude Code/Codex. Which currently saps some of the enjoyment out of agentic development for me and requires tedious upfront work setting up effective rules and guardrails to work around those deficits.

ewoodrich··on Claude Opus 4.5
I was pretty disappointed to learn that the METR metric isn't actually evaluating a model's ability to complete long duration tasks. They're using the estimated time a human would take on a given task. But it did explain my increasing bafflement at how the METR line keeps steadily going up despite my personal experience coding daily with LLMs where they still frequently struggle to work independently for 10 minutes without veering off task after hitting a minor roadblock.

  On a diverse set of multi-step software and reasoning tasks, we record the time needed to complete the task for humans with appropriate expertise. We find that the time taken by human experts is strongly predictive of model success on a given task: current models have almost 100% success rate on tasks taking humans less than 4 minutes, but succeed <10% of the time on tasks taking more than around 4 hours. This allows us to characterize the abilities of a given model by “the length (for humans) of tasks that the model can successfully complete with x% probability”.

  For each model, we can fit a logistic curve to predict model success probability using human task length. After fixing a success probability, we can then convert each model’s predicted success curve into a time duration, by looking at the length of task where the predicted success curve intersects with that probability.
[1] https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...
ewoodrich··on New OS aims to provide (some) compatibility with macOS
Android permissions began to ask for individual confirmation on first use in Android 6.0 (released in 2015) so the grant-all-on-install model hasn't been how it works in a very long time.

Also your narrative about iOS moving from locked down to opening things up over time isn't entirely accurate, when iOS (iPhoneOS) was first released, it didn't have any concept of permissions at all! Apps could use whatever API the OS offered with the user none the wiser. At that time Android Market forcing developers to disclose which permissions were required was seen as unusually transparent and secure. Random iPhone apps scanning contacts deceptively pushed Apple to adopt a permissions model several years after the iPhone was first released.

The two platforms have historically leap frogged each other in various ways but at this point have started to converge as mobile settled into a boring appliance instead of groundbreaking new computing paradigm. Apart from sideloading, notifications and some minor annoyances here and there I can almost forget which OS I'm using as I switch between iOS and Android (thanks to gestures removing the trademark home/back navigation distinctions).

ewoodrich··on Microsoft AI CEO pushes back against critics after recent Windows AI backlash
> It's wild that most developers in a corporate environment pay Microsoft to use Chat GPT, Gemini, and Claude

It definitely didn't help back when my manager asked me to recommend a subscription to buy everyone on our team that Anthropic didn't offer any plan with a predictable monthly cost for Claude Code with SSO/externally managed billing (I think that changed fairly recently).

Github Copilot for Business with an easily digestible flat monthly rate + straightforward per request rate beyond the quota (for devs who actually ended up using it heavily) made it extremely painless to get approved.

Cursor was really the only other subscription offering that checked all those boxes but our team uses the official Microsoft VS Code extensions and there was 0% chance of getting buy-in if it meant disrupting everyone's workflow for a 6 month trial period.

ewoodrich··on Microsoft AI CEO pushes back against critics after recent Windows AI backlash
Agreed, some days I code for 4-6 hours with agentic tools but 2025 or not I still can't stomach using any of the big three LLMs for all but the most superficial research questions (and I currently pay/get access to all three paid chatbots).

Even if they were right 9/10 (which is far from certain depending on the topic) and save me a minute or two compared to Google + skim/read-ing a couple websites, it's completely overshadowed by the 1/10 time they calmly and confidently lie about whether tool X supports feature Y and send me on a wild goose chase looking through docs for something that simply does not exist.

In my personal experience the most consistently unreliable questions are those that would be most directly useful for my work, and for my interests/hobbies I'd rather read a quality source myself. Because, well, I enjoy reading! So the value proposition for "LLM as Google/forum/Wikipedia replacement" is very, very weak for me.

ewoodrich··on Microsoft AI CEO pushes back against critics after recent Windows AI backlash
Reminds me of the classic in the genre:

  how about a couple of weeks of gratitude for magic intelligence in the sky, and then you can have more toys soon?

  Sam Altman, Sep 12, 2024
https://xcancel.com/sama/status/1834351981881950234?lang=en
ewoodrich··on Gemini 3
I can sympathize with being mistakingly accused of using LLM output, but as a reader the above format of "Its not x - it's y" repeated multiple times for artificial dramatic emphasis to make a pretty mundane point that could use 1/3 the length grates on me like reading LinkedIn or marketing voice whether it's AI or not (and it's almost always AI anyway).

I've seen fairly niche subreddits go from enjoyable and interesting to ruined by being clogged with LLM spam that sounds exactly like this so my tolerance for reading it is incredibly low, especially on HN, and I'll just dismiss it.

I probably lose the occasionally legitimate original observation now and then but in a world where our attention is being hijacked by zero effort spam everywhere you look I just don't have the time or energy to avoid that heuristic.

ewoodrich··on Windows 11 adds AI agent that runs in background with access to personal folders
The very light moderation (that even shows dead comments from banned accounts) and clean, minimal frontend with essentially no restrictions on creating throwaways also makes it pretty attractive for "my first AI app" experiments. Ever since GPT 3 was released I see a graveyard with a scattering of dead, green, obvious LLM replies on most articles, sometimes with account names like "accounttest14" that don't even try to hide it.
ewoodrich··on Gemini 3
> The last point is how it usually fails in my testing, fwiw. It usually ends up borking something up, and rather than back out and fix it, it does a 'git restore' on the file - wiping out thousands of lines of unrelated, unstaged code. It then somehow thinks it can recover this code by looking in the git history (??).

Man I've had this exact thing happen recently with Sonnet 4.5 in Claude Code!

With Claude I asked it to try tweaking the font weight of a heading to put the finishing touches on a new page we were iterating on. Looked at it and said, "Never mind, undo that" and it nuked 45 minutes worth of work by running git restore.

It immediately realized it fucked up and started running all sorts of git commands and reading its own log trying to reverse what it did and then came back 5 minutes later saying "Welp I lost everything, do you want me to manually rebuild the entire page from our conversation history?

In my CLAUDE.md I have instructions to commit unstaged changes frequently but it often forgets and sure enough, it forgot this time too. I had it read its log and write a post-mortem of WTF led it to run dangerous git commands to remove one line of CSS and then used that to write more specific rules about using git in the project CLAUDE.md, and blocked it from running "git restore" at all.

We'll see if that did the trick but it was a good reminder that even "SOTA" models in 2025 can still go insane at the drop of a hat.

ewoodrich··on Google Antigravity
Could you give an example of what they're gatekeeping for Copilot exclusively? I'm kinda confused because Copilot in VS Code isn't exactly a powerhouse of unique features in my experience, it still feels well behind Roo/Kilo Code in most ways I can think of, although much closer to the competition than it was a year ago.
ewoodrich··on Google Antigravity
What exactly are the "extras" that Copilot gets?

I've had a Github Copilot subscription from work for 1yr+ and switch between the official Copilot and Roo/Kilo Code from time to time. The official Copilot extension has improved a lot in the last 3-6 months but I can't recall ever seeing Copilot do something that Roo/Kilo can't do, or am I missing something obvious?

ewoodrich··on Homebrew no longer allows bypassing Gatekeeper for unsigned/unnotarized software
Thanks! I'll give that a try.
ewoodrich··on Homebrew no longer allows bypassing Gatekeeper for unsigned/unnotarized software
I didn’t say it was invalid, just that it was needless. When I bought the laptop Gatekeeper was a tolerable nuisance and I was fine with the tradeoff given the security benefits.

The removal of the hotkey (which also required changing a setting before it worked at all) didn’t actually make it harder for a regular user to access, just 5x as aggravating every time it's necessary.

If they made developers go through some long and tedious process to re-enable it I would grumble but understand, but the only solution to get back to the 2024 status quo being entirely disabling a critical security feature certainly doesn't benefit me in any way.

ewoodrich··on Homebrew no longer allows bypassing Gatekeeper for unsigned/unnotarized software
I ran into this exact same thing recently with CSVs downloaded from my own app. I tried a few different filetypes and was baffled how seemingly any filetype I downloaded triggered Gatekeeper regardless of the app I set to open it (including stock apps).

I eventually found on Reddit that setting the default via the Get Info dialog was the only path that worked, so now I can click a CSV and open it in VS Code without needing to send Apple my passport and fingerprints. I keep seeing mixed opinions whether it's a bug that Get Info associations work differently vs the right click context menu, or if it's a deliberately obtuse garden path like the Settings/Open Anyway routine and "working" as intended.

Either way I hate it but it would be slightly more forgivable as a bug (assuming it was then fixed).

ewoodrich··on Homebrew no longer allows bypassing Gatekeeper for unsigned/unnotarized software
It really bothers me that Apple removed any convenient shortcut to bypass Gatekeeper like the old Control-click [1] hotkey. Apple's relentless ratcheting of the difficulty/annoyance of Gatekeeper has just about pushed me over the edge to completely disable it, despite the risk.

The ridiculous song and dance of "File is dangerous, delete it?"->No->Settings->Security->Open Anyway->"File is dangerous, delete it?"->No is getting ridiculously old after literally doing it a hundred times at this point. And soon enough Apple will inevitably come up with some additional hurdle like, idk, closing Settings three times in a row while reading a fingerprint during an odd numbered minute.

So in the name of "increased security" they've needlessly turned it into a binary thing where it's completely unprotected or accept my own computer that I paid for will deliberately waste my time constantly. It makes Windows 11 seem elegant in comparison where all I need to do is run Win11Debloat once on install and it gets out of my way.

[1] https://developer.apple.com/news/?id=saqachfa

ewoodrich··on GPT-5.1: A smarter, more conversational ChatGPT
I've only had that happen when I use /compact, so I just avoid compacting altogether on Codex/Claude. No great loss and I'm extremely skeptical anyway that the compacted summary will actually distill the specific actionable details I want.
ewoodrich··on The last-ever penny will be minted today in Philadelphia
Dispensaries in OR/WA love $2 bills, for some chains they're as unremarkable as a $1 and must special request them in bulk to keep on hand for making change.
ewoodrich··on AI documentation you can talk to, for every repo
> The text sections take implementation details that don't matter and present them to the user like they need to know them.

Yeah this seems to be a recurring issue on each of the repos I've tried. Some occasionally useful tables or diagrams buried in pages of distracting irrelevant slop.

ewoodrich··on Eating stinging nettles
Hah I misinterpreted it a different way as a kid, for a long time I thought it was like a collective delusion where the shared experience of contributing insubstantial garnishes to a pot of water tricked everyone into finding it filling and enjoying it.
ewoodrich··on Tell HN: X is opening any tweet link in a webview whether you press it or not
I set up a URL redirect rule in Edge/Brave/Chrome with the extension URL Auto Redirector (previously used Redirector but it was removed, there are other alternatives available for Firefox I'm sure). I also found a similar front end for Instagram but just added a rule yesterday so haven't tested it extensively yet.

I avoid most Twitter/X content after I deleted my account but it's helpful when it gets linked in HN.

  Source                           |  Destination
  -----------------------------------------------------------                                 
  ^https?://x.com/(.*)             |  https://xcancel.com/$1
  ^https?://twitter.com/(.*)       |  https://xcancel.com/$1
  ^https?://instagram.com/(.*)     |  https://imginn.com/$1
  ^https?://www.instagram.com/(.*) |  https://imginn.com/$1
URL Auto Redirector:

https://chromewebstore.google.com/detail/url-auto-redirector...

ewoodrich··on Claude Code can debug low-level cryptography
Also there is a fixed price Claude Code plan for GLM 4.6 from z.ai, I pay for the cheapest ($6/mo) as an alternate/fallback for Claude Code and Codex. I've been surprised by how similar in capabilities all three of them are, definitely not seeing a big CLI agent moat...

GLM is maybe slightly weaker on average but on the other hand it's also solved problems where both CC and Codex got stuck in endless failure loops so for the price it's nice to have in my back pocket. I also see some tool use failures sometimes that it always works around which I'm guessing are due to slight differences with Claude.

https://z.ai/subscribe

ewoodrich··on Claude Code can debug low-level cryptography
There is now also copilot-cli that's a clone of Claude Code and by default runs with Sonnet 4.5. I haven't spent much time with it on really complex stuff yet but it's a nice option to have available if you have a Copilot Business/Enterprise plan at work.

https://github.com/github/copilot-cli

ewoodrich··on Claude Code can debug low-level cryptography
The weak points raised by the parent comment are specifically examples where the problem exists outside the model's "peripheral vision" from its context window and speaking from personal experience, aren't as simple as as adding a line to the CLAUDE.md saying "do this / don't do this".

I agree that the popular "one shot at all costs / end the chat at the first whiff of a mistake" advice is much too reductive but unlike a colleague, after putting in all that effort into developing a shared mental model of the desired outcome you reach the max context and then all that nuanced understanding instantly evaporates. You then have to hope the lossy compression into text instructions will actually steer it where you want next time but from experience that unfortunately is far from certain.

ewoodrich··on ChatGPT's Atlas: The Browser That's Anti-Web
It was certainly jarring reading that immediately after accusing someone else of having never played the games.

(Who is clearly playfully poking fun at something he enjoyed playing but can recognize as being constrained by computers of the era and no longer a common format of games thanks to 3D rendering).

ewoodrich··on ChatGPT's Atlas: The Browser That's Anti-Web
It’s less rigid than a command line but much less predictable than either a CLI or a GUI, with the slightest variation in phrasing sometimes producing very different results even on the same model.

Particularly when you throw in agentic capabilities where it can feel like a roll of the dice if the LLM decides to use a special purpose tool or just wings it and spits out its probabilistic best guess.

ewoodrich··on Passkeys: They're not perfect but they're getting better
I'm struggling to think of a reason why being "treated as an always-idiot" is an actual negative in this specific example.

I use Bitwarden and when the password autofill doesn't work as expected my first assumption from many previous experiences is that it's because a website changed something slightly in their auth flow or a particular page has a weird redirect/embedded login scheme different than the primary login, or similar "modern" web weirdness.

So if I get phished and let my guard down just that one time due to panic, sleep deprivation, or whatever else I'm glad that it gives me a second layer of defense against me reflexively clicking a couple times to copy/paste the password manually. A passkey dropdown with "No passkeys saved for this site" would be a massive red flag and stop me in my tracks before trying to do something else stupid.

ewoodrich··on AI can code, but it can't build software
> aren’t yet seeming to train on how to build maintainable software.

A while ago I discovered that Claude, left to its own devices, has been doing the LLM equivalent of Ctrl-C/Ctrl-V for almost every component it's created in an ever growing .NET/React/Typescript side project for months on end.

It was legitimately baffling seeing the degree to which it had avoided reusing literally any shared code in favor of updating the exact same thing in 19 places every time a color needed to be tweaked or something. The craziest example was a pretty central dashboard view with navigation tabs in a sidebar where it had been maintaining two almost identical implementations just to display a slightly different tab structure for logged in vs logged out users.

I've now been directing it to de-spaghetti things when I spot good opportunities and added more best practices to CLAUDE.md (with mixed results) so things are gradually getting more manageable, but it really shook my confidence in its ability to architect, well, anything on its own without micromanagement.

ewoodrich··on What happened to Apple's legendary attention to detail?
MacOS and iOS both seem to have an insatiable hunger for passwords. The most aggravating scenario for me by far is when the App Store on iOS, with no consistent pattern I have been able to identify, makes me reenter my entire massive Apple ID password instead of the usual Face ID prompt to download ... a free app.
ewoodrich··on Accessing Max Verstappen's passport and PII through FIA bugs
Variant of this I've hit is the phone number validation rules at signup differs from the actual API call to send 2FA texts (or was changed between the time of original signup and login attempt) so I create an account successfully with a Google Voice number and then when I actually need to receive 2FA the message goes into the aether with no error surfaced at any point.
ewoodrich··on LLMs can get "brain rot"
Perhaps because it didn't stick out like a sore thumb? Or because it became so prevalent they observe the exact same tics in every other article they read nowadays?
← PreviousPage 4 of 24Next →