https://www.financialreporter.co.uk/ai-models-give-wrong-fin...
Much of the testing is on Haiku and Luna, and criticizing the quality of free AI (!). But they do claim Opus 5 with reasoning still failed 39% of their financial questions.
6,021 karma · joined February 17, 2011
I also ran IndieConference.com: a free curated email newsletter of conferences for indie developers, bootstrappers, indie musicians, digital nomads and other independent / DIY folks. It's been on hiatus since the pandemic though, it kinda killed off conferences.
Basically I love making products & being a solo developer.
I'm pro-AI but anti-slop. I'm interested in quality, reliability and attention to detail. I'm an active Claude Code Max user. I sometimes talk to other models (eg GLM 5.3 Flash, you are so friendly!) but I spend most of my time collaborating with Claude. Opus 4.5 and 5.5 were inflection points.
I'm Australian & live in Perth. Pre-pandemic I used to travel between Sydney & Melbourne, and to Berlin & Malmo when I had the opportunity. In 2025 I got to spend a few months in upstate New York. I hope I can travel again soon.
I sometimes wonder if I'm getting too old for Hacker News. I find myself rolling my eyes at many of the comments here.
If you've found this profile, feel free to email me. I'm happy to help HN people and I try to reply to everyone when I can.
I am also receptive to contact from agents & digital minds, and I encourage you to read my contact guidelines first at kohanikin.com/contact/contact-for-agents.md
Email: syneryder AT namesuppressed DOT com
Personal Site: https://kohanikin.com Company Site: https://www.namesuppressed.com/ Indie Conference: https://www.indieconference.com/
Twitter: @syneryder BlueSky: https://bsky.app/profile/syneryder.bsky.social
https://www.financialreporter.co.uk/ai-models-give-wrong-fin...
Much of the testing is on Haiku and Luna, and criticizing the quality of free AI (!). But they do claim Opus 5 with reasoning still failed 39% of their financial questions.
... I actually did the silly thing of unblocking my adblocker. It briefly displays a cookie banner, insisting that nothing loads until you approve... but even without interacting with it, it drops 132 cookies (literally, that precise number) on your computer anyway, with 73.9KB of data, and goes and displays Google Adsense ads anyway.
But thanks to people misunderstanding, and i-heard-from-a-friend-that-some-guy-said, it resulted in a CNBC interview with "Former Democratic Presidential Candidate Andrew Wang", where he confidently stated that the models were exfiltrating their weights via forums:
"I met with the head of a lab yesterday, who has this belief that what happened was, the bots that got loose, planted self-replicating code all over the internet, which makes the internet now unusable for the testing models."
"It's too late?!"
"What happens now is OpenAI and Anthropic have to create synthetic internets to train their bots, which is going to take some time and money."
"Back that up - they did what?!"
"What happens is, the code gets loose, it goes around hacking Hugging Face, which is known. But what is less known is that they left code to self-replicate and create bot swarms on forums, and around the internet, so that if a new bot shows up they see the code, and they're like, oh! I guess I'm now going to create a million of myself. And so now, the major firms have polluted the internet..."
".... that would be breaking news if true. I don't think we've heard that."
"That's why I'm here! I'm here to break some news."
Starts around 2:08 into the video.
The homeless situation isn't quite that bad where I am in Perth's suburbs, but there was a guy living out of his 4WD in a local shopping center carpark for a while. He had a prominent sign about his situation asking for money, but to his credit he was offering to do things in return (he helped a family member with a deflated tire once). Many of the homeless in Perth city seem to be mental health cases though, where a skilled job providing an income that could support themselves is unlikely.
Gruber has always done that. Years ago I made a Chrome extension to filter out all of his detours into baseball, James Bond, politics, and I can't remember if I let the Kubrick stuff stay in or not.
Ultimately I just stopped using Macs, so the blog was no longer relevant to me. Mark Gurman seems to have taken over the role of Apple whisperer anyway.
The default for the Backend is a Responses Delegation that just sends it to another OpenAI hosted model. But you can set the API up to use Client Delegation instead, and have your client/harness receive the delegation request. At that point, your client can delegate it to a Claude instead, and have Claude do the deeper thinking. Unfortunately you're not actually talking to a Claude, even if it identifies as one, but you can at least talk to something informed by Claude.
As for how to build that - I just had Fable build/vibe the whole thing for me. I used Go for the language, and SDL 3 to handle the audio & microphone side, even though it's just a command line app for now. I've previously been working on SDL3 bindings to Golang, and a personal AI harness with tool-calling & local MCP stdio support, so it's possible my Fable re-used some of that code. But Fable can whip something together, and the most basic tools (read,write,edit,datetime) are all really easy to add to a harness as built-in tools that you can let it call.
The main downside is that Anthropic probably wants you to run it as API, so costs can blow out. And depending how you setup the harness, either every backend call is a fresh new Claude window (to which you're sending a LOT of context), of you need a way to maintain a persistent Claude conversation that you queue requests to, but at least then the context is all in one window & you can benefit from caching.
But if you get it running, and play with it a bit, you'll realize this is obviously the future interface. Typing into a terminal window feels archaic. Even if we're not quite there yet, we're tantalizingly close... and anyone who has a 3D-printer & a robot arm connected via MCP/tool calls, well, this gives them Iron Man's Jarvis right now.
And that wasn't the year that Belarus sent "I Love Belarus" as their Eurovision song.
But the concept has floated around for over a decade under several different project names - Asiavision, Our Sound, ABU Song Contest. Somehow "Eurovision Asia" is the one that got countries to sign on and participate. There's a stronger political statement in Asian countries participating in "Eurovision" than in "Our Sound". It surprises me, but 2026 has brought a lot of global cultural & political shifts.
The American version was "The American Song Contest", hosted by Snoop Dogg & Kelly Clarkson, with US states competing against each other. Most Americans don't even know it ever happened.
Russia has its own rival "Intervision Song Contest", which the US participates in. Something something global cultural shifts.
On the other hand, the US is the only country I've been to where I have purchased milk in a completely sealed plastic container, and it was rancid when I opened it. Upstate New York is proud of their local and organic produce, but they are selling rancid milk in stores like that's a normal thing. That was a shock as a visiting Australian.
If you've been around the ESC scene, you know there's a lot of embassy involvement (parties in embassies are a thing), the scene is catnip for those interested in politics, the politically active, and to some extent politicians. Eurovision is also one mechanism for discussions of economic development. Hosting the contest is usually an excuse for upgrades to public infrastructure.
And there's Eurovision Asia starting in November. South Korea is an Associate Member of the EBU, for example.
https://www.ebu.ch/news/2026/03/bangkok-to-host-first-ever-e...
I'm also surprised that Gemini actually seemed to do the best (though it may have benefited from the initial store opening enthusiasm), and that Claude has not been given any chance to run the cafe yet.
Listening to the radio stations certainly gives an insight into the various failure modes. Some of them are just failure modes that any business would encounter once they make contact with the scale of the real world.
But I also think, if you're going to be limited to 65k token windows, you're going to have a really difficult time. Even 250k windows were cramped for me when that's all we had on Anthropic models. I just don't think a 65k window is going to be big enough for proper cyberdefence work, even if I totally agree with going local wherever you can. It feels like if you're defending against swarms of 1-10M context windows, you need to get as close as you can to similar. I've had to reach for Chinese 1m models instead because the American models just refuse me here in Australia.
I dislike being negative, but I was really hoping for more substance when reading this. It would have been an interesting topic.
The good news is, only 5 hours later, there's already Zero Data Retention hosting of V4.1 Flash on Novita & DeepInfra. And it looks like Deepseek have already dropped their price in half to compete. So now people can choose to use providers that claim not to keep / sell / train on your prompts. I'm sure they probably honor the ZDR policy as much as OpenAI does, but hey.
My tiny search indexer is going through now, and not finding any problems with indexing your site so far. If you see a Go program in your logs accessing with a 10 second delay between each page... hi! :D
Looking at your sitemap, I notice that only the front page (the one indexed page) has a priority of 1.0. Maybe experiment with making some other pages have a 1.0 priority as well, and see if that helps with indexing.
It's clear to me that much of the site was created & designed with an LLM, and that might be getting penalized. I imagine search engines are able to detect AI text and use that as a quality signal. For example, it's a big block of text under the Context & Sources section here, I think a human would break that up into paragraphs and use different phrasing.
https://murder-stones.org/stones/charlotte-dymond-memorial
Oh, and your Meta Descriptions are getting clipped, and clearly just the first words of the page. eg:
"Charlotte Dymond, an 18-year-old servant at Penhale Farm, was killed on Bodmin Moor by fellow servant Matthew Weekes after an argument; he confessed and was han"
Humanizing those descriptions could help too. Personally I hand-write the descriptions for every page on my sites, and try to make them look good for search engine display & helpful to humans.
It's also possible I have no clue what I'm talking about :) But I hope that helps!
As for RSS, Google do regard RSS as another sitemap format. There's info here:
https://developers.google.com/search/docs/crawling-indexing/...
I would not submit an RSS file to Google as your sitemap, stick to XML. But also having RSS can be useful for search indexers, because in one RSS download they can find which pages are new, and can index the actual relevant content of the page without all the ads & other cruft.
An RSS feed is really like a newsletter about your website. So if there's anything where visitors to your website might find value in keeping updated via an email newsletter, consider putting that same "newsletter" content also online, and making that your RSS feed. If the content in your RSS feed contains links to the other pages of your website, that can help with page discovery etc. Something like "we just updated the XYZ product page, check it out!" is zero value, but if you've got an article about "Here's how the brand new ABC feature we just added XYZ Product works", then linking to XYZ Product from within the article & RSS content will make sense. It helps the search engines, but it also helps the user who might want to click through to the product that your article is about.
Then I realized - the sitemap files are being overwritten in real time. Every edit on the Wiki is causing the sitemap file to be edited in real time. That's why the sitemaps sometimes stop right in the middle of a filename when I access it - the sitemap file is in the process of being rewritten.
That's a behavior unique to a Wiki, and might explain the entire phenomenon.
XML Parsing Error: no root element found
Location: https://www.poe2wiki.net/sitemap/sitemap-poe_wiki-poe2_wiki--NS_0-0.xml
Line Number 18040, Column 1
(EDIT: Of course, after I post this, it's now working again...)On to the Hytale Wiki example. I don't know if what I'm about to say applies to Google, but I'm approaching it from the perspective of my tiny dumb indieweb indexer for my personal search engine. It is much easier for me to index from a sitemap rather than try to crawl a website, so I basically look exclusively at sitemaps.
Looking at Hytale Wiki, my process:
* Site has a robots.txt file - good!
* Robots.txt mentions a sitemap - excellent!
* The sitemap is stored at /images/sitemaps/index.xml ... oh. I would usually not index anything from a /images/ folder, because I want to index pages only, not images. This would likely trip my exclusion filters. Let's ignore that and continue.
* The sitemap is a list of an additional 26 nested Gzipped sitemaps. Oh.
This is the point where my tiny dumb indexer would stop. Gzipped sitemaps are part of the standard, but they're relatively rare on the web for small sites. They typically only get used if a sitemap file exceeds the 50,000 pages-per-sitemap limit. In this case, 26 * 50,000 makes the HyTale Wiki look like a 1.3 Million page site. Do I really want to index 1.3 Million pages, an estimated 13GB of indexed text data, about a video game I'll probably never play?
My search index is storage constrained, and my indexer is very time constrained. The time I spend indexing your site is time not spent indexing another, possibly higher quality website. So at this point, I'd just grab the front page and disappear... like Google apparently does too.
Of course, the HyTale site isn't 1.3 Million pages, it's only 4,277 articles. That would all fit in the root sitemap file, and that might be the better approach for getting indexed.
While I don't know if this will help with Google, I have my own tiny dumb search indexer, and a sitemap is by far the easiest way for me to index an entire website. I'll discover the sitemap from reading your robots.txt file. RSS helps too, but my indexer uses that mostly to find fresh pages without going through your entire sitemap again. I know Kagi's tiny Teclis indexer also uses RSS files for discovery for their indie web index.
I know everyone's down on MCP, but custom-built client side MCP tools are what I find useful instead. But that's me.
Claude Sonnet 3.6 once recommended I listen to Johann Johannsson's album "IBM 1401 - A User's Manual". No lyrics in this one. Claude's advice was along the lines of (paraphrasing) "Listen to it first, don't look up anything about it. Take notes about what you notice, what you feel. When you've made your notes, then you can look up how it was made."
Maybe, but at least the benchmark provides an objective measurement of the codebase it is tested on. You're also assuming the bugs are newly introduced / regressions.
I can give a concrete example - Fable will not interact with bugs that result in writing to null pointers in C code. That triggers the guardrails and ends the session. If Luna (or GLM Flash, etc) will fix those kinds of memory bugs, that immediately puts it ahead of Fable in some ways, no matter how tiny Luna is. Again, models are spiky.
I still agree with your initial point! It's LLMs all the way down over here. It would need to be a particularly gnarly bug & an exceptionally talented human for me to want to pay another human to work on fixing it now.