Hacker News Reader

Tried it out, asking it for a broad suite of real dev work:

- UI component polish on a React FE

- General review of a complex cross-cutting Node backend changes

- Minor changes to a Rust server with reviews by Codex

Well, my first impression is that it's still a step below Astra/Opus5.5/Fable here. Intelligence is there I think, but 'taste' and 'judgment' is not there yet. Personality seems okay but that doesn't help when it got stuck into a loop by being fixated on the review comments (on iteration 7 of review->fix last I checked) without taking a step back and reconsidering, and this is on a relatively simple change.

Limits seems okay-ish on the $50 plan but not too generous I think.

ekojson GPT-6 Astra makes major gains in the Artificial Analysis Coding Agent Index

Well, seems like ECI [0] and the AA index is diverging quite a bit. Benchmarking LLM is tough and I think we are seeing the limitations of current benchmarks and applicability to real tasks.

[0]: https://x.com/EpochAIResearch/status/2095602754282783108

ekojson Kimi K3: Open Frontier Intelligence

I copied that from https://platform.kimi.ai/docs/guide/kimi-k3-quickstart but it seems they updated the page to remove the benchmark score now.

ekojson Kimi K3: Open Frontier Intelligence

Yeah, bad wording it seems. Though a charitable interpretation is that Fable 5 and GPT 5.6 Sol are joint 1st place in the measurement.

ekojson Kimi K3: Open Frontier Intelligence

> In our evaluations, Kimi K3 delivers frontier-level performance. Among the models tested, its overall intelligence ranks second only to Claude Fable 5 and GPT-5.6 Sol. For the complete benchmark results, see our tech blog. The full model weights of Kimi K3 will be released in the coming days. More details on the architecture, training, and evaluation will be published together with the Kimi K3 technical report.

> K3 pushes the boundary of end-to-end knowledge work. On the GDPval-AA v2 leaderboard, Kimi K3 scores 1687. The benchmark evaluates AI models on real-world tasks across 44 occupations and 9 major industries; Kimi K3 ranks behind only Claude Fable 5 Max and GPT-5.6 Sol Max, and ahead of Claude Opus 4.8 Max at 1600.

> On AA-Briefcase, Kimi K3 scores 1527, ranking second among all models — behind only Claude Fable 5 Max and ahead of GPT-5.6 Sol Max (1495). AA-Briefcase is a private agentic knowledge-work benchmark developed by Artificial Analysis to evaluate frontier agentic capability in long-horizon knowledge work.

Really good benchmark score it seems. Maybe another DeepSeek moment right here.

ekojson Gemini 3.5 Flash

Seems like the only good thing about 3.5 Flash is its speed. Not cost-competitive or benchmark-leading by any means.

ekojson Apple Silicon costs more than OpenRouter

Not disagreeing with your argument, but:

> If you want a good dense model, use qwen3.6 27B instead, speed will be up, and if you don't take my word for it being smarter, take openrouter's prices of it against the bigger, slower and less memory-efficient gemma do the talking.

Don't know if this is the correct read. I think those providers are simply taking cue from Alibaba's first-party pricing for the 27B Dense. It's kinda overpriced imo. Perhaps it can be explained by how 'reasoning-inefficient' (relative to frontier models or even Gemma) the Qwen models are and longer sequence lengths are expensive to serve.

ekojson How to make SSE token streams resumable, cancellable, and multi-device

> HTTP is just not a good transport for streaming LLM tokens and for building async agentic applications

I don't know if I agree if this is a problem with SSE or HTTP. Something like a Redis Streams-backed SSE would solve most of the 'challenges' presented in the post.

ekojson Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

> You cannot run these models at 8-bit on a 32GB card because you need space for context

You probably can actually. Not saying that it would be ideal but it can fit entirely in VRAM (if you make sure to quantize the attention layers). KV cache quantization and not loading the vision tower would help quite a bit. Not ideal for long context, but it should be very much possible.

I addressed the lossless claim in another reply but I guess it really depends on what the model is used for. For my usecases, it's nearly lossless I'd say.

ekojson Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Yeah, figure the 'nearly lossless' claim is the most controversial thing. But in my defense, ~97% recovery in benchmarks is what I consider 'nearly lossless'. When quantized with calibration data for a specialized domain, the difference in my internal benchmark is pretty much indistinguishable. But for agentic work, 4-bit quants can indeed fall a bit short in long-context usecase, especially if you quantize the attention layers.

ekojson Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Not at all, I actually run ~30B dense models for production and have tested out 5090/3090 for that. There are gotchas of course, but the speed/quality claims should be roughly there.

ekojson Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

As this is a dense model and it's pretty sizable, 4-bit quantization can be nearly lossless. With that, you can run this on a 3090/4090/5090. You can probably even go FP8 with 5090 (though there will be tradeoffs). Probably ~70 tok/s on a 5090 and roughly half that on a 4090/3090. With speculative decoding, you can get even faster (2-3x I'd say). Pretty amazing what you can get locally.

ekojson Gemini API Billing Bug Causing Erroneous Charge for 'Image Generation'

Seems pretty widespread. We got mistakenly charged for ~$800 over the weekend.

Other Sources:

[0]: https://aistudio.google.com/status

[1]: https://www.reddit.com/r/GeminiAI/comments/1mycmtk/google_cl...

[2]: https://www.reddit.com/r/GeminiAI/comments/1myg04q/gemini_25...

ekojson Gemini with Deep Think achieves gold-medal standard at the IMO

> Btw as an aside, we didn’t announce on Friday because we respected the IMO Board's original request that all AI labs share their results only after the official results had been verified by independent experts & the students had rightly received the acclamation they deserved

> We've now been given permission to share our results and are pleased to have been part of the inaugural cohort to have our model results officially graded and certified by IMO coordinators and experts, receiving the first official gold-level performance grading for an AI system!

From https://x.com/demishassabis/status/1947337620226240803

Was OpenAI simply not coordinating with the IMO Board then?

ekojson How I Use Kagi

Maybe not a popular sentiment here on HN but I cancelled my Kagi subscription (9+ months) just recently. Increasingly, most of my queries/search have been through LLMs and Google search is just fine (and even better for restaurants, places, and the like). I don't think the improved search experience is worth the subscription anymore.

ekojson GCP Outage

https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

> Multiple GCP products are experiencing impact due to Identity and Access Management Service Issue

IAM issue huh. The post-mortem should be interesting at least.

ekojson GCP Outage

Super duper frustrating having the status page being green. Why can't Google do this properly?

ekojson Next.js 15.1 is unusable outside of Vercel

I share the sentiment. I think we will only be using Next.js for static sites/prebuilt SPA in the future.

ekojson Meta got caught gaming AI benchmarks

I think it's most illustrative to see the sample battles (H2H) that LMArena released [1]. The outputs of Meta's model is too verbose and too 'yappy' IMO. And looking at the verdicts, it's no wonder by people are discounting LMArena rankings.

[1]: https://huggingface.co/spaces/lmarena-ai/Llama-4-Maverick-03...

ekojson Gemini 2.5

> This will mark the first experimental model with higher rate limits + billing. Excited for this to land and for folks to really put the model through the paces!

From https://x.com/OfficialLoganK/status/1904583353954882046

The low rate-limit really hampered my usage of 2.0 Pro and the like. Interesting to see how this plays out.

ekojson Fine-tune Google's Gemma 3

> The bottleneck then becomes how to self-host the finetuned model in a way that's cost-effective and scalable

It's not actually that expensive and hard. For narrow usecases, you can produce 4-bit quantized fine-tunes that perform as well as the full model. Hosting the 4-bit quantized version can be done on relatively low cost. You can use A40 or RTX 3090 on Runpod for ~$300/month.

ekojson How much traffic can a pre-rendered Next.js site handle?

Normally, yes. But there's a couple rendering modes with these frameworks. In this case, the rendering is most likely 'hybrid'. Some routes are statically pre-rendered, some are served via SSR. You'd need a JS server for the SSR ofc.

ekojson How much traffic can a pre-rendered Next.js site handle?

Interesting. My hunch is that Next.js is not optimized for the dockerized Node server deployment. I would say that you could get much greater prerendering performance from Next.js by just fronting the assets directly using Caddy/Nginx.

ekojson GPT-4.5

> Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models.

Seems like it's not going to be deployed for long.

$75.00 / 1M tokens for input

$150.00 / 1M tokens for output

That's crazy prices.

ekojson How streaming LLM APIs work

I do actually wonder if it's more efficient to use something like MessagePack instead of using JSON. It's a lot of strings so it may not matter too much I guess.

ekojson How streaming LLM APIs work

EventSource is really really limited. However, you can instead use Fetch via something like https://github.com/Azure/fetch-event-source to consume SSEs.

ekojson Qwen2.5: A Party of Foundation Models

Actually really impressive. They went up from 7T tokens to 18T tokens. Curious to see how they perform after finetuning.

ekojson Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI

Congrats on the launch!

We're definitely looking for something like this as we're looking to transition from Azure's (expensive) GPUs. I'm curious how you stack against something like Runpod's serverless offering (which seems quite a bit cheaper). Do you offer faster cold starts? How long would a ~30GB model load takes?

ekojson Direct Nash Optimization: Teaching language models to self-improve

While the math seems intimidating, it does not look all too different from SPIN and previous researches. Pretty surprising how effective this is though. The costs here seems to be way higher too (with all the GPT4 calls).

> We also do a brief cost analysis associated with the scaled-up experiment on 600k training inputs. The major line items are the cost of sampling outputs, annotating them with GPT-4 to construct training pairs, and then training the next iteration against those pairs. For _each_ of the six iterations:

> 1. Sampling: it took about 18-24 hours to inference 5 outputs for all 100k examples on 10 8xA100 80GB pods, depending on the average length, costing about $6,000 based on spot pricing.

> 2. Annotation: the average number of prompt tokens sent to GPT-4 for annotation across iterations was about 450M, with an average of about 60M completion tokens, amounting to about $34,000 based on the version of the endpoint we were using.

> 3. Training: ironically, training was the cheapest step, taking only 12-24 hours on two 8xA100 80GB nodes

ekojson Rethinking serverless with FLAME

I don't know if I agree with the argument regarding durability vs elastic execution. If I can get both (with a nice API/DX) via something like Temporal (https://github.com/temporalio/temporal), what's the drawback here?