paradite

I recently did a livestream on trying to understand attention mechanism (K, Q, V) in LLM.

I think it went pretty well (was able to understand most of the logic and maths), and I touched on some of these terms.

https://youtube.com/live/vaJ5WRLZ0RE?feature=share

I've came to the pain conclusion on Next.js:

It's a bad framework for building full stack apps, but it's better than anything else.

https://x.com/paradite_/status/1941016421934551477

Claude Sonnet 4 is the best coding model. Period. Nothing else comes close.

Anthropic probably has 80% of AI coding model market share. That's a trillion dollar market.

There is automatic code indexing from Cursor.

Autocomple is also automatically triggered when you place your cursor inside the code.

Claude models have been weaker in vision tasks compared to models from OpenAI and Google.

https://eval.16x.engineer/evals/image-analysis

For them to roll out a browser extension must mean that they have found a walkaround or alternative method to solve the vision performance.

It won't work because of too many false positives. People are already trained to ignore warnings, like how they blindly accept T&C without reading.

Hey. I like your roast on benchmarks.

I also publish my own evals on new models (using coding tasks that I curated myself, without tools, rated by human with rubrics). Would love you to check out and give your thoughts:

Example recent one on GPT-5:

https://eval.16x.engineer/blog/gpt-5-coding-evaluation-under...

All results:

https://eval.16x.engineer/evals/coding

It's pay wall for me.

JSR does that? Now that might be a good reason to move my packages over to get rid of tsup.

Is this from Moonshot AI (company behind the Kimi K2), or a 3rd party?

Judging from the design, I assume it's not officially related to the model.

The performance not only depends on the tool, it also depends on the model, and the codebase you are working on (context), and the task given (prompt).

And all these factors are not independent. Some combinations work better than others. For example:

- Claude Sonnet 4 might work well with feature implementation, on backend code python code using Claude Code.

- Gemini 2.5 Pro works better for big fixes on frontend react codebases.

...

So you can't just test the tools alone and keep everything else constant. Instead you get a combinatorial explosion of tool * model * context * prompt to test.

16x Eval can tackle parts of the problem, but it doesn't cover factors like tools yet.

https://eval.16x.engineer/

What kind of questions / domains were you encountering false information on?

It's actually more complex than just input and output tokens, there are more pricing rules by various providers:

- Off-peak pricing by DeepSeek

- Batch pricing by OpenAI and Anthropic

- Context window differentiated pricing by Google and Grok

- Thinking vs non-thinking token pricing by Qwen

- Input token tiered pricing by Qwen coder

I originally posted here: https://x.com/paradite_/status/1947932450212221427

I believe everyone should run their own evals on their own tasks or use cases.

Shameless plug, but I made a simple app for anyone to create their own evals locally:

https://eval.16x.engineer/

I am not pro or against AI-generated posts. I was just making an observation and testing my AI classifier.

"Hard truth" and "reality check" in the same post is dead giveaway.

I read and generate hundreds of posts every month. I have to read books on writing to keep myself sane and not sound like an AI.

This is obviously AI generated, if that matters.

And I have an AI workflow that generates much better posts than this.

Agents have been a field in AI long since 1990s.

MDP, Q learning, TD, RL, PPO are basically all about agent.

What we have today is still very much the same field as it was.

This is an incredibly fascinating read into how OpenAI works.

Some of the details seem rather sensitive to me.

I'm not sure if the essay is going to stay up for long, given how "secretive" OpenAI is claimed to be.

I think you are confusing "I don't like it" with "It's not going to happen".

Just because you don't like it, it doesn't mean it's not going to happen.

Observe the world without prejudice. Think rationally without prejudice.

As dang said, presume good faith. It's part of the HN guideline.

Also, "Never attribute to malice that which is adequately explained by stupidity"

I chanced up some endpoint that mentioned "makersuite" that seems to be what's behind genai.

It's interesting to see how one team / product gets rebranded to another in Google.

"To continue, please install the Microsoft 365 Copilot app"

I got this on mobile. Seems to be pretty apt.

The results are not surprising, but it's good to have these findings formalized as publications, so that we (or LLMs) can refer to them as ground truth in the future.

I hate things that are translucent. I find them very distracting, and hurt my eyes.

I hope Apple gives the option to turn this whole thing off.

I notice the borders now also have shadows / gradients due to reflection, that's also something I'd like to remove personally.

It can open PR via GitHub actions integration. I just did:

https://x.com/paradite_/status/1931644656762429503

Docs: https://docs.anthropic.com/en/docs/claude-code/github-action...

Okay maybe I need to make myself more clear, and start from your claim:

> It has to do with mental capacities to "bring objects under a concept", partition experience into its conceptual structure ("conceptualise"), simulate scenarios based on compositions of concepts ("the imagination") and so on. These are mental capabilities a wide class of animals possess, who know no language; that LLMs do not possess.

Assume it is true that humans are capable of these capabilities, why do you think LLMs are not capable of them? We don't know if they are capable of these capabilities, and that's what explainable ai is.

Take a step back and assume that LLMs are not capable of these capabilities, how do you prove that these are the fundamental concepts in the universe, instead of projections of higher level concepts from higher dimensional space that humans and animals are not aware of? What if there exist a more universal set of concepts that contains all the concepts we know and others, in a higher dimension, and both LLMs and humans are just using the lower dimension projections of such concepts?

You have biased view on the definition of "concept" based on English language and logic.

In Chinese language, concept is 概念.

In Chinese language, happy dog is 快乐的狗.

Notice it has an extra "的" that is missing in English language. This tells you that you can't just treat English grammar and structure as the formal definition of "concept". Some languages do not have words for happiness, or dog. But that doesn't mean the concept of happiness or dog does not exist.

The reverse is also true, you can't claim a concept does not exist if it does not exist in English language. Concept is something beyond any particular language or logical construct or notations that you invent.

You seem to believe "concept" is a concept in English language. It is not.

Concept is an abstraction layer above human languages.

Here's a good article that touched on this topic: https://www.neelnanda.io/mechanistic-interpretability/glossa...

> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window. Only humans can can gain and retain program theory.

False.

> An LLM is a token predictor. It works only at the level of text. It is not capable of working at a conceptual level: it doesn't reason about ideas, diagrams, or requirements specifications.

False.

Anyone who have spent time in machine learning or reinforcement learning understands that models are projections of higher dimension concepts on to lower dimensions as weights.