hbrn

I have a bridge to sell you.

> basically everyone asks for a JSON schema for the tool call and uses that to validate and re-prompt the LLM until it emits something with a valid schema.

Basically no one is doing that since 2024, have you been living under the rock? Read about constrained decoding.

> I have had failures when I want a field to be specifically formatted Markdown or something.

And Jev has an advantage here because it can’t generate anything?

You gotta admit that turning "our model is very dumb and can only make simple decisions" into "System One Model" is a pretty clever marketing move.

Yet another absolutely fake demo where Jev is flipping the coin and harness is doing 99% of the work.

The harness reads the ROM/RAM, builds a collision grid, computes reachable paths with A-star, constructs a region graph, calculates distance to the current objective, determines whether an exit leads toward or away from the objective, identifies Pokémon Centers and Marts, detects NPCs and items, and then turns all of that into natural-language choices for Jev.

Jev is given a list of "choices", already annotated information that tells Jev whether the choice should be taken or not.

For example, Jev might receive options like:

  * Enter ROCK_TUNNEL_1F — Leads toward the objective (2 areas away)
  * Go west to ROUTE_9 — Leads away from the objective (5 areas away)
  * Talk to NPC X — Mentioned in the current objective
  * Enter POKECENTER — toward nearest Pokémon Center

Those "toward the objective" judgments are not Jev figuring out the map. The harness computes shortest path distances through its region graph and literally annotates the choices with that answer.

Once Jev says "Enter X" the harness already has the exact A* path and walks it automatically, including replanning around moving NPCs and retrying when blocked.

Puzzle solving is also done by the harness - it gives Jev a set of choices, annotated with sentences like "brings the boulder closer to the floor switch", "after this push no sequence of pushes can bring this boulder onto a floor switch anymore".

Story progression contains hardcoded instructions for Jev, e.g.: 'Go south from Cerulean through Route 5, the Underground Path and Route 6 to Vermilion City; board the S.S. Anne at the dock and get HM01 (Cut) from the captain.'

And even with all of those hardcoded decisions, it still plays terrible. I just saw it get stuck in a loop - but of course harness annotates repeated actions, so eventually Jev is steered to escape the loop.

You should be ashamed of yourself.

That's really funny, nice find.

To be fair, he's just asking how to get customers. And the post is 2 days before Jev's launch date? I don't think he's trying to sell Laya there (though he probably will at this point).

I think you're right about Laya (and confirmation bias).

But like you said, at the end of the day he's just a rando.

He's not asking for $40m, not saying "I made ChatGPT, but i hate it, so I built the next big thing". Not claiming to co-invent RLHF.

Laya is just noise. Jev's bullshit affects me today - I see people injecting it into the codebases where it has no place.

My understanding is that Laya (or whatever it was called in 2025) was yet another fine-tuned classifier, not a general purpose one.

That said, Typesafe false marketing caused Laya to fit perfectly into pretty much every advantage that they are claiming: "system one decision model", cheap, fast, no hallucinations, structured, confidence output, parallel, calibrated. Their BS is their own demise.

I think Laya's author genuinely bought their BS and thinks he built the same thing. Unlike Typesafe, I don't think he's intentionally misleading people.

The only unique thing about Jev is that it's a general purpose classifier. Funny enough, they were so busy spreading marketing bullshit that they forgot to mention the only real thing that makes Jev unique.

"Decision model" is just marketing jargon.

decision model = classifier

system one model = small non-reasoning LLM

noul = boolean

confidence = f(probabilities)

It's sad to see how gullible engineers are today.

I suspect that it's as handy as a coin flip.

And if I'm being charitable to Jev (which is nearly impossible at this point), detecting whether blob of text is AI generated is not a "system one" question.

But we still don't have a fast and accurate classifier.

All we have is a company that claims to have created one, with no proof.

So if benchmark says Jev sucks, it's benchmark's fault. And yet Typesafe doesn't share anything about their internal benchmark that apparently proves how amazing Jev is.

Funny enough, when I tell people that Jev cannot play tic-tac-toe I hear a similar argument - it's not what Jev was built for. Jev has this elusive use case that noone can describe, so when Jev fails, it's just because it wasn't built for it. Convenient.

Don't you find it suspicious that Jev cannot play tic-tac-toe or checkers, but can play Doom? Don't you find it suspicious that nothing of the Doom demo was shared: no harness, no control loop, no state encoding, no prompts - nothing.

And the explanation is obvious - Jev isn't playing doom. The harness is. They essentially built a Doom bot, dumbed down it's control loop for Jev, and gave reins to Jev. Look mom, Jev is playing Doom!

The state says "you're pointing at the cacodemon" or "you're not pointing at the cacodemon" and Jev has to decide whether to press fire. Frontier intelligence!

You can build a harness where a coin flip is playing Doom.

$40m in funding, 2 years in stealth.

Performs on-par with SemIf which was built in a couple days and apparently uses raw Qwen, with no fine-tuning. SemIf runs in your freaking browser. Oh and Jev is twice as expensive?

Is it surprising that Jev consistently thinks it's Qwen?

I'm almost convinced that Jev is a scam. Take Qwen, fine tune it a little, tell investors it cost $10m, spend $1m on advertising, profit.

I think opencode subscription issue is just a different marketing strategy. Neither company wants it, but OpenAI believes it's worth it as a marketing expense in the long run.

And Dario's "AI will kill us all" is the same as Sam's "AI will discover ALL science and we'll be building Dyson spheres".

Different flavors of the same BS.

Plenty of examples online, e.g.

https://github.com/kyle-pena-nlp/jevchat

https://www.reddit.com/r/LLM/comments/1winnju/jev_the_new_ai...

Despite Typesafe claims that Jev is not an LLM, it obviously is.

I heard that Jev is amazing at classifying people who understand ML, and people who don't.

> It's output probabilities are calibrated to actually mean something

Don't fall for marketing BS so easily.

Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.

I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.

The big problem is that Jev is only useful when you both:

- need fast response

- can tolerate Jev's mediocrity compared to real frontier models

(I explicitly ignore cost, because if you desperately need a cost-optimized classifier you would just build one)

Outside of fun demos these two rarely come together: if it's critical enough to require sub second speed, then it can't be mediocre.

The reason so many OSS models are being built is that Typesafe team made a ton bombastic claims about Jev being a huge breakthrough, and ML folks are realizing they can build Jev-like model in 2 days instead of 2 years.

Why would you use Jev over Sol/Sonnet though?

Speed/cost benefits of Jev are mostly useless for triage.

Feels like you would be using a dumber model to achieve worse results.

If all you have is a Jev, everything looks like it can be solved with a classifier?

I really don’t know what we’re arguing about here. You can google or ask your favorite LLM “who invented RLHF?”.

Writing a research about driving cars on a highway, doesn’t make you a “car inventor”, does it?

Same way as I did before.

Are they interesting to talk to? Are they passionate, curious, authentic, opinionated? Do they have a product mindset? Do they need to be told what to do? Would you hang out with them after work? Can they explain themselves in a way that’s easy to understand?

In my experience these are much better predictors of success than leetcode.

His LinkedIn post says “At OpenAI, I co-invented ChatGPT and RLHF”.

And while ChatGPT claim is a marketing stretch, RLHF one is a pure lie.

But is there any proof that Jev is frontier intelligence?

It thinks there are two Rs in strawberry.

It fails to assign probabilities to die roll outcomes or coin flips.

When used as an LLM it thinks it is Qwen.

Apparently reordering the list of possible answers can change assigned probabilities by up to 20%.

I’ve seen claims that it struggles to play tic tac toe.

So far there is a lot of evidence that it behaves exactly like a tiny open weight model. The only argument against this claim is “trust me bro” claims from it’s author.

But didn’t you hear?

> Jev is neither small nor an LLM

>all you need to do to make an agentic RAG is to have the LLM be able to write/rewrite its own search queries (possibly in multiple passes)

I think this is a huge oversimplification, the term "search query" is doing a lot of heavy lifting here.

When Claude Code calls something like

  find . -type d -maxdepth 3 -not -path '*/node_modules/*'

to understand the project hierarchy before doing any of the grep calls, I don't think it's fair to call it just a "search query", it's more like "analyze query". Just because text goes in and out in both cases, doesn't mean that it's all the same.

When you give the agent the ability to query the nature of the data (e.g. hierarchy), and not just data itself, it means that you need to design your product around it. Agentic RAG has entirely different implementation, product implications, cost, latency, and primarily, outcomes. I don't think it's useful to pretend that it's just a different flavor of the same thing, simply because at the end of the day it's just some text flying over the network.

While it's true that people are naturally predisposed to invent the "secret quantizing" conspiracy regardless of whether the actual conspiracy exists or not, I think there's more to the story.

I've seen Sonnet consistently start hallucinating on the exact same inputs for a couple hours, and then just go back to normal like nothing ever happened. It may just be a combination of hardware malfunction + session pinning. But at the end of the day the effects are indistinguishable from "secret quantizing".

> it's ridiculous to reserve the term "RAG" for just the earliest most basic implementation

Whether we like it or not, dumb semantic search became the colloquial definition of RAG.

And when you hear someone saying "we use RAG here" 95% of the time this is exactly what they mean.

When you inject user's name into the system prompt, technically you're doing RAG - but nobody thinks about it that way. I think it's one of those case where colloquial definition is actually more useful that the formal one.

> doing it properly doesn't require a fundamentally different technique

But agentic RAG is fundamentally different.

The furnaces I'm comparing are Claude Code vs hiring more engineers. Not Claude Code vs Codex vs Gemini. If $20/mo makes an engineer even 10% more productive, purchasing Claude Code is a no-brainer.

Most engineers feel like Claude Code is a multiplier for their productivity, despite all the flaws that it has. You're arguing that CC is unusable and is a net negative on the productivity, but this is the opposite of what people are feeling. I am able to tackle problems I wouldn't even attempt previously (sometimes to my detriment).

> What a weird blast furnace! Would anyone try to use this tool in such a scenario? Not most experienced metalworkers.

Absolutely wrong. If this blast furnace would cost a fraction of other blast furnaces, and would allow you to produce certain metals that were too expensive to produce previously (even with high error rate), almost everyone would use it.

Which is exactly what we're seeing right now.

Yes, you have to distinguish marketing message vs real value. But in terms of bang for buck, Claude Code is an absolute blast (pun intended)!

1. Humans are capable of writing good code. Most won't, but at least it's possible. If your company needs good code to survive, would you take 5% chance or 0% chance?

2. Even when humans write crappy code, they typically can maintain it.