make3

Training neural networks is requires extremely high speed interconnect between the humongous GPUs

I'm not sure we played the same game

the depth comes from the character stories and development, like Laezel and Shadowheart

personally I think it's because of models trained so hard to be good at Math and Coding with reinforcement learning, which makes them kind of crazy hard reasoners, and bad for casual discussion

maybe obvious, but Baldur's Gate 3 is really that level (or better) of a game (recommendation)

The models are solving Millennium Prize questions and making progress on the Riemann Hypothesis, yet the top comment is still looking at pelicans somehow. To me, it feels even more superficial and pointless than arguing about the presidential candidates' tie choices in a debate.

> "Python used to have limited parallelism, due to its global interpreter lock (GIL). Only one thread could run Python code at a time.”

It is still limited this way in the regular build, there's no "used to". Author should fix that.

Importantly though, LLMs can generate questions in parallel, and there can be much fewer questions (q) than segments to evaluate against (n), so the cost of jev on questions (jev(q * n) + llm(q)) vs the cost of putting all the segments in the LLM can be smaller (llm(n)), (llm(n) >> llm(q) + jev(n * q) if n >> q)

The real benchmark that's missing for me is MaxSim on a real embedder from MTEB.

The real guide is here: https://huggingface.co/convaiinnovations/laya

Well the numbers don't lie Laya is ModernBert-Large (421M) fine-tuned and has a similar to better performance than TypeSafe Jev https://raw.githubusercontent.com/NandhaKishorM/laya/main/as...

People are allowed to defend themselves man. In this case, economic pressure is the only thing that's likely to work medium term.

Take ModernBert, add a fully connected layer with 255 outputs, take a bunch of classification datasets from Huggingface, write the code to have the datasets fit the jev format on these 255 outputs, do supervised fine tuning on the datasets with that format. then use a confidence loss of some kind

Laya is just ModernBert fine-tuned. It's still a language model (just a bidirectional one). Let's call it what it is instead of this weird Decision Model mysticism.

Jev and decision models are *a derivative* of a pretrained LLM trunk, and one that's super easy to build once you have an LLM. Put a single step fully connected NN on the transformer, create a training dataset for it, add fancy but bread and butter multi step decision RL, nothing crazy.

*They have zero moat* against established AI companies, as the recent copies show, and will do worse than frontier labs.*

> it'll look a lot more like putting Jev at the center of a much more expensive suite of software

but anyone can do that, that's not their product, Jev is

language models reason from human examples partly, and a lot of the abstractions are simply actual useful compressions of information that are useful for LLMs as well

I guess there would be new interesting problems emerging, & AI would solve them, until it's completely impossible for us to understand

Imagine you get up one morning and most open questions in math are solved lol.

I think it's a damned if they do, damned if they don't scenario. I think they would be afraid to seem like they're claiming that the selected scientists deserve the praise of an invention or something, when it's "just the machine who did the work"

It was OpenAI that fucked up the Navier Stokes explosion situation, not Anthropic

Americans just make fun of Europe for no reason, it is what it is

The output key is determined through code by the harness from the inputs, and the model generates 255 floats in order that follow that schema by reading the expected return type "rank" in the input. The harness then programmatically uses the floats of the 255 floats that are useful. The harness can assume that the correct float will be in the correct position as the model is trained to follow the schemas

no, 255 numbers come out at once all the time, the order is determined by the inputs

logits assumed some form of softmax or logistic, which may not be the case

FLAN-T5 generated text (Jev does not generate text), and BERT wasn't able to do tasks without fine-tuning.

Jev is basically a kind of FLAN-BERT, if you want, where it has built-in multi-task ability, but doesn't generate text. It only generates 255 floats all at once, making it much faster, and what those floats mean (if anything) depends on the prompt.

Eg, the following query is put in the encoder model:

{"question": "Rank these 5 things by increasing order of how big they are", "choices": ["truck", "cow", "mouse", "ant", "building"] }

The model returns [3., 2., 1., 0., 4.], and 249 other meaningless floats that are hidden from you by the UI.

The UI stitches the first 5 floats with the choices and returns something like:

{"rank": ["ant", "mouse", "cow", "truck", "tower"]}

the appeal would be if they can deliver it at a much higher performance and similar speed, which is plausible

Your joke also shows why it's dumb though.

We have general AI capable of proving professional mathematician level math theorems and capable of taking down global infrastructure through hacking and that completely changed what it means to be a software engineer, but we're evaluating it by generating fucking SVGs for the same animal in the same conditions, and it's supposed to mean something somehow

they're is obviously joking

This benchmark is useless and should die. LLMs have likely trained on it, it's too easy to game by training specifically for it, & it doesn't mean much

parent means that they could get more client / a larger part of the market, which would lead to more income (more tokens) despite lower marginal prices