jerpint

This looks like it would be really useful for multimodal LLMs

We’re definitely going to need better benchmarks for agentic tasks, and not just code reasoning. Things that are needlessly painful that humans go through all the time

The yolo models are “dumb” black box object detectors, it’s a supply chain attack, the model itself was very likely never touched

Their docs are very fun to read. I’d probably recommend starting with the “transformers” library for python if you want to play with some language models e.g. Bert:

https://huggingface.co/docs/transformers/en/model_doc/bert

Basically equivalent to GitHub but for models. Anyone can upload anything, but it kind of standardizes tools and distribution for everyone. They also have a team that helps integrate releases for easier use and libraries for fine tuning

I have a sneaking suspicion OpenAI will announce something very similar in a few days

StyleTTSv2 is pretty good and open source, you can easily traverse its latent space for voice

im not sure what you mean - the embedding model is independent of the embeddings themselves. Once generated, the embeddings and vector store should exist 100% locally and thus not part of any secret sauce

Somewhat agreed for use cases of text classification, but for anything requiring more language understanding it is a desirable property

> In summary, we used a 2.4-million-parameter small language model to achieve accuracy comparable to a 100-million-parameter BERT model in text classification.

Neat, but the question will be how the scaling laws hold up

I’m really excited for where this is going. From the demo videos, it seems to be a step up from Oasis, which itself came out only 2 weeks ago. I expect to see a lot of innovative use cases in this field

Sadly this is seen at so many prestigious ML conferences, a trimmed X axis which makes performance seem significant when it’s sometimes incremental

I wonder at what point it will be ~as much overhead to pass through a subset of the data with a small yet capable and fast LLM vs. using a crude dot product when doing retrieval

Same here, to be honest I never gave much thought to seeking alternatives, but this one looks really nice, especially having the line numbers and similar layout to GitHub diffs

Does anyone know of a “copilot” style autocomplete in the CLI? I don’t want it to run anything for me, just predict what command I might type next

Do you have proof for this? Why accuse one team and not the other?

This is a very impressive demo, this seems very early in a what might in hindsight be an “obvious” direction to take transformers towards

Having a hotkey to access the app removes a lot of friction

Just remember that retrieval is the real bottleneck in RAG, so your problem could very much be related to how you create and embed your chunks and not the model you’re using

The biggest problem these days is that it’s very easy to hack together a solution for a problem that, at first glance, seems to work just fine. Understanding the limits of the system is the hard part, especially since LLMs can’t know when they don’t know

I made a clock out of MNIST digits that are obtained travelling the latent space of an autoencoder:

https://www.jerpint.io/blog/mnist-clock/

Could also work for social media which can be hard to scrape

I found grounding-dino better than Florence and faster

Funny, I was actually thinking of micromouse when I made my comment, thank you for clarifying

I’m not sure I agree, text is full of compressed information but lacks all of the visual cues we all use to navigate and understand our world. Video data also has temporal components which text is really bad at.

They mention that the input grid contains the layout of the maze, but isn’t the point of solving a maze that you don’t have a global view of it?

Video data is still very much untapped and likely to unlock a step function worth of data. Current image-language models are trained mostly on {image, caption} pairs with a bit of extra fine tuning

If these models could run offline, 100% locally, with expected privacy, Im up for it, i want to use a product, not be training data

Just a few seconds of speech gets you pretty far with a lot of models

Im assuming there are many open source alternatives to do this? Why pay to run edge devices?