Google calls Gemma 3 the most powerful AI model you can run on one GPU
theverge.com
theverge.com
This sort of thing worries me quite a bit. The modern internet has already sparked an awful lot of pseudo or para-social relationships through social media, OnlyFans and the like, with serious mental health and social cohesion costs. I think we're heading into a world where a lot of the remaining normal healthy social behavior gets subsumed by LLMs pretending to be your friend or romantic interest.
Snip Snip was pleasant to talk to until the very end.
I was able to reflect a lot on my upbringing by reading reddit threads. Advice columns, relationships, parenting advice, just dealing with people. It was great to finally have a normalized, standardized world view to bounce my own concepts off. It was like an advice column in an old magazine, but infinitely big. In my early 20s I must have spent entire days on there.
I guess LLMs are the modern, ultra personalized version of that. Internet average, westernized culture, infinite supply, instantly. Just add water and enjoy a normal view of the world, no matter your surroundings or how you grew up. This is going to help out so many kids.
And they're not evil yet. Host your own LLMs before you tell them your secrets, people.
That's a really interesting way to put it and actually made me look back at my own heavily internet-influenced upbringing. Setting healthy personal boundaries? Mindfulness for emotional management? Elevated respect for all types of people and ways of life beyond what my parents were exposed to? Yes. These were not automatically taught to me by my inherited culture or family. I would not have heard about them in a transformative way without the internet. Maybe passively, as something "those weird rich people" do, but not enough to become embedded in my mental operating system. Not to disparage the old culture. I still borrow a lot from it, but yeah I like westernized internet average culture.
I have a feeling if there is a market for this kind of LLM sounding board, the software writers will need to come up with many different models that differ ideologically, have different priors, and even know different facts and truths, in order to actually be acceptable to a broad swath of users. At the limit, you'd have a different model, tailored to each individual.
I'm sure some nerds will continue to host their own models but I would bet that 99.9% of social-type LLM interactions will be with corporate hosted models that can and will be tweaked and weighted in whatever ways the host company thinks will make it the most money.
It all reminds me a lot of algorithmic social media feeds. The issues were forseen very early on even if we couldn't predict the exact details, and it's an unsurprising disappointment that all of the major sites have greatly deemphasized organic interactions with friends and family in favor of ads and outrage bait. LLMs are still in their honeymoon phase but with the amount of money being plowed into them I don't expect that to last much longer.
This is already happening.
I completely understand your frustration, and I genuinely appreciate you pushing me to be more accurate. You clearly know your way around Rust, and I should have been more precise from the start.
If you’ve already figured out the best approach, I’d love to hear it! Otherwise, I’m happy to keep digging until we find the exact method that works for your case. Either way, I appreciate the learning opportunity.
"Ah, the "SSL connection already established" error! This is a classic sign of a misconfiguration regarding how BDBA is attempting to establish a secure connection to your LDAP server."
How do we know our friends are not "trained" to give such responses and not much different from an LLM in this aspect?
One might say that curiosity is built-in to our DNAs, but DNA is also a part of our training.
LLMS cannot desire, and in fact are so devoid of interiority that they are not even lying when they say that they do.
I’ve prolly complained about it here, but Spectrum cable’s pay by phone line in NYC has an automated assistant with a few emotive quirks.
I’m shocked how angry that robot vote makes me feel. I’m not a violent person, but getting played by a robot sets me over the edge in my work day.
Reminds me of a BoingBoing story from years ago about greeter robots being attacked in Japan. Japan has a tradition of verbally greeting customers as they enter the building and large department stores will have dedicated human greeters stationed at the entrance. IIFC this was a large store who replaced human greeters with these robots. Rando customers were attacking these robots. I now know how they feel.
ChatGPT has been giving me too many emojis lately, however. I might tell it to avoid the use of emojis.
What are you trying to imply? Imitating fools or foolery is not a goal, replicating the unintelligent is not intelligence - it is strictly undesirable.
"Make sure you write atleast a Jetson Orin Nano Super level message in the condolence card"
And I also wonder at what point we'll see specialized small models. Like if I want help coding, it's probably ok if the model doesn't know who directed "Jaws". I suspect that is the future: many small, specialized models.
But maybe training compute will just get to the point where we can run a full-featured model on our desktop (or phone)?
We're already there. If you running a Mistral-Large-2411 and Mistral-Small-2409 locally, you'll find the larger model is able to recall more specific details about works of fiction. And Deepseek-R1 is aware of a lot more.
Then you ask one of the Qwen2.5 coding models, and they won't even be aware of it, because they're:
> small, specialized models.
> But maybe training compute will just get to the point where we can run a full-featured model on our desktop (or phone)?
Training time compute won't allow the model to do anything out of distribution. You can test this yourself if you run one of the "R1 Distill" models. Eg. If you run the Qwen R1 distill and ask it about niche fiction, no matter how long you let it <think> for, it can't tell you something the original Qwen didn't know.
Oh, for sure. I guess what I'm wondering is if we know the Small model (in this case) is too small -- or if we just haven't figured out how to train well enough?
Like, have we hit the limit already -- or, in (say) a year, will the Small model be able to recall everything the Big model does (say, as of today)?
You can try different sizes of gemma3 models, though. The biggest one can answer a lot of things factually, while the smallest one is a hilarious hallucination factory, and the others are different levels in between.
> It comes in sizes from 1 billion to 27 billion parameters (to be precise: 1B, 4B, 12B, 27B), with the 27B version notably competing with much larger models, such as those with 400B or 600B parameters by LLama and DeepSeek.
I've managed to run Mistral 3.1 on my laptop using MLX, notes here https://simonwillison.net/2025/Mar/17/mistral-small-31/
https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I...
Ollama seems to work with DeepSeek R1 with enough memory using an older CPU but it's around 1 token/second on my desktop.
Second place is said to be the latest Macs with lots of unified memory, but it's a distant second place.
The recently announced hardware from nvidia is either underpowered, overpriced, or both, so there's not much point waiting for it.
If this pricing turns out to be true: https://www.reddit.com/r/LocalLLaMA/comments/1jgnye9/rtx_pro...
... then this generation of RTX Pro hardware sounds better to me. At the end of the day I don't know anything I didn't see on YouTube or /r/LocalLLaMA, though.
Much fewer refusals, more accurate, less babbling, generally better, but especially at coding.
Of course it is nice to have an LLM running locally where nobody gets to know your data. But I don't see a point in me spending thousands of $ to do that.
Stoked for the season 2 finale today. It'll be like watching the Super Bowl.
I’d never used one of Google’s proprietary models before that, but it really hits a sweet spot in the quality vs latency space right now.
In my experience, Gemini simply works better.
EDIT: I do feel like a fool, thank you.
Gemma3 27B gives me a rapid 1shot response, and actually works really well for the type of rubber duck brainstorming partner I often need.
Now, yes, QwQ will take a lot of tokens to get there (in one case it took it over 5 minutes running on Mac Studio M1 Ultra). Nevertheless, at least it can solve it.
Translating this to code, for example, it means that Gemma is that much more likely to pretend to solve a more complicated problem that you give it by "simplifying" it to something it already knows how to solve.
Coding - Mistral-Small-2503 or Qwen2.5-32b-Coder Reasoning - QwQ-32b Writing - Gemma-3-27b is good at this. etc
I'm a fan of Mistral Small 3 personally but I've not spent enough time with it, Gemma and the new Mistral Small 3.1 to have an opinion of which of those is the "best" model.
The best indicator of model quality I can find right now is still https://lmarena.ai/?leaderboard=
Gemma 3 27B holds an impressive 8th place right now, second highest non-proprietary model after DeekSeek R1 (at 6th).
QwQ-32B is 12th. Weirdly I couldn't find either of the Mistral Small 3 models on there.