HNHacker News
TopNewBestAskShowJobs

segmondy

9,039 karma · joined January 30, 2013

Builder of thingz, lover of old robust tech like unix, prolog, lisp, APL, forth, postgresql, but don't mind the shiny new things ... right now, I'm all in on GenAI, Local LLMs & LLM driven agents, team llama.cpp, building LLM agents since 2023

segmond AT gmail dot com

submissionscomments
segmondy··on Ask HN: What hardware are you using to run local LLMs?
Local clusters built around old GPUs. I have had cluster of P40 GPUs, MI50s gpu, and now I have a cluster of cmp170hx. I'm always looking for the next cheap GPU dump and then I get as many as I can before it get's popular.
segmondy··on China's First Frontier AI Export Controls: Mofcom Announcement No. 48
They gained popularity because THEY ARE GOOD, not because they are open weight. There are so many other models that are not popular. Do you know about arcee and their models? every used inkling? how about laguna? these are American open weights, they didn't take off because they are not as good as the rest. Do you know about the command-x series from COhere? probably not, this is a Canadian model, folks into local were really into it 2 years ago when they were keeping. do you know about dbrx? Yeah, databricks an American company also gave us a model. Anyways, my point is the competition is strong and giving away for free doesn't matter if it can't keep up. It's expensive, to send bandwidth, storage, gpu compute to run a model when there are better options.
segmondy··on We're releasing the weights of Mistral Large 4
Yes, quite a bit, but you can lookup OlMo from allen.ai for starts
segmondy··on We're releasing the weights of Mistral Large 4
4 bit quality would be good, but you need more GPU, the 4bit weight is 610gb. EXL3 in 3 bit has been okay the few times I used it. https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B-NVF...
segmondy··on US economy beats forecasts to grow at 3.4% rate in third quarter
Do we still trust the data coming out from this admin?
segmondy··on We're releasing the weights of Mistral Large 4
Their last 2 models were a disaster and didn't live up to their self reported evals. I hope this time around is much better. I'm happy tho that they are still figuring out how to get better even if only measured against themselves. This is great, the more options we have the better for everyone.
segmondy··on US Government is hiding a secret court ruling about how it spies on Americans
It will never get declassified. Never. The people get weaker and more passive with time, the govt gets stronger and reckless and less accountable.
segmondy··on Genie 4
With Anthropic going public, I expect a lot of model releases from the competition signalling to the market. "We are still around, please leave some money for us too"
segmondy··on Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)
why would they think that? I'm not a pro, but I know you can't serve more than kv cache pool that you have. If you want to run concurrent requests, you need to have more kv cache pool than your context window. The entire thing is weird, my bet is they were thinking most people will be using about 100k cache, compacting, etc and not nearing 600k ever.
segmondy··on Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)
Doesn't matter, they could have marketed it as 1 million token window, you would then infer, get to 600k and get an error. You won't be able to finish your work, and you won't get credit for 600k tokens.
segmondy··on Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)
"We deployed GLM 5.3 with a 1,048,576-token context window, but the KV cache pool on our prefill GPUs only had room for 681,280 tokens. We were aware of this mismatch before launch. "

Did they advertise it as 1,048,576 token when they only had room for 681,280 tokens?

segmondy··on OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?
This is exactly the point! The chinese labs have been sharing amazing research and breakthroughs.
segmondy··on OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?
They implemented efficient caching and ideas from Deepseek and the Chinese labs. They can now server cheaper and faster.
segmondy··on Behind the scenes at the NeXT Computer office
You can vibe up your own distinctive user interface today! it's 2026.
segmondy··on Show HN: TitleArt – Make text like it's 1997
This is pretty awesome! Thanks for sharing. I was playing with it and had to stop! Brings back lots of good memories.
segmondy··on IBM CEO says open weight models are now good and cheap on-prem
Cloud models sometimes get degraded. You can trust your local model and know what you are running.
segmondy··on IBM CEO says open weight models are now good and cheap on-prem
Have you heard the cost of health insurance and seeing a doctor? Some people don't see going to hospital as an option. Better for them to have a local model than no options. But let me tell you how I use it for health, I take all my health records, lab results, every thing, I throw it in there for a second opinion, it explains things to me, it suggests questions I should ask on follow up, possible options, etc.
segmondy··on IBM CEO says open weight models are now good and cheap on-prem
My local model never says no. I never get banned. If I want to do security stuff, reverse engine, there it goes! Do I wanna chat about politics? No problem! Do I want it to analyze medical stuff? I don't get to hear that it's not a doctor and I should call 911. Do I want to explore legal docs, no problem! There are lots of things that cloud models will say no to! I read that don't even agree to help with advanced biology and chemistry topic much. I also never run out of tokens, no 5 hr limit. I can run it 24/7. I don't run out of tokens. I can run 16 sub agents at the same time, non stop! 1-4 billion tokens a month? No problem. Any agent harness I want! No restrictions. Privacy all day. My own true system prompt, not something someone has determined, no filters. Furthermore, there are many open weight cloud models that don't have equivalent online. For a while before cloud models got good at math, the best model at math was DeepSeekv2 Math and there was no API. The only way to use it was to run local. All sorts of audio models, image/video models, etc. No multiple subscription needed. Private UI for my family, kids included (I can guard what kids infer). Got a friend that really needs some tokens? I give em access through private VPN. Telling someone that they are paying for decades of Claude for local AI is like telling someone they are paying for decades of public transport when they buy a car. Sure, you are right, but ...
segmondy··on America has lost 200k of jobs in media, film and the arts. How much is AI?
Why? AI is taking jobs. getty images just filed bankruptcy, stackoverflow went from heavy use to almost no use. AI is taking jobs, tipping toeing around it isn't the best thing. It needs to be said, and then what next needs to be figured out because first we talk about 200k jobs, then we will next talk about 2 million and then 20million
segmondy··on Kimi K3.1 Pro and K3.1 Pro Highspeed
They don't want to be banned, and maybe their best offensive cyber capability models will be for their govt.
segmondy··on PHP 8.2.35
Thanks to OpenAI, everything is a PR stunt.
segmondy··on DeepInfer: High-performance inference framework for DeepSeek models
someone already beat me to it, I'm building now - https://github.com/HalfVulpes/Deepinfer-SM80
segmondy··on DeepInfer: High-performance inference framework for DeepSeek models
If anyone is interested in getting this running on sm80s, please let me know. :)
segmondy··on Ask HN: What are you working on?
Fiddling and tooling around with AI locally.
segmondy··on DeepSeek V4.1 Flash 1017 (Max)
DeepSeek is popular, in my circle some of us are crazy about Hy4-preview & MiMo2.6-Pro
segmondy··on DeepSeek V4.1 Flash 1017 (Max)
Yes you can, but IMO, the mistake people make when running huge models at slow speed is that they try to put it through an agent loop. Don't, but they are great for one shot output that you can then pick up and clean up manually
segmondy··on DeepSeek V4.1 Flash 1017 (Max)
... and DSv4.1Flash is more free. I ran the model all weekend. All my data is belong to me and me alone.
segmondy··on DeepSeek V4.1 Flash 1017 (Max)
Too early to call, I ran it through the weekend and compared with Qwen3.8-Flash-Next, DSv41F-1017 at FP8 and Q3.8FN at Q8. Both were great, but surprisingly, Q3.8FN had the better result. I was modelling and generating simulation code for a complex system. I used both because I had a bunch of images that needed to be used for input to steer the output. I think these models have strengths and it's best to experiment and find what works of you. I didn't find MiMo2.6-Flash in the same tier, but MiMo2.6-Pro is a beast.
segmondy··on Wine ported to OpenBSD
Then vibe up the ones you need for yourself.
segmondy··on DeepSeek-v4.1-Flash-1017
skill issues, brute forcing with AI.
← PreviousPage 2 of 34Next →