HNHacker News
TopNewBestAskShowJobs

Gracana

4,137 karma · joined December 29, 2013

submissionscomments
Gracana··on I’m joining OpenAI
> He made a social network for bots, FFS.

Matt Schlicht made Moltbook, not Peter Steinberger.

Gracana··on An AI agent published a hit piece on me – more things have happened
> I realize it makes you uncomfortable

I think you're misunderstanding or misrepresenting them. The fight to have the most jaded or pessimistic take, the hottest flame, the spiciest rant, it's all so predictable and it's just a bunch of the same people saying the same things and agreeing with each other for the nth time. It brings nothing new to the table, and the posts that actually respond to the new information get drowned out or worse downvoted for insufficient vitriol.

Gracana··on GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
There's a reddit comment here https://www.reddit.com/r/LocalLLaMA/comments/1r4m4it/comment... that says:

my system is running GLM-5 MXFP4 at about 17 tok/s. That’s with a single RTX Pro 6000 on an EPYC 9455P with 12 channels of DDR5-6400. Only 16k context though, since it’s too slow to use for programming anyway and that’s the only application where I need big context.

Gracana··on An AI agent published a hit piece on me – more things have happened
How?

I think you’re imagining that these hypocrites exist.

Gracana··on GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
It's important to have enough VRAM to get the kv cache and shared trunk of the model on GPU, but beyond that it's really hard to make a dent in the pool of 100s of gigabytes of experts.

I wish I had better numbers to compare with the 2x M3 Ultra setup. My system is a few RTX A4000s on a Xeon with 190GB/s actual read bandwidth, and I get ~8 tok/s with experts quantized to INT4 (for large models with around 30B active parameters like Kimi K2.) Moving to 1x RTX Pro 6000 Blackwell and tripling my read bandwidth with EPYC Turin might make it competitive with the the macs, but I dunno!

There's also some interesting tech with ktransformers + sglang where the most frequently-used experts are loaded on GPU. Pretty neat stuff and it's all moving fast.

Gracana··on I now assume that all ads on Apple news are scams
Yup, you're right.
Gracana··on GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
Six months ago I'd have said EPYC Turin. You could do a heck of a build with 12Ch DDR5-6400 and a GPU or two for the dense model parts. 20k would have been a huge budget for a homelab CPU/GPU inference rig at the time. Now 20k won't buy you the memory.
Gracana··on Do not apologize for replying late to my email
Same. I tried really hard to quote properly, because I was so annoyed by the top-posting mess that everyone else did, and it frustrated me that people would add you to an email where you need to read 100 things that came before it (with increasingly garbled formatting) to understand what was going on.

I felt people were unwilling to take the responsibility for communicating properly, and so they took the easy route where they could shrug their shoulders and say "I included all the context."

I only ever got complaints from people who were confused by the quoting style or didn't know what the email was about. I'm not sure if it's still true, but at the time, Outlook didn't use threaded view mode by default and most people didn't know about it. FWIW I work in manufacturing and not in tech, I expect the level of competence in tech is a little higher, though I also hear how people moan about having to learn the tools they use every day, so maybe there's little difference.

Gracana··on The switch to Linux and the beginning of my self-hosting journey
Thanks for your review of five minutes of using Linux after 30 years of Windows use. Sorry to hear that zero effort didn't work out, but what can you do.. it's just not ready for the desktop. Enjoy Windows 11.
Gracana··on I now assume that all ads on Apple news are scams
A phone is a poor replacement for a laptop.
Gracana··on Building a 24-bit arcade CRT display adapter from scratch
Having dealt with the errata sheets for microcontrollers with all those fancy IO devices that solve real marketing problems etc, I'd kill to fix those problems with a software upgrade.
Gracana··on Hacking Moltbook
You could have an LLM answer that, and then still interact as a human.
Gracana··on Kimi K2.5 Technical Report [pdf]
That is a very good point and I would love to do it, but I built this machine in a desktop case and the motherboard has seven slots. I did a custom water cooling manifold just to make it work with all the cards.

I'm trying to figure out how to add another card on a riser hanging off a slimsas port, or maybe I could turn the bottom slot into two vertical slots.. the case (fractal meshify 2 xl) has room for a vertical mounted card that wouldn't interfere with the others, but I'd need to make a custom riser with two slots on it to make it work. I dunno, it's possible!

I also have an RTX Pro 6000 Blackwell and an RTX 5000 Ada.. I'd be better off pulling all the A7000s and throwing both of those cards in this machine, but then I wouldn't have anything for my desktop. Decisions, decisions!

Gracana··on Antirender: remove the glossy shine on architectural renderings
Same here. Disappointing. I wanted to run it on that picture of a church that looks like a chicken.
Gracana··on Kimi K2.5 Technical Report [pdf]
I'm running the Q4_K_M quant on a xeon with 7x A4000s and I'm getting about 8 tok/s with small context (16k). I need to do more tuning, I think I can get more out of it, but it's never gonna be fast on this suboptimal machine.
Gracana··on OpenClaw – Moltbot Renamed Again
What does scaling a person mean?
Gracana··on Trinity large: An open 400B sparse MoE model
I thought paging was so inefficient that it wasn't worth doing vs using CPU inference for the parts of the model that are in system memory. Maybe if you have a good GPU and a turtle of a CPU, but still somehow have the memory bandwidth to make shuffling data in and out of the GPU worthwhile? I'm curious to know who is doing this and why.
Gracana··on Maine’s ‘Lobster Lady’ who fished for nearly a century dies aged 105
I went looking for information on the oldest known lobster, and found this article about a 20lb lobster named George who was estimated to be 140 years old. Neat!

https://en.wikipedia.org/wiki/George_(lobster)

Gracana··on Microsoft forced me to switch to Linux
I'll have to remember that one, that's a good way to put it.
Gracana··on Microsoft forced me to switch to Linux
> CUDA .. nvidia-only

Well, duh.

> unless you use the binary-only drivers

Which you also have to do with nvidia cards.

> which only work on a handful of very new cards

So get a new card?

> You can't use it for editing video.

Yes I can.

----

I actually switched from a 7900 XTX to a 4090 BITD because I wanted CUDA, so I get that angle, but that doesn't mean I go around telling people "AMD isn't accelerated," because it's not true and it's a silly thing to try to claim.

Gracana··on Microsoft forced me to switch to Linux
> AMD isn't accelerated

This is a bewildering assertion.

Gracana··on Microsoft forced me to switch to Linux
Now I'm shocked by the cost of Netflix.
Gracana··on Microsoft forced me to switch to Linux
This is a surprising opinion to encounter, given my experience with scaling on Windows, where simple things like taking my laptop off its dock (going from desktop monitors to laptop screen) causes applications to become blurry, and they stay blurry even when I've returned the laptop to the dock. Or how scaling causes some maximized window edges to show up on the adjacent screen. Or all manner of subtle positioning and size bugs crop up.

Is this more of an aspirational thing, like Windows supports "doing it right", and with time and effort by the right people, more and more applications may be able to be drawn correctly?

[edit] I guess so, I see your comment about setting registry keys to make stuff work in Microsoft's own programs. That aligns more closely with my experience.

Gracana··on Microsoft forced me to switch to Linux
I did this and was happily Windows-less for quite a few years. I ended up building a PC with a big GPU and so I switched back to PC gaming with a Windows installation alongside Linux, but I still think the console route is a great option.
Gracana··on Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model
With 32B active parameters, Kimi K2.5 will run faster than your 70B model.
Gracana··on Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model
The level of deceit you're describing is kind of ridiculous. Anybody talking about their specific setup is going to be happy to tell you the model and quant they're running and the speeds they're getting, and if you want to understand the effects of quantization on model quality, it's really easy to spin up a GPU server instance and play around.
Gracana··on Ask HN: Books to learn 6502 ASM and the Apple II
Assembly Lines is the one I came to recommend. Great book to sit down with and work your way through.
Gracana··on Challenges and Research Directions for Large Language Model Inference Hardware
Yeah, it is quite specialized for inference. It's unlikely that you'd see this stuff outside of hardware specifically for that.

Development systems for AI inference tend to be smaller by necessity. A DGX Spark, Station, a single B300 node... you'd work on something like that before deploying to a larger cluster. There's just nothing bigger than what you'd actually deploy to.

Gracana··on Challenges and Research Directions for Large Language Model Inference Hardware
Density is the key here, not persistence.
Gracana··on Extracting a UART Password via SPI Flash Instruction Tracing
article says 60MHz
← PreviousPage 4 of 34Next →