Another possibility is that Pemulis is simply bad at math :D
2,096 karma · joined August 11, 2022
Another possibility is that Pemulis is simply bad at math :D
Based on this part:
> We set out to support a new engine that makes multimodal models first-class citizens, and getting Ollama’s partners to contribute more directly the community - the GGML tensor library.
And from clicking through a github link they had:
https://github.com/ollama/ollama/blob/main/model/models/gemm...
My takeaway is, the GGML library (the thing that is the backbone for llama.cpp) must expose some FFI (foreign function interface) that can be invoked from Go, so in the ollama Go code, they can write their own implementations of model behavior (like Gemma 3) that just calls into the GGML magic. I think I have that right? I would have expected a detail like that to be front and center in the blog post.
whatever the issue is, "setting an env var is unsafe" is so interesting to me that I'm now craving a blog post explaining this
What's the process like for getting such a low-level primitive added to Linux? Especially a low-level primitive that 1) exist basically just to emulate behavior of a completely different kernel, and 2) is only needed for a subset of users to play games?
I'm not complaining, just curious. I would assume a patch to add the above would be heavily scrutinized. Is it because the popularity of the Steam Deck / proton?
This framework looks really well designed, I'm going to take it for a spin.
Right as I was learning about Q-Learning in college, they published Deep Q-Networks / AlphaGo. It really lit a spark in me.
Is it still true that they're the only ones who can make LiFePO batteries? Is anyone else working on production of these?
What accounts for this gap - patents (ha), secret research, materials, manufacturing prowess, all of the above?
From the title, my best guess was they applied some kind of RL/GRPO to an existing model.
But... they took an existing model that had already undergone SFT for reasoning... and then used it to generate data to SFT the exact same model... nothing wrong with that, but it doesn't seem to warrant the title they chose.
There isn't any kind of "dry run" or "phantom" mode, where requests are not actually blocked, but appear marked in the log UI as "would be blocked". This is super important because I want to see all the things my home network is doing that would be blocked before I actually hit the big red button. I want to fix up the allow/denylist before going live.
It's also not possible (or not clear) how to have different behavior for different clients. For my "smart tv" which I begrudgingly have to allow on my network occasionally for software updates, I want to treat it with the strictest possible list. But for my phone, I don't want that same list. There's a concept of "groups" so perhaps this is user error on my part, but the UI does not make this clear.
I guess you could make the argument "well what if one day they stop using homomorphic encryption", but that argument doesn't make much sense since 1) why would they and 2) you could already ask the same question today "what if they just started sending info anyway"
It's like, "oh, you want all your photos to be searchable, like 'dogs' or 'Eiffel tower'? Fine, we'll create an on-device embedding of each photo, use homomorphic encryption so you can share it with us and we can match it to its contents without even knowing what they are, then we'll send that back to your device for storage. Oh, and we'll use a relay so we don't even see your IP address while doing this, not that it matters since we can't decrypt the content anyway." It's pretty wild, like they could have easily skipped all this and only a fraction of a fraction of a fraction of users would even know or care.
In fact, I was pretty annoyed that the news story from the above example was "Apple is looking at all your photos and violating your privacy", since they spent so much effort doing it the right way, in a way that respects your privacy, it makes it less likely they will bother going through the effort again
https://www.theregister.com/2025/01/03/apple_enhanced_visual...
Their audience here is the incoming administration, which is founded on party loyalty and an "us-vs-them" mindset, where "them" are the corrupt elites.
They're making a bet that whoever is prosecuting them in 4 months is going to be a lot friendlier to them if they act like victims of a vengeful democratic elite, and they're probably right. Really smart, honestly.
either 1) it's a real phenomena that is under-diagnosed and ignored by medical professionals (something I believe happens) or 2) it's not a real condition and patients who suffer from it are suffering from a mix of mental illness combined with other real diseases that are exacerbated (something that I believe happens)
both sound like very real, very possible problems, and neither one seems more likely to me than the other. Or perhaps it's a big mix of both, on a case by case basis.
is anyone else besides Intel making ~4nm* node wafers on US soil?
*yes I know I know I know about the misnomer with using nm measurements nowadays
The hallmarks of this style are easy to spot. Lots of vacuous sentences, which (usually) make grammatical sense but no logical sense. It's very "first-draft-esque". Lots of sentences which, if read a second time, signify nothing, or otherwise just don't hold up to scrutiny.
By "SpaceX moment", I mean a startup entering an impossible market, where the barrier to entry is billions of dollars of research and manufacturing, a market dominated by industry giants from the 60s, and yet still coming out on top somehow.
like, say you are building code, and all the below functions are compilers, and * denotes an evil compiler. Every link in the chain is a compiler building another compiler, until the last node which builds the code.
A() -> B() -> Evil*() -> D() -> E(code) -> binary
how in the world would the evil compiler in this situation inject something malicious into the final binary?
But even my dumb self makes the correlation that in 2020/2021, we handed out free money to keep people afloat (a very good thing), and then immediately following there was a surge in inflation.
So I guess I don't understand, how do you give out free money without devaluing the currency? Am I making an incorrect correlation between the stimulus checks and the subsequent inflation? Again, I don't know anything about this topic and I think the stimulus checks were a good idea that kept a lot of people afloat, but was that not the cause of the subsequent inflation?
Is the input/output of these models any different? Are they all just "text context goes in, scores for all tokens in the vocabulary come out" ? Is the difference only in how they achieve this output?
Is FlashAttention simply a drop-in replacement for the attention operation in an LLM? Can it be used anywhere that an "attention" operation is used? Or does a LLM need to be trained specially to use FA?
How does FA relate to attention strategies like GQA (grouped query attention) or sliding-window attention? Are they orthogonal concepts? Or you need a specific FA implementation for each strategy?
Recently llama.cpp added flash attention support - does this just mean they started consuming a flash attention-provided CUDA kernel or something?
lastly, in this post, they compare FlashAttention to Triton. I thought Triton was like an abstraction layer? Couldn't FA be implemented in Triton? I just don't really get what it means to say "FlashAttention vs. Triton".
(not saying this out of any support or non-support for such a GPU blockade; I'm just genuinely curious)