brrrrrm

I’ve definitely met people whose stats are juiced. Probably not as good at scrolling tiktok as me tho

This and the lack of infix operators was one of the main reasons Shumai stalled out. The language has so much to offer, but JS just isn't quite there for native ML coding

I think it's a trap of visual elegance. When you start thinking of models this way you miss the way a lot of models are actually written.

E.g. how do you represent an online fine-tuning process? I want to randomly switch between a reference impl and an approximation method, but when using the approx method I want to back-propagate so that it gets better over time.

full disclosure: I've written plenty of these little visualizers and also fallen for the trap of "everything should be a declarative graph."

I think it uses TF's graph construct which has that built in? it's like a weird mix of dataflow and control flow graphs.

I wonder if you could track provenance by operating at the highest layer of the dispatcher and capture any calls to GPU operations (ala Cuda Graph)?

> in the unused bits

I feel like this is a PT rite of passage :P

I've never really understood the point of these visualizer things. The idea that a model is always well represented by a directed acyclic graph seems extremely dated.

I really would love a PyTorch/JAX profiler that shows, in annotated Python, where your code is allocating memory, using compute or doing device copies.

A single 8xH100 node hits 15.6 fp16 petaFlops

Eye of the beholder I guess. I personally wouldn’t offer moral judgement on the uninvented

Have you been on r/localllama? I’d wager this tech will make it to open source and get tuned by modern creatives just like all the text based models. Individuals are a lot more empowered to develop in this space than is commonly echoed by HN comments. Sure the hobbyist models don’t crack MMLU records, but they do things no corporate entity would ever consider

I use it all the time?

if anything the controversy increased sales. I watched the ad and thought it was pretty cool

It mentions immersion methods and then never revisits it :(

I used to use a Chemex but found the whole process so fickle and involved. I've since switched to a Clever dripper (similar to the hario switch) and found that my coffee life has improved substantially. The basic idea is to let the coffee sit with the water for a bit and then flip a switch to drain it into your coffee. I prefer the consistency of the method and haven't noticed a difference in flavor.

My latest project has been attempting to speed up the cold brew process (to get low acidity cold coffee for the summer) and I've taken to using a sous-vide. The idea is that low heat will reduce the time needed for extraction without actually causing the water to leech acidity from the beans. It's worked, but results were kinda meh. I just bought some PH strips to run a more thorough experiment, though.

“Increasing control points” hides a lot under the covers here. Your answer and the paper provide virtually no reason to believe one type of continuous function approximation is better than another. The comparisons made are superficial and only serve to address contrived issues like representing sinusoidal function families concisely.

It’s weird to just ignore MLPs when approximating a continuous univariate function. But if the paper did use MLPs theyd have ended up with something that looks a lot more like conventional neural networks, so maybe thats why?

doesn't KA representation require continuous univariate functions? do B-splines actually cover the space of all continuous functions? wouldn't... MLPs be better for the learnable activation functions?

At small input size, yes the MLP dominates compute. At large input attention matters more

You can quantize KV caches

HN isnt really the best space for LLM news - r/LocalLlama and twitter are much better. I think HN has some cultural issues with “AI” news

Maybe depth rather than parameter count.

Yep

bf16

I've been trying to think about how you'd amp up the batch size here. it's a bit tricky since the memory access would be way higher, but I think you can actually still save on compute by chunking things up in a clever way to utilize the tensorcores

not sure why I got downvoted for asking this. seems like a reasonable thing in the ML space

anyway here's a numerical simulation written in PyTorch for those who want to consider this algo on their projects before fully integrating: https://gist.github.com/bwasti/78d7058aad7f42dc893d906c877b7...

happy hacking

do you have a simple python impl? :)

companies that run these things care - they run at huge batch size and are compute bound in the limit

Yea. It’s one less hop through slow memory

It doesn’t. It simply trades compute efficiency by transposing matrix multiplications into “the future.” It doesn’t actually save FLOPs (uses more) and doesn’t work at large batch size

to add to this, "bagging" tracks are determined by how many shortcuts they have (ones that require good items to take, such as mushrooms or stars).

this one groups the characters/cars by identical stats: https://www.bettermk8dxbuilder.com

that one doesn't have mini-turbo, which has largely superseded acceleration as a stat (it correlates but isn't 1:1)

using AVX/FMA and unrolling loops does extremely little in the way of compiling to fast (>80% peak) GEMM code. These are very much intro steps that don't take into account many important ideas related to cache hierarchy, uop interactions, and even instruction decode time. The Fortran implementation is entirely and unquestionably inadequate for real high performance GEMMs.