977 karma · joined April 4, 2023
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
So far humans failed at those problems. Also IIRC there was a guy that proved a substantial problem 2-3 months ago by basically pasting over and over "keep looking for a solution" or something like that for 2 days with little formal math background.
https://www.france24.com/en/live-news/20221027-israel-s-far-...
Also interfering to read: https://en.wikipedia.org/wiki/Baruch_Goldstein
Asymetrical (shape of the mole)
Border (irregular? Unclear delimitation?)
Color (has it multiple colors?)
Diameter (has it a >6mm diameter? An indicator that it might start vertical growth iirc)
Evolution (has it changed recently?)
I'm interested in your take on it. IIRC Gemma family models too have a ~250k vocabulary size
Page 51 is interesting:
> What matters is not the eggs It is the Goose itself True value = The power to keep laying eggs
That's a big claim. I believe on the other hand that biology is applied chemistry which is applied quantum physics. And frontier models are so good at biology + biochemistry + chemistry + quantum physics that it's bound to trickle down into biology by sheer knowledge.
And secondly, even without pipettes i'm pretty sure computer simulation can teach us a tremendous amount still.