489 karma · joined May 3, 2017
my company was easily spending $1m/month on tokens
sonnet 5.5 drops - we barely cracked half this month
it has to go down too, carbon is interesting, china has qwen 4 soon
coding isn’t sustainable growth - RLRVF is really effective for software development - labs are cracking the code, these models will have to get better at other things to be worth it
article seems very salty and im not sure why
also the economics is you open source something and its saves every one money and opens up more cross compile support, i bet this will scare companies to support linux more properly before linux users just start getting very good at cloning
i don't really think distilling is even that fundamental anymore (nor was the evidence ever that strong - https://msukhareva.substack.com/p/is-kimi-k3-really-distille...), the game is all RL now, and you don't need american progress for that at all (just compute and data, and the data brokers will sell to anyone no problem)
i do think there are black market proxies from china to the US trying to sell "claude" and "deepseek" access mixed together, and i do think that prompt distribution is probably going to be very nice datasets
there was never a need to rush other than for capitals self interest to control
if frontier labs wanna pace the frontier, they should be honest and return the compute back to the people
the ds4 quants were very good beating the unsloth quants https://github.com/michaelasper/benchmarks/blob/main/deepsee...
i make it for anything, i made one as a food guide for the UW chow down https://uw-chow-down-2026.corruptbytes.chatgpt.site/
you didn’t have to use China as an example, the US clearly does not care what its citizens want as the most popular policies are never even discussed or proposed in congress
meanwhile, China destroying their housing market to decommidify it so everyone can have housing…they seem to care about their people more
the sparks have much slower memory bandwidth is the trade off
i can imagine insane amount of capital is wasted on these two companies compared to the efficiency elsewhere