1,431 karma · joined March 14, 2016
This method aligns the transcript of a 60 hour book in 20 minutes on my GPU using Parakeet CTC 0.6B which is english only, but there's similarly sized models in other languages, the models don't need to be that good at transcribing to generate good alignments.
Image models are essentially a mapping of text embedding space -> image. So the clearer and cleaner the text embedding space is the better model you get as you need less weights trying to interpret a text prompt, recent image models even use multimodal versions of LLMs (Flux uses Qwen3 VL) so all the visual separation was done beforehand making the model even more efficient.
edit: Maybe a better analogy is that the image models are only half the model in a encoder-decoder sense, where the LLM is a pretrained frozen encoder and the image model is the decoder.
I know that sounds confusing, let me break down how I think about this.
1. LLMs don't pick the token that ends up being used. This is by design, if the LLM gives a wide choice, it can better adapt to real world scenarios. i.e. generalize.
2. Without reasoning, this means that the LLM either locks in on whatever the sampler picked. Or decides mid-sentence/response to correct itself. This is what used to happen before reasoning, still happens if you turn reasoning off.
3. With reasoning, the LLM can make as many mistakes as it wants and explore its sampling space. Then use its vast pattern matching capabilities to decide which parts of the reasoning make sense and which were idiot ideas.
4. Enabling reasoning makes it so LLMs are much more confident on the final response, and the logits should theoretically all be near 99% on a single token for every token, i.e. much closer to greedy decoding. It analyzed all the possible options and figured out the best outcome, so a stray sample doesn't cause the answer to go awry.
This is why reasoning traces are filled with "but wait". I don't know if those were added in organically or artificially in the RL training, but regardless they're a good way to let the LLM keep generating other options and explore it's sampling space to the fullest.
Note: I haven't tested any of this and it's just my theory, but I'm sure if you really wanna know you can have claude run some smoke tests :)
top_a is not very common and is better explained if I explain how the much more common min_p works. min_p filters out tokens below a certain threshold. The formula is <filter threshold> = <min_p> * <top token probability>. So if the top token has 0.5 probability, min_p = 0.1 would cut out tokens below 0.05. This is a tunable that lets you filter out other tokens depending on how confident the model is.
top_a is almost the same formula but you just square the <top token probability>. So <filter threshold> = <top_a> * <top token probability> ^ 2. This makes the filtering ramp up faster (cut out more tokens) if the model has a much more confident top choice, but keep more choices if the model is not so confident.
A models output is not a single token, but a list with the probability for all the tokens that it knows, so we need to use a sampler to select the token that it's going to be the next token in the sentence. For example a simple greedy sampler will choose the token with the highest probability, but samplers normally pick a random token weighted by probability. A model usually knows about ~250 thousand tokens and the probability of some of these tokens are gonna be high, but the vast majority is close to but not actually 0% so there's a chance the sampler might pick some random token that doesn't make much sense, so we filter tokens.
top_k filters the tokens so that only the k top tokens are selected. So top_k=50 will filter those 250k tokens to only 50. This is assuming the list of tokens is sorted by probability.
top_p filters the top tokens until a percentage is accumulated. So if for example if you set the top_p to 0.6 and the model gave the top token a 0.5 (50%) probability and the second top token a 0.2, those 2 token accumulated to 0.7 which is greater than what you set it to (0.6) so no more tokens are selected. If this ran after top_k=50 it'll turn the list of 50 tokens into one of 2.
After each filter parameter is processed, the probability of the tokens is adjusted to sum to 1 (100%), Also note that order of operation here matters, i.e. top_p could be applied before top_k, but most providers follow what's on huggingface, I think I've only seen different implementation in certain local model hosting frameworks.
Cloud models are usually protected by trade secret laws, leaking them would get you in trouble. However if the model is made available publicly, as long as you don't break the law to get them, anything after that would be fair game unless Apple can prove that humans have significant authorship over the weights, which hasn't been tested and is a significant burden to prove/disprove.
G1 and G2 are given names. Usually 2 "first names" that you see in english, but there's common combos and sometimes there's a word joining them. Examples: "Maria Jesus" vs "Maria de Jesus". Some names are more common to be put first, but almost every name can be put in any order, example: "José António" vs "António José".
FM and FF are easy. FF is the family name of your father (your father's FF), and FM is the family name from your mother (your mother's FF).
Where I was raised 99% of my friends had 4 names structured like this, I only knew a few that didn't. When I moved to Lisbon the 3 name structure was much more common, dropping the second given name.
In Portugal there's rules for naming your kids (at least there were when I lived there), but I think in Brazil such rules don't exist. The author is brazillian but his name seems to follow the traditional portuguese naming style, as you guessed his name in english could be translated to "Robert Anthony Smith of Almeida" (Almeida is a portuguese town).
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
When asking specifically from the video, it did fool the AI somewhat[1], but no link. I tried changing it to retrieve the revenue as that's probably a more sensitive/worthwhile metadata.
Stable Diffusion 3, an open weights model, was laughed at at release for not being able to even generate a woman laying in grass. The community attributed this to the heavy dataset filtering. Since then other open weights releases have been made with no NSFW capabilities and the community claims they're not as good as anatomy as well.
You can google "stable diffusion 3 woman in grass" and press the images tab to see how the model failed spectacularly.
It's a race metaphor not a football metaphor.
In foot/cycling races there's often a pack leader, that leader is often not the winner of the race, all they're doing is taking the brunt of the air resistance while everyone else slipstreams behind. For a casual observer it seems that the pack leader will win, but everyone knows that it's gonna be someone that paced themselves that's going to overtake the first spot at the tail end of the race.
What's the point of leading the race for 90% of it, if they're gonna slip on their own sweat and fall down by the end? In non metaphorical terms, what's the point of spending billions of dollars rushing to get the best AI tech at all costs, when the competition can distil your progress and catch up in 6-12 months while only spending 1% of what you spent.
Even in the aspect the article cares about, commercialization, the US is starting to lose marketshare, I've seen people move from cc/codex plans to use glm/opencode plans due to the recent squeeze the US companies put on plan usage, the US companies are screwed if that sticks, not everyone needs the bleeding edge models, they just want to pay $20/month and have the models be decently capable.