I legitimately dont understand why anyone would want a 4B model.
They might as well call all models 4B and smaller after psychedelics because they be hallucinating.
They might as well call all models 4B and smaller after psychedelics because they be hallucinating.
I wonder when the penny will drop?
I grabbed qwen3:4b. Cranked it to the max of 32k tokens.
It's fast to be sure, and im struggling to get it to hallucinate; but it is giving me a ton of 'The provided context does not include specifics'
Resource-wise its like running 12-16B, but faster. But soon as you expand the 12B to like 10k tokens it's clearly better for barely anymore resources.