This is a great example of how to make a minimal app bundle with Swift. Thank you!
All an App needs on MacOS seems to be a binary and a little .plist
6,920 karma · joined January 1, 2010
All an App needs on MacOS seems to be a binary and a little .plist
1. Reduce the number of parameters
2. Reduce the resolution of each parameter (quantization)
For 1, changing the architecture is typically only possible by the labs producing the models, which is why each OSS model release tends to feature a small number of carefully chosen model sizes (for example, Gemma4 comes in e2B, e4B, 12B, 26Ba4B, and 31B sizes).
Generally, models with higher parameter counts have more world knowledge. For coding models, this shows up as a stronger command of uncommon libraries/languages. Very small models (<20B) also lack “smarts.”
Reducing the resolution of each parameter is easier which is why lots of practitioners have their own quantizations, but this makes it harder for a model to “think” fluently. Interacting with heavily quantized models feels like interacting with someone who didn’t get any sleep the night before.
Models that have higher-fidelity quantization take more RAM and have higher “smarts,” but don’t necessarily have more world knowledge. Models with aggressive quantization tend to be more likely to make rookie mistakes, emit malformed tool calls, get stuck in loops, or even exhibit signs of “neuroticism” / “distress” in their thinking tokens.
Parameter counts = world knowledge, quantization = “smarts.”
This is a soft rule of thumb, the difference isn’t very strong.
Subnormal floating point shenanigans maybe?
The image contains the previous block’s hash.
Wouldn’t this establish both a lower bound and an upper bound on the time the image could have been produced?
Like most projects, some indie productions (like TADC) end in creator burnout, others (like iron lung) end in moderate success, most probably end somewhere in between. We saw this with undertale, lights out, hazbin hotel, primer, el mariachi…
I guess today’s post represents a change.
I don’t have any public evidence to support my claim, sorry. Take it or leave it
Since Microsoft owns GitHub, their decision to shut down Windows 7/8 action runners is a sneaky way to help drive customers toward Windows 11.
Real-world systems need to be able to say "I don't know." This is a test about misinformation after all, and overconfident responses contribute to that.
Teasing out the difference between "avoid" and "unknown" could be a different research question
TL;DR Ed argues that the deal between Anthropic and xAI could have been negotiated in such a way as to make Anthropic only appear profitable during its “ramp-up” period in June, which incidentally is also the month that Anthropic is making tons of other pricing changes.
However, I've found that speculative decoders don't help much if you're running a model locally on limited hardware (for instance, my 32GB VRAM M1 Max from 2021). For one, you have to fit both the large and the small drafter model in memory. For another, if you're running a quantized model, the activation distribution is different enough that the draft model has a hard time guessing what's coming next.
My take is that speculative decoding is most useful on _very expensive_ prosumer/hobbyist setups where you have 128GB of VRAM and are running your local models with full fidelity. It's also helpful for inference providers where they can send output tokens at a computational cost slightly higher than their input token cost.
When Claude acts up, my strategy is to rewind the conversation to the point where the misunderstanding started, revise what I said just before, and then continue from there. I’ve found that letting one mistake enter the conversation seems to make further mistakes more likely.
The user already has omnipotent power over the agent’s sense of time and memory. We can rewrite what Claude sees and hears. Can’t do that with humans, but rewinding such an important function in Claude that it has a top-level keystroke.
Why spend time, tokens, and cortisol arguing and demanding the pet rock step through an apology protocol?
If you don’t read the tests to check they confer your intent or specifications, they’re more like tautologies than tests, you know?
LM-Studio doesn't support certain parameter combinations. For instance, LM-Studio supports KV quantization....but if you're using the MLX backend, you can't set the context length when KV quantization is used? Why? Running a model with certain settings requires keeping a little SAT solver going in your head. I found that overwhelming, so I just stopped using it.
The Ollama devs want to offer a central curated experience, but I perceive their approach as "playing fast and loose." They've re-implemented unique code for every model they support in their own Go runtime, so certain parameter choices aren't supported. On my hardware, their MLX backend just doesn't work at all without segfaulting the server process for example. It doesn't smack as vibe coded the way oMLX does, but it also doesn't smack as professional or battle-tested.
Ultimately, just dropping down to llama-cpp's GGUF model support and asking for default settings has provided faster inference speeds than anything I've been able to benchmark with them, but everything's within 10% of each other anyway so it's not a huge deal for me.
If you’re curious about what a particular switch does, clone the llama-cpp repository to your computer and try asking your favorite pet rock prompts like “This is llama-cpp. Can you look at what the -ctk parameter does and explain to me?” Giving Claude/codex/whatever access to the actual code goes a long way, but it is just one opinion.
If you’d like to learn how transformer-based language modeling works in detail, I suggest starting with chapter 0 or 1 of https://arena-chapter0-fundamentals.streamlit.app/ depending on your skill level, then use that to work your way to reading research papers.
Graduate students who study these topics are generally as annoyed by the “get rich quick” style of advertising as you are, so the deeper you go toward academic research the quieter those voices tend to get, mercifully. That said, this is balanced by the unfortunate fact that top labs have strong posturing signals they try to send, so it can be hard to see which preprints actually have good ideas, which are trying to promote their group’s tech instead of doing science out of curiosity, and which have authors who’ve innocently deluded themselves into overfitting their own pet projects. Read widely but adversarially, test everything but hold fast to the good stuff, etc etc