Once the subsidization ends and cost becomes significant I will take a serious look around for the best value models and switch off the expensive providers, but that time hasn't come yet.
2,076 karma · joined December 30, 2022
Once the subsidization ends and cost becomes significant I will take a serious look around for the best value models and switch off the expensive providers, but that time hasn't come yet.
But there also is another type of discovery that requires taking a step back and looking at the problem from a different angle. If you are an engineer, how often has an LLM told you (without you explicitly prompting for it): Wait, what you are doing here doesn't really make sense, there exists a much more elegant abstraction that nobody has thought of, let's remove all that code, let's tackle the problem in a different way by thinking from first principles. Pretty much never. But in science a lot of the biggest discoveries have come from this kind of first principle thinking, questioning existing work and approaches and going against what already exists, not combining all existing data which is likely to be just a local optimum.
That doesn't mean it's not a bubble. Good enough models are commodities now. If OpenAI and Anthropic disappeared tomorrow I would be a bit sad because their models are maybe 10% better than the others, but it wouldn't matter that much really.
Another useful property is the readonly nature. I can easily let multiple agents plan in parallel without having to worry about annoying worktrees or conflicts and then I can come back to the plans later.
Of course this can be done with just another prompt, but that's exactly what plan mode is. It's nothing more than a predefined prompt in the harnesses with maybe some extra guardrails (that don't always work)
I also use Astra at work where I don't need to worry about token cost on highest effort and same story there, I don't see any difference in everyday work other than it being more expensive. Of course my experience is highly subjective, but with how meaningless/overfit the benchmarks are, subjective experiences are imo what matters.
But of course, 10 years ago this wasn't obvious.
I download the raw audio track with yt-dlp, give it to qwen asr for raw transcription, then DeepSeek flash for transcription cleanup, formatting, corrections with web search enabled, etc. I use openrouter to call the models. It's a few cents per video (mostly the asr model at ~12 cents per hour) and quality is great. You could probably use a local asr model if you want to save money, I just don't bother because it's so cheap already.
That only makes sense for videos that don't have official (not auto-generated) transcripts of course. If there are official transcripts that are good you can just get them directly with yt-dlp.
That is still context switching for me. I do the same. While the agent works, which may be 10 minutes, I start planning follow up tasks, etc. But then eventually I still need to come back to the original task to review the agents work.
I don't switch across projects usually. But I can never get into a flow state like that. But I also don't want to sit there for 10 minutes doing nothing
Especially in the age of LLMs I think a lot of us hate the current tools that force you to context switch between N agents and terminals. Constant context switching takes the joy out of work and prevents you from getting into a flow state. I hope someone figures out the solution.
I understand it has the agent pane on the left that tells you what is running but... that's it?
A frontend dev doing tailwind integration for his day job is gonna see very different speedups than someone working in a niche scientific codebase. Taking the average makes about as much sense as taking the average of the speedup from calculators for a mathematician, a farmer, and an elementary school student.
A junior employee doesn't have that knowledge, so they are expected the follow rules to ensure that they don't inadvertently break something.
What exactly are these dozens of breakthroughs? Most frontier models architectures today still look very much like GPT2 at their core. There were various of improvements like instructgpt, finetuning techniques, efficiency improvements with kv caches, faster attention, lora, better tokenizers, etc. Most of these are for making things run faster. The biggest differentiator has probably been data curation and post-training data and the ability to fit more into the model. But I think we had few breakthroughs that would fall into the category of different technologies.