That paired with comments criticizing the Dear Leader were enough. This is a new low in corporate cowardice toward Trump bullying.
1,646 karma · joined February 14, 2022
Get in touch at hello@tombedor.dev, or at https://x.com/middleagedc0der
That paired with comments criticizing the Dear Leader were enough. This is a new low in corporate cowardice toward Trump bullying.
I suppose there's a distinction between new less capable models, I can see why those would be more efficient. But maybe the older frontier models are less efficient to serve?
An unknown to me: are the less powerful models cheaper to serve, proportional to how much less capable they are than frontier models? One possible explanation for why e.g. OpenAI was eager to retire GPT 4 is that those older models are still money losers.
> In fact, even if you remove the cost of training models from OpenAI's 2024 revenues (provided by The Information), OpenAI would still have lost $2.2 billion fucking dollars.
See also: Google Docs
I heavily discount same day commentary, there's a quid pro quo on early access vs favorable reviews (and yes, folks publishing early commentary aren't explicitly agreeing to write favorable things, but there's obvious bias baked in).
I don't think it's all particularly concerning, you can discount reviews that are coming out so quickly that's it's unlikely the reviewer has really used it very much.
I think one thing it does is help you get rid of the UX where you have to manage a bunch of distinct chats. I think that pattern is not long for this world - current models are perfectly capable of realizing when the subject of a conversation has changed
I think there is some degree of curation that remains necessary though, even if context windows are very large I think you will get poor results if you spew a bunch of junk into context. I think this curation is basically what people are referring to when they talk about Context Engineering.
I've got no evidence but vibes, but in the long run I think it's still going to be worth implementing curation / more deliberate recall. Partially because I think we'll ultimately land on on-device LLM's being the norm - I think that's going to have a major speed / privacy advantage. If I can make an application work smoothly with a smaller, on device model, that's going to be pretty compelling vs a large context window frontier model.
Of course, even in that scenario, maybe we get an on device model that has a big enough context window for none of this to matter!
Shameless plug for my project, which focuses on reminders and personal memory: elroy.bot
But other projects include Letta, mem0, and Zep
https://www.wsj.com/business/retail/de-beers-diamonds-price-...
In it he calls lab grown diamonds a "huge con", though doesn't really explain why...
I wrote a small shell function that loads the output commands as user input rather than executing them directly. This way, you can search through generated commands later!
If I'm building a local program, I am going to want tighter control over the toolsets my LLM calls have access to.
E.g. an MCP server for Google Calendar. MCP is not saving me significant time - I can access the same API's the MCP can. I probably need to carefully instruct the LLM on when and how to use the Google Calendar calls, and I don't want to delegate that to a third party.
I also do not want to spin up a bunch of arbitrary processes in whatever runtime environment the MCP is written in. If I'm writing in Python, why do I want my users to have to set up a typescript runtime? God help me if there's a security issue in the MCP wrapper for language_foo.
On the server, things get even more difficult to justify. We have a great tool for having one machine call a process hosted on another machine without knowing it's implementation details: the RPC. MCP just adds a bunch of opinionated middleware (and security holes)
It feels very akin to the Uber vs Lyft situation, two companies with very different perceptions pursuing identical business models
The biggest long term competitor to Anthropic isn't OpenAI, or Google... it's open source. That's the real target of Amodei's call for regulation.
I tried to have an LLM fully write a Python library for me. I wrote an extensive, detailed requirements doc, and it looked close enough that I accepted the code. But as I read through it more closely, I realized it was duplicating code in confusing ways, and overall it took longer than just getting the bones down myself, first.
Some coding agents are now more actively indexing code, I think that should help with this problem
LLM's are so-so coders but incredible teachers. Today's students get the benefit of asking copying and pasting a piece of code into an LLM and asking, "How does this work?"
There's a lot of young people that will use LLM's to be lazy. There's also a lot that will use them to feed their intellectual curiosity.
But with actually checked in code, the tradeoff in readability is pretty substantial