MitziMoto

Ah, the old "Do your research!".

Yeah, I don't know what model that guy is using. He's talking about Sol like it's GPT4.

I haven't had these issues in multiple model generations of models.

Failing to understand English grammar? Give me a break.

You can override it to 24 hours:

https://dev.meta.ai/docs/prompt-caching#cache-retention

Even at 5 minutes, if you're doing 100 agent runs in those 5 minutes, and 1 of them bills at full input price, it still hardly matters.

These are also the orders of magnitude of our production agents for our business (NOT coding). Cache reads are so heavy compared to anything else that it's the only price point that really matters, regular input and output are negligible.

I need aggressive cache read pricing with full prompt_cache_key support to have a model be financially viable for our workload. Right now Meta Muse 1.3 Contributor is the only one that makes sense--but we are starting Evals on the new MiMo 2.6 class to see how it holds up.

We use a Twilio integration to make it work over a real phone line, but it could just as well be an app or websocket connection in a browser.

Voice transport medium aside, the actual use case is not quite what you described.

Companies like Eleven Labs and Vapi provide a full end to end voice agent platform. They handle the STT -> LLM -> TTS pipeline and infrastructure for voice agents. Think of a customer talking to a virtual receptionist to schedule an appointment.

On the ElevenLabs platform, you provide them a system prompt (or an entire workflow/graph of system prompts) that instruct the voice agent on how to talk to the customer, tone, guardrails, how to answer specific questions, etc.

At some point we need that LLM agent, running on an infrastructure we don't control, to talk to our "CRM" (for simplicity sake). Enter MCP.

The MCP server we build and host supplies the "tools", like list_appointments, schedule_appointment, cancel_appointment-- whatever they may be.

When eleven Labs voice agent connects, it sees all the tools available and can use them per the instructions in the system prompt.

We use MCP in production for our customer facing voice agents. Our custom MCP server defines tools and resources that the voice agents need access to to interact with our customers. (E.g. scheduling appointments, checking order status, etc).

Now we can point any voice agent platform we choose -- eleven labs, vapi, pipecat, whatever -- at our custom MCP server and it instantly has an understanding of the tools available, their inputs, and how to use them.

Compared to the alternatives everyone on HN champions, like clis and APIs, this is a no brainer. I'm honestly not even sure what the realistic alternative would even be.

Am I supposed to package and distribute a cli to ElevenLabs and ask them to use it? Give them a full API spec to implement for me?

I give them an endpoint and credentials and their platform instantly knows how to talk to mine. No one at ElevenLabs knows or cares about our implementation details.

HN has trouble seeing past the "developer in a terminal coding with Claude Code" use case for using AI. Real production agents have use cases that are very different!

When you don't own every piece of an integration with another system, there needs to be a well defined standard. That's what MCP provides.

Some of us own and run companies? Cost per performance is a huge deal.

The fact that we don't know is part of the problem. Subscription usage has always been pretty opaque.

5.6 class models have a brand new caching architecture. Make sure you were using the prompt cache breakpoints and cache keys correctly - it's much different than < 5.5 and makes a huge difference for many production workloads.

I heard you have some free time now, so let's see it!

Grok's look like a truly disturbed child. Like the drawings from that kid in "The Ring".

As far as I can tell, these ads still look like ads, clearly marked as sponsored?

Something similar has been happening to me with Ghostty, but there is no error message or link and it only happens when Claude uses it's interactive questions interface.

It becomes completely unresponsive to any input except scrolling. I can't select options or even cancel out.

(I am in no way implying this is related to Bun)

Although the site is very fast for me as of this writing, it looks like they're running into an issue:

https://lobste.rs/s/ko1ji1/lobste_rs_is_now_running_on_sqlit...

I hope they get it sorted soon, I'm rooting for sqlite.

Their largest one, which is smaller than a standard piece of paper, starts at $200 and goes up to $270.

For a notebook. With what might be proprietary refills.

What.

Hey, don't forget: Google is also bad at search!

What a save! What a save! What a save! (Chat disabled for 3s)

I'm no longer in corporate America, so maybe I'm out of touch a bit, but could you just...not...use an LLM? You can still solve interesting problems on your own if you choose to do so?

Also building voice agents and have found GPT 5.4 with no thinking to be the sweet spot for latency vs intelligence vs cost.

GPT 5.5 with no reasoning is actually slightly faster, and much smarter, but too expensive.

What I'm really looking forward to are the next gen speech to speech models. gpt-realtime-2 is almost there, but not quite good enough for our use case. 5.4 actually beats it on answer latency even cascaded with stt/tts.

I'm trying not to be the "you're holding it wrong" guy, but ... have you just tried telling it to explore the codebase for things it might break?

Yeah, so was building moon rockets.

You static typed evangelists have lost your damn minds. You seem to have completely misunderstood what this library even is because you have some primal urge to boast static typing at every chance.

You can build high quality software with dynamically typed languages, and Ruby is an absolute dream to read and write.

We use and love RubyLLM! A wonderful and easy to use framework.

Agreed with another commenter on the frustration with the responses API not being naively supported; that seems like a huge miss. There is a connector from another dev, but it's buggy and not as high quality as the main gem.

Really looking forward to future development and especially 2.0!

Edit: Just saw that responses API is now native? I will definitely check that out.

I think something the article glazes over is that all these AI search results, whether it's Google AI Overviews, ChatGPT, or whatever, are mostly synthesizing and summarizing existing search results from classic search algorithms.

If the results the LLMs are grounded in haven't changed, then LLMs haven't changed SEO much.

...except, of course, stealing clicks from the actual creators.

Exactly. Not Starlink, but Tesla, their stupid non AI allocation tracker said my model X would be ready in a month for about 13 straight months (Early Model X plaid allocation)

I think you are wildly underestimating how much of a hard on people have for calling businesses, even when there are better and faster options clearly available, like clicking a button online.

Staffing our phone lines is the absolute bane of my small (non tech) business.

It's crazy to me that someone as distinguished as PG, as well as so many of the comments here on HN, completely missed this point.

So a Tesla Roadster? :)

Exactly! Many Ferraris of the past have gotten single digit MPG, no one cares. All of a sudden they have to make a Chinese looking EV because of "efficiency"? Give me a break.

Litestream exists?