I don't think the models are doing this, time to first token is more of a hardware thing. But people writing agents are definitely doing this, particularly in voice it's worth it to use a smaller local llm to handle the acknowledgment before handing it off.
It's weird they don't document this stuff. Like understanding things like tool call latency and time to first token is extremely important in application development.
OpenAI made a huge mistake neglecting fast inferencing models. Their strategy was gpt 5 for everything, which hasn't worked out at all. I'm really not sure what model OpenAI wants me to use for my applications that require lower latency. If I follow their advice in their API docs about which models I should use for faster responses I get told either use GPT 5 low thinking, or replace gpt 5 with gpt 4.1, or switch to the mini model. Now as a developer I'm doing evals on all three of these combinations. I'm running my evals on gemini 3 flash right now, and it's outperforming gpt5 thinking without thinking. OpenAI should stop trying to come up with ads and make models that are useful.
For my apps evals Gemini flash and grok 4 fast are the only ones worth using. I'd love for an open weights model to compete in this arena but I haven't found one.
I'm more curious how Gemini 3 flash lite performs/is priced when it comes out. Because it may be that for most non coding tasks the distinction isn't between pro and flash but between flash and flash lite.
It's important in a democracy for people to pay taxes, it includes them in the system in a way that doesn't exist in a place like Saudi Arabia. We shouldn't aim for a bread and circuses society, but for one where we aggressively commoditize technology for the benefit of everyone.
I guess this was very American of me, but when I was so sick I wanted to know if my travel insurance would cover the hospital stay. ChatGPT confirmed that it did, and told me to get to a hospital. Ultimately the hotel owner was the person who carried me to the hospital, but i wasn't lucid enough to read through my travel insurance's benefits pdf. I suppose I should have just gone to the hospital with or without insurance, but sometimes when you're very sick you don't think straight.
I hate to be cagey here but I just really don’t want to make anyone’s life harder than it needs to be by revealing their identity. Microsoft is a really tough place to be an employee right now.
Thanks for signing up. I’m going to try really hard to open up some beta slots next week so more people can try it. There’s some embarrassingly bad bugs in prod right now…
Rafael was the absolute best. He also made sure the hospital saw me right away since I was so weak. But once I was hooked up I used ChatGPT to scan the ivs they had me hooked up to since I had no idea what they were pumping into me since it was all in Spanish.
I feel the same way about those embarrassingly solved problems! Though oftentimes the trick is knowing what to ask for. I remember grinding for weeks on a front end but until I realized what the problem was (not the exact bug just what the general concept should be) Claude then fixed it in 10 seconds.
I’m glad it resonated. I’ve found a lot of people in Microsoft have some shared struggles right now. It’s really hard get excited about jobs after that, but you only need one job to be the right fit. It sounds like you were working on some great stuff and you should keep pursuing that interest in the meantime. You never know where it might lead you.
That’s why I built all my geo infrastructure from scratch from osm, there’s still some issues but for AI location grounding it outperforms google places for $300/mo
The hotel owner eventually half carried me to the hospital because I got so weak from dehydration, though I'm glad I left my hotel room when I did I had difficulty avoiding fainting.