HNHacker News
TopNewBestAskShowJobs

cootsnuck

626 karma · joined May 22, 2017

Voice AI Engineer & Consultant

https://insidevoice.ai/

submissionscomments
cootsnuck··on Ontario auditors find doctors' AI note takers routinely blow basic facts
Not my term! Some real academics came up with it: https://www.normaltech.ai/p/new-paper-towards-a-science-of-a...
cootsnuck··on Ontario auditors find doctors' AI note takers routinely blow basic facts
I work specifically in voice AI and am very familiar with how these tools and systems work.

> I would expect an "AI Note Taker" to faithfully transcribe the entire conversation. With the same quality I see in a lot of automated video subtitles.. ie they use the wrong word a lot but it's easy to tell what they mean by context.

That's a reasonable expectation, but would not be a safe one. All transcription tools are not made the same. First it depends on what kind of STT/ASR (speech-to-text / automatic speech recognition) model they are using. A lot of tools like to use some flavor of OpenAI's Whisper model. It works well generally but I would never use it in a critical use case like healthcare. Because it can hallucinate. That's specific to its architecture and how it was trained.

There's a fairly large variety of architectures that can be used for STT/ASR. Some of them are designed for "offline" / "batch" / pre-recorded audio. Some are designed for fast real-time streaming transcription.

There are more factors too like training data. And not just demographics of the speakers in the training data but audio environments too. Was the model trained on echo-y doctor offices with two people being recorded from a crappy smartphone mic or desktop mic? (It could've been! But it's an important distinction.)

And there's more factors than that, but you get the picture (e.g. are they trying to "clean up" the transcript afterwards by feeding it to an LLM, are they attempting to pre-process audio before transcription also in an attempt to boost accuracy)

There's a lot of ways to do it, meaning, there's a lot of ways to screw it up.

cootsnuck··on Ontario auditors find doctors' AI note takers routinely blow basic facts
Yup, spot on. There's a capability-reliability gap that the industry does not like to talk about too much.

It often feels like the AI industry is continually glossing over the fact that capability and reliability are fundamentally different qualities. We tend to use "accurate" and "reliable" interchangeably, but they describe different things. A model can ace a benchmark (capability/accuracy) and still be a liability in production (reliability).

Just look at recent reactions to yet another release from METR showing improved capabilities. But the less talked about part is how their measure is for a 50% success rate (and the even lesser talked about secondary measure they have at 80% success rate has a drastically lower time-horizon for tasks). https://metr.org/

I implement AI systems for enterprises and I don't know any that would ever be okay with 80% reliability (let alone 50%).

cootsnuck··on Ontario auditors find doctors' AI note takers routinely blow basic facts
They absolutely need deterministic tools. What you just described is exactly how the current popular AI agents work. They use "harnesses", which to me is just a rebranding of what we have known all along about building useful and reliable software...composable orchestrated systems with a variety of different pieces selected based on their capabilities and constraints being glued together for specific outcomes.

It just feels like for some reason this is all being relearned with LLMs. I guess shortcuts have always been tempting. And the idea of a "digital panacea" is too hard to resist.

cootsnuck··on Cartoon Network Flash Games
Yes! I was just about to comment the same thing. I sank so many hours into that Dragon Ball Z game. Was called Dragon Ball Z Tournament. And its background music was an instrumental version of Sisqo's Thong Song. Wild.

https://www.youtube.com/watch?v=Cdaf8ehjuX4

cootsnuck··on Our agent found a bug with WireGuard in Google Kubernetes Engine
I think "AI-tone" is a much better way to characterize this stuff than accusing people of using AI. The problem has always been the same. Putting out slop feels disrespectful to the people you want to read/watch your stuff.

Makes me think of how pre-chatGPT I still could barely handle most recipe blogs because of their well known attempts at "filling space". And yea the problem is significantly magnified now everywhere else.

Anyway, my point is, whether or not someone uses AI is almost secondary in a way (even though it can seem pretty obvious to most of us when it's being used). All that matters is if the writing seems like it cares more about throwing words at people instead of actually conveying its points in a way to elicit understanding.

cootsnuck··on The FCC is about to ban 21% of its test labs today. I mapped them all
Their comment does not give off "LLM-written" really... It drives forwards actual points without superfluous segments. I don't think it's helpful to try and discredit people whenever we want by throwing around accusations of "LLM-written".
cootsnuck··on Online age verification is the hill to die on
Sam, we're not going to use your weird eye scanning orb.
cootsnuck··on Online age verification is the hill to die on
Because you can't have a tech company offering third party identity verification solutions if you just go with something like an RTA header.
cootsnuck··on Cal.com is going closed source
About the friction, not the capabilities...I haven't switched off my biz calendar/appointment provider I'm paying for even though I've kinda outgrown it.

I wouldn't under estimate switching friction.

cootsnuck··on Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
Yup, you're absolutely right. The open source models do have their rough edges. I use NVIDIA's Parakeet v3 model a lot locally, and it will occasionally do this thing where it just repeats a word like a dozen times.
cootsnuck··on Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
Handy has Windows support. https://handy.computer/
cootsnuck··on Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
Yup, Handy is the one that made me stop looking for local open source alternatives to Wispr Flow.

I'll give a shoutout as well to Glimpse: https://github.com/LegendarySpy/Glimpse

cootsnuck··on Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS
Interesting. My Pixel 7 transcription is barely usable for me. Makes way too many mistakes and defeats the purpose of me not having to type, but maybe that's just my experience.

The latest open source local STT models people are running on devices are significantly more robust (e.g. whisper models, parakeet models, etc.). So background noise, mumbling, and/or just not having a perfect audio environment doesn't trip up the SoTA models as much (all of them still do get tripped up).

I work in voice AI and am using these models (both proprietary and local open source) every day. Night and day different for me.

cootsnuck··on Tinybox – A powerful computer for deep learning
How many businesses have the capabilities and expertise to train their own models?
cootsnuck··on Show HN: How I Topped the HuggingFace Open LLM Leaderboard on Two Gaming GPUs
Super cool. Love seeing these writeups of hobbyists getting their hands dirty, breaking things, and then coming out on the other side of it with something interesting.
cootsnuck··on Show HN: DenchClaw – Local CRM on Top of OpenClaw
Yea, it has been a little shocking to me that the rising narratives around "AI agents everywhere" and "enable the web for AI agents" requires what we've all been wanting for awhile on the web (openness and interoperability) but that the same big players in tech have been clearly against for a long time. Like the fact that Google recently released that Google Workspace CLI (https://github.com/googleworkspace/cli) is a perfect example.

They could've released something like that years ago (the discovery service it's built on has existed for over a decade) but creating a simple, accessible, unified CLI for general integration apparently wasn't worth it until agents became the hot thing.

I wonder when / if there will be a rug pull on all of this. Because I really don't see what the long-term incentives are for incumbent tech platforms to make it easy for automated systems to essentially pull users away from the actual platform. I guess they're focused on the short term incentives. And once they decide the party's over, promising upstarts and competition can get absorbed and it'll be business as usual. Idk, we'll see.

cootsnuck··on Show HN: I built a sub-500ms latency voice agent from scratch
In OpenAI's own words about semantic_vad:

> Chunks the audio when the model believes based on the words said by the user that they have completed their utterance.

Source: https://developers.openai.com/api/docs/guides/realtime-vad

OpenAI's Semantic mode is looking at the semantic meaning of the transcribed text to make an educated guess about where the user's end of utterance is.

According to Deepgram, Flux's end-of-turn detection is not just a semantic VAD (which inherently is a separate model from the STT model that's doing the transcribing). Deepgram describes Flux as:

> the same model that produces transcripts is also responsible for modeling conversational flow and turn detection.

[...]

> With complete semantic, acoustic, and full-turn context in a fused model, Flux is able to very accurately detect turn ends and avoid the premature interruptions common with traditional approaches.

Source: https://deepgram.com/learn/introducing-flux-conversational-s...

So according to them, end-of-turn detection isn't just based on semantic content of the transcript (which makes sense given the latency), but rather the the characteristics of the actual audio waveform itself as well.

Which Pipecat (open source voice AI orchestration platform) actually does as well seemingly with their smart-turn native turn detection model as well: https://github.com/pipecat-ai/smart-turn (minus the built-in transcription)

cootsnuck··on Show HN: I built a sub-500ms latency voice agent from scratch
I've been working solely on voice agents for the past couple years (and have worked at one of the frontier voice AI companies).

The cascading model (STT -> LLM -> TTS), is unlikely to go away anytime soon for a whole lot of reasons. A big one is observability. The people paying for voice agents are enterprises. Enterprises care about reliability and liability. The cascading model approach is much more amenable to specialization (rather than raw flexibility / generality) and auditability.

Organizations in regulated industries (e.g. healthcare, finance, education) need to be able to see what a voice agent "heard" before it tries to "act" on transcribed text, and same goes for seeing what LLM output text is going to be "said" before it's actually synthesized and played back.

Speech-to-Speech (end-to-end) models definitely have a place for more "narrative" use cases (think interviewing, conducting surveys / polls, etc.).

But from my experience from working with clients, they are clamoring for systems and orchestration that actually use some good ol' fashioned engineering and that don't solely rely on the latest-and-greatest SoTA ML models.

cootsnuck··on Show HN: I built a sub-500ms latency voice agent from scratch
Yea, Deepgram Flux is the secret sauce. Doesn't get talked about much.

For anyone curious: https://flux.deepgram.com/

cootsnuck··on We do not think Anthropic should be designated as a supply chain risk
Except an LLM actually is a piece of software. And the brain is not what you said.
cootsnuck··on Microgpt
It was: https://news.ycombinator.com/item?id=47000263
cootsnuck··on Discord cuts ties with identity verification software, Persona
"Good" is subjective. But yes, all wealth creation requires working with other people. No one is an island. And most people are increasingly disturbed by the types of decisions required to amass more wealth than sovereign nations.
cootsnuck··on Discord cuts ties with identity verification software, Persona
Yea, it's puzzling to me that this isn't asked of folks like Altman and Amodei in every interview. Maybe it's because Altman would just start shilling his eye scanning orb and start repeating "WORLD COIN" ad nauseum. Either way, they should be getting pressed on this by all media.
cootsnuck··on Nvidia and OpenAI abandon unfinished $100B deal in favour of $30B investment
Ed is the anger translator in my head. Good stuff.
cootsnuck··on "Token anxiety", a slot machine by any other name
Fellow Midwesterner?
cootsnuck··on 14-year-old Miles Wu folded origami pattern that holds 10k times its own weight
Totally agree. Adult life is just mentally taxing. I'm more curious and more eager to learn now in my 30s than I was in any of my schooling. The learning isn't hard but the energy regulation is.

I think it's so easy for people to discount "mental energy" since culturally we don't often acknowledge it as a finite resource the same way we do physical energy. Well maybe the problem is we view them as separate things in the first place.

When I was younger I just didn't have to worry about so much stuff.

cootsnuck··on IBM tripling entry-level jobs after finding the limits of AI adoption
They said they're going to invest like $150B over five years. Which is quite a bit smaller than other big tech firms.

They have their Granite family of models, but they're small language models so surely significantly less resources are going into them.

cootsnuck··on Ask HN: The Coming Class War
There's no "end" per se, but shifting dynamics. I personally think we are currently seeing a "slow", but noteworthy-in-hindsight, shift in interest and development of alternative options and platforms from the big tech monopolies. From things like Bluesky, Framework laptops, minimalist cell phones, and even smaller local language models and other types of useful local ML models.

I don't think it's going to be anything like 50% or even 30% of users using non-flagship hardware or software products. But it could still be significant. And I think the more important thing isn't going to be market share as much as proof of viability. More successful examples will beget more.

It's about planting seeds from which future digital ecosystems can grow -- that have interoperability, functionality, and openness built into their foundations.

I believe that what drove you to make this post and the way I feel is not unique and are part of a larger swell in similar sentiments.

You throw in other factors too like the mass tech layoffs and the continued doubling down of tech barons on their cravings to intermingle with the surveillance state and military industrial complex... I just can't see how the future doesn't have more people disillusioned with the current state of the tech industry.

I think big tech will continue to overplay their hand and the mess that comes after will be an opportunity to give people what they want and show alternatives to what's already been done that we know won't work out.

cootsnuck··on The AI boom is causing shortages everywhere else
They got the current administration to ban state level regulation for them. Not to mention various defense contracts. They are government subsidized.
← PreviousPage 3 of 10Next →