AI Device Template Featuring Whisper, TTS, Groq, Llama3, OpenAI

github.com

24 points·indigodaddy··9 comments

Open articleView on HN

The best AI device is your phone or laptop. I simply don't understand the reason for a dedicated device.

Maybe to prevent access to data you don't want to give to an AI.

it's not big enough to run a local LLM

But the Cloud LLM can't get what's not on the device.

One wrong permission and your pictures, contacts etc get uploaded.

So you only put on the device what you want to upload anyway.

I'm seeing all of these "AI in a box" experimental products, and none of them are "in a box".

The AI is always server-side. So the device is useless.

Whenever someone figures out how to package the GPU (or LPU, TPU, *PU) into a wearable or portable "box", that device will finally make sense.

... and it will most likely just be a smartphone.

Yes, perhaps the biggest ai tragedy is that it makes everyone think they can jump to "throw data into the cloud! ship it boys!" instead of actually designing a real working system.

So for something like this: It has to start with minimal hardware for voice recording, some local small computer just capable enough for simplest local NLP processing. A common open protocol for mesh communication and other local offline hardware to hardware features. Then it should decide when its necessary to reach out and communicate one to one, or one to a group, or send the command up to AI cloud processing because the lower layers cant do whats being asked.

I’m almost more interested in that search api. Unlike others it’s not monthly sub but valid for 6 months. Much better pricing structure

You should try Exa.ai API

That just has the same meh pricing model as all the other providers. Ie 50 to 100 spend minimum.

Reply on news.ycombinator.com