Hacker News Reader

Comment by damascus | Hacker News Reader

Is anyone working on an Ender's Game style "Jane" assistant that just listens via an earbud and responds? That seems totally within the realm of current tech but I haven't seen anything.

View on HN

I’ve been thinking about this and I’d go a step further. I feel that current iterations of digital assistants are too passive. They respond when you directly ask them a specific question. This leaves it up to the user to: 1. Know that an assistant could possibly answer the question. 2. Know how to ask the question. 3. Realize that they should ask the question rather than reaching for google or something.

I would like a digital assistant that not only has the question answering ability of a LLM, but also has the sense of awareness and impetuous to suggest helpful things without being asked. This would take a nanny state level of monitoring, but imagine the possibilities. If you had sensors feeding different types of data into the model about your surrounding environment and what specifically you’re doing, and then occasionally have an automated process that silently asks the model something like “given all current inputs, what would you suggest I do?” And then if the result achieves a certain threshold of certainty, the digital assistant speaks up and suggests it to you.

I’m sure tons of people are cringing at the thought of the surveillance needed for this and the trust you’d effectively have to put into BigCorp that owns the setup, but it’s fun to think about nonetheless.

This but with models running on my infra that I own.

Basically take this: https://www.meta.com/pl/en/glasses/products/ray-ban-stories/... And feed data from that to multiple models (for face recognition, other vision, audio STT, music recognition, probably a lot of other stuff has easily recognizable audio pattern etc.)

combine with my personal data (like contacts, emails, chats, notes, photos I take) and feed to assistant to prepare a combined reply to my questions or summarize what it knows about my current environment.

Also I would gladly take those glasses just to take note photos (photos with audio note) right now - shut up and take my money. Really if they were hackable or at least intercept-able on my phone I would take them.

Oh man, if i could run this on my own network with no internet access i'd do it in a heartbeat.

It would also make so many things easier for the AI too. Ie if it's listening to the conversation and you ask "Thoughts, AIAssistant?" and it can infer enough from the previous conversation to answer this type of question.. so cool.

But yea i definitely want it closed network. A device sitting on my closet, A firewalled internet connection only allowing it to talk to my earbud, etc. Super paranoia. Since it's job is to monitor everything, all the time.

Then the police come and confiscate the device for evidentiary reasons, finding you have committed some sort of crime (most people have).

Well it's in your control and FOSS - ideally you're not keeping a full log of everything unless you want that.

Without a full log of everything, it cannot give context-aware advice tailored to you (i.e. useful advice). It'd be like relying on the advice of a random person on the street instead of someone who knows you.

It could encrypt everything and have a kill switch to permanently erase the crypt key.

"Alexa, forget yourself."

I don't mean infinite context. I mean context like a human would have, about the current conversation. Ie it just records everything in a streaming fashion, but deletes N old entries.

I have IP cameras on a private network but i don't infinitely record them either. It's just not of much practical use in the raw form.

Now if i want to have it take some concise, daily notes? I might choose to keep those forever. But the police can steal my daily journal too. I don't see how that's any different.

Ah yes I forget that most people get raided by the police on regular occurrences so anything on-prem has to be out of the question (/s)

Surely there'd be ways to make sure the data isn't accessible.

Bizarre, had same thought today.

My thought conclusion was that the assistant needs to know or learn my intentions.

From that it can actually pre-empt questions I might ask and already be making decisions on the answers.

Now what would that do to our productivity!

If I could harness both the near limitless knowledge of the internet and somehow filter it through my needs that could be used to train my assistant ( and I worry that Black Mirror already did an episode on that that was a little too direct ), I think that could be a viable product and I would consider buying. Naturally, I would need to feed it my thought patterns/writings and so on ( and then hope they actually reflect my reality ). Not exactly an easy task.

That said, it is a fascinating thing to think about.

It would be really easy to make such a thing as awful as Microsoft Office's Clippy ("It looks like you're writing a letter..."), and really hard to make it genuinely useful. I can imagine it being mostly annoying or unhelpful enough to be left turned off.

The monitoring and surveillance could in principle be avoided by running the whole thing offline on the user's hardware.

+1! That scene in Her in the opening where the guy is walking down the hall and going through his email, "skip that, unsubscribe from them, tell so and so I can get that by tomorrow..." without having to look at a screen had been a dream for me ever since I saw it.

Have you watched it recently? I haven't seen it since it came out, I think I'm gonna watch it again this afternoon and see how differently it hits

No. I watched it twice in one day and haven't come back to it since.

That movie broke my heart and I cried. I had recently been cheated on during a long distance relationship and there were just too many parallels with the reality of talking to someone you love on the phone - and finding out you hardly knew them at all.

My parents thought the movie was creepy and hated it.

Your parents are weird, have you considered getting a new Model, maybe ParentsGPT-4?

Don't have the link on me but I remember reading a blog post where someone set up ChatGPT with a STT and TTS system to converse with the bot using a headset.

The open source Talk to Chat GPT extension works remarkably well, and its source is on Github

https://chrome.google.com/webstore/detail/talk-to-chatgpt/ho...

I want this. I'd be happy with an earbud but I really want an embedded AI that can see and hear what I do and can project things into my optic and auditory nerves.

Rumor has it that Humane will release a Her style earbud soon. https://hu.ma.ne/

Reply on news.ycombinator.com