Hey, Alexa, What Can You Hear? And What Will You Do With It?
nytimes.com
nytimes.com
This is why we need AI at the edge, and not in the clouds, and Privacy-by-design thinking in our architectures. This is the only way for people to know their data won't be compromised and misused, because it never leaves their devices.
Disclaimer: I'm a co-founder of https://snips.ai and we are building a 100% on-device and private-by-design platform to build Voice AI assistants
We would be happy to know what you do with it! You can take a look at what some people have built with it already https://github.com/snipsco/awesome-snips
We are open-sourcing it over time, starting with the NLU: https://medium.com/snips-ai/snips-nlu-is-an-open-source-priv.... Snips is available in English, French, German, and soon Japanese and Korean with more European languages coming this year.
You can start building your own private-by-design smart speaker on the platform in under 1h with this tutorial: https://medium.com/snips-ai/building-a-voice-controlled-home...
Excellent! How will you guarantee that your devices will not leak data?
We are providing the source, and people can rebuild it from scratch and ensure that their device is running the open-source code. In theory, it is each user who should deploy his device. If an industrial partners want to ship our software to many devices, indeed it will be needed for them to prove he hasn't modified the software and we are happy to learn about best practices to help them do so, so we will look at this
This might be a good starting point:
Yea, as a privacy conscious user this would be the attack vector that I’d be most nervous about. “Partners” taking the open source engine and loading it up with surveillance. Being able to re-compile and deploy a vanilla version from source is critical. There are a lot of “smart” devices out there that I would buy if only I had root and/or could decide what software ran on it.
edit: I thought you were also doing something similar to Mycroft and their 'mark 1' thing. I was wrong.
It will be open-source and free for all users to use and create assistants.
If an industrial device maker wants to use the open-source version and add to it proprietary components, it will be his decision and he should inform his customers.
The best we can do when building and open-sourcing our assistant platform is to make it possible for commercial vendors to create state-of-the-art private-by-design voice assistants if they want
Put some of the information of your posts here on your page and things would be much clearer.
Is this monitoring built into your product? If not, wouldn't a user of any AI assistant have this same ability?
If you build something with it, we would love to hear about it and feature it on our website! Come talk to us on our Discord channel :)
Wondering as we have to make a similar decision for our startup.
Yes, we know. You post to literally every voice-assistant related comment thread. If you aren't a bot, you must have a (voice powered?) bot sniffing out threads to post on.
If you're going to do that, please at least learn the difference between a disclaimer and a disclosure. :sigh:
I ask because most of what we hear is that online tools will get better over time because you are sharing your data. How do you work around this, or is that even an obstacle?
as far as Alexa goes, I unplugged it a few months ago and it's collecting dust now, I use my cellphone to check weather and for alarm clock instead, I don't see much use for Alexa other than those two simple tasks.
Ha, I got a google home mini free as part of some promotion, and I haven't even broken the shrink wrap. I need to find a way to sell that thing for a few bucks before it becomes outdated, because I sure have no interest in it.
What's up with the animated drawing on your front page of an alien poorly disguised as a human female at a picnic using voice to control her music player? It's kind of distracting--I keep staring at it instead of reading the site.
I infer that she's an alien for three reasons.
First, her head is on backwards compared to human heads.
Second, she appears to have only one upper arm, attached to her left shoulder, which forks at the elbow into two lower arms, one of which is going behind her back over to her right side. The forearms are also about twice as long as human forearms.
Third, the neck is way longer than human necks, but quite consistent with many the aliens some people have claimed to have seen.
Seriously...is this some recognized art style? I don't know much about art, but I do recall that around the late 19th/early 20th century there were some prominent artists and styles that took big liberties with human anatomy.
It's well drawn sufficiently (as are the other drawings on your site) that it gives the impression that the artist is referencing something known.
There's nothing wrong with posting links to your own work when it's relevant, but please don't use HN only for this. The site is intended for intellectual curiosity, i.e. submitting, reading, and commenting on stories that you run across and personally find interesting. So if you use it exclusively for promotion, you're not really participating in the community as intended. Does that make sense?
I wouldn't be willing to put an open wiretap in my home, even if it did something amazing, like extend my lifespan. The quality of my life is not significantly improved with these devices, and all of their practical uses can be duplicated with the minimal effort of tapping a screen a couple of times.
I have an Echo and use it daily for music, weather, timers, home automation (ie controlling lights and fans), etc...
I also drive a car almost every day and that has a relatively high probability of killing me. Compared to that, having an Echo in my house seems somewhat harmless.
That entirely depends on your definition of "harm". Those of us who see intrinsic value in privacy would be grievously harmed.
Even if we accept that sharing anything private is harm, I'm guessing most purchasers of the Echo understand that the device uses the internet to answer queries. How are those foolish people being harmed?
Why should I be less concerned about a mobile device that waits for me to say "Ok Google" or "Siri" than a stationary device that waits for me to say "Alexa"?
I've never understood this argument. Why is the portable wiretap in your pocket inherently safer?
If I turn off voice assistant on my phone, ethically speaking, it shouldn't be recording my voice. Companies/governments doing so is an actual wire tap, and they should have a warrant.
Putting these devices in your home is giving Amazon/Google/Apple permission to record your every conversation.
How can a device that is only activated by listening not always be listening? If I turn off Siri on my phone the microphone is supposed to go off.
Edit: I know about the wake word. It was a statement on the quote that a device that only works on voice wouldn't be "listening".
That said: right now, common sense would dictate that your phone's battery really can't handle recording full audio at all times, but once that's no longer the case, it does seem problematic that we don't yet have the audio equivalent of putting a piece of tape over your laptop webcam.
https://www.cnet.com/news/fbi-taps-cell-phone-mic-as-eavesdr...
I don't know how easy it is with today's smartphones, but it was definitely a thing in the dumbphone era.
Similar issues with your laptop or these smart speaker devices.
Who says it is? Its astounding to me that people willingly carry around portable (wi-fi enabled!) wiretaps.
- The phone is not designed to listen around the room explicitly, more near field, so it can't hear everything.
- An iPhone seems to have pretty decent privacy settings (only the app can listen while app is open, etc)
- I personally have a phone I can reflash with mostly open source software (only baseband/fivers proprietary), so I am reasonably sure that my phone is not spying on me.
The Echo has pretty decent privacy settings (only listens when you say the "wake word", etc).
But any wider use of audio processing to listen for keywords besides the wake word apply equally well to a cell phone. So again, the cell phone is at least as big a concern as an Echo, if not bigger since many people are rarely outside of arms-reach of their phone.
The problem is not understanding the tradeoffs or even not understanding that there's a choice at all.
That's only true if you're trying to do complex analysis on the device. Cell phones in the early 1990s had even less computing power, but it was enough to encode speech down to 13.2 kbit/s[1] (or 5.6 kbit/s[2]). A simple noise gate[3] would reduce recording duty cycle down to maybe 1% while costing a trivial amount of CPU load.
Modern hardware - even tiny embedded devices - can probably do a lot more than simple gat4ed+compressed audio.
[1] https://en.wikipedia.org/wiki/Full_Rate
1. Or more nefarious things, of course.
If I was designing this kind of spyware, I would put a hard limit that cuts off uploading any "extra" data after sending some low multiple of the "legitimate" data.
This needs to be required for any type of algorithmic decision making. Without algorithmic transparency ulterior motives, intentional or unintentional biases, and unnoticed mistakes are hidden from public review.
A common response is that we don't know how some types of machine learning make their decisions. I agree this is occasionally true. Find a way generate an explanation, or use a different algorithm; transparency is a critical requirement.
What I find interesting is that they keep quoting "consumer watchdog" an anti google organization turned anti technology (they're against self driving cars and robots now).
The scary thing is that mainstream respected news outlets casually traffic in technophobia as evident form articles about automation and AI for example and how every piece mentioning a tech company is permeated with FUD about their motivations or their "power".
But why isn't there a great open-source version of any $iot_appliance? IOT, in general, is crawling with companies that are looking for a monthly cash flow, and are therefore creating an ecosystem of non-federated, walled garden type devices that report back to some central server to keep the user dependent upon and paying monthly cash to some company. All of these IOT projects are generally simpler than voice-assistants, and I look forward to the day when we have IOT projects that talk to each other or a local server, without linking us to some company forever.
Yes, I get that. But so was writing the Linux kernel.
You can take a look at our blog to get started if you want to build your own assistant: https://medium.com/snips-ai/building-a-voice-controlled-home...
Under what license are you open-sourcing the platform?
And what is your business model?
How well do you perform in benchmarks? (Assuming there are benchmarks)
Its also completely open source, and can do text, voice, and picture search.
Note that this removes the warrant requirement in general, even if you personally don't own an Alexa/etc. The test is if the public expects audio in the home might be recorded and sent to a 3rd party. If the answer is "yes", then the police can use their own hardware to do the recording.
[1] http://caselaw.findlaw.com/us-supreme-court/533/27.html
[2] Ibid.
I don't think it ever went anywhere precedent setting however because the defendant eventually gave permission to Amazon to turn it over.
Still pretty muddy waters. I would trust Amazon a little more than I would trust the smart tv makers, but that's just my bias.
Note that re: Kyllo v United States, trust in any specific manufacturer is irrelevant. The SCOTUS ruled that use of a technology - not a product - requires a warrant when it is "is not in general public use".
The Kyllo case involved two federal agents that used their own thermal imaging camera to search a house for the presence of people and grow lights.
In the end, this is their platform of which we are sharecroppers.
Honest question, are you an attorney or can one comment? I am not, but I don't read this opinion the same way.
The case referenced concerns using thermal imaging from outside a home to look inside it, which I guess makes sense then that if thermal imaging was super common maybe they would argue that its not a big deal to point one at a house by a cop, but thermal imaging is not common so it was ruled an unconstitutional search.
But you seem to have taken that as the police being able to enter a residence and install a transmitting microphone just because most people have transmitting microphones inside the house. There's a huge difference in a search that requires entering a home and one that does not.
SCOTUS actually seems pretty explicitly concerned in the text about technology eroding the expectation of privacy by the progressing deeper into a private residence without having to enter it, so that again implies this case was about technology being used to search a home without entering it.
This isn't to say courts will rule one way to the other, or that the courts won't change their mind, but a well reasoned opinion form a different court is powerful.
I think the public expectation part was specifically whether thermal imaging was too invasive and not an observation that the general public would make from the street.
This worries me. First the children are watched 24/7, them the adults.
I wonder how human intrigues (and the literature/narratives around it) will be affected by ubiquitous computing and networking.