Hacker News Reader

Comment by aRationalMoose | Hacker News Reader

I have an open ended question-- mostly born out of ignorance; But why is this a bad thing? Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful? In the process, sure they'll collect your info for better advertising, but short of Totalitarian Surveillance or Data Breach Concerns (The former is a bit of a reach if you live in the west, and they can survey you anyway if they really want to, the latter also seems somewhat unlikely)-- whats the issue here? Genuinely asking because I'm trying to understand.

View on HN

> But why is this a bad thing?

Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large.

Totalitarian Surveillance is here. In the west. Secure document releases aside, it's too easy to do to imagine a state actor not doing it.

Data breaches of differing severities occur every day, at nearly every company. I would have thought Yahoo was big enough and smart enough to avoid it; but no. Not Yahoo, not Sony, not security contractors, not credit bureaus, not Apple (a'la celebrity photo leaks), not Google (stories abound of individual GMail accounts being hacked).

I agree with you. To help convince people, I realize that we often imagine benevolent leadership, so it helps to give an example such as, "Imagine if you were a Muslim or illegal and Donald Trump were elected president. What could he do with your data?" E.g. find you, search your residence based on your purchasing and travel habits and send you home.

E.g. Wakes up at 5:30 am, travels to a construction site, lives in a house with a large number of people -> signals possible immigrant. Or this:

Detecting Islamic Calendar Effects on U.S. Meat Consumption: Is the Muslim Population Larger than Widely Assumed?

https://mpra.ub.uni-muenchen.de/41554/

We have to think about data not just in terms of our relative safety, but in terms of what could happen in adverse circumstances. And not even just in terms of our own government, but foreign governments.

>Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large.

(Have worked at google in the past, may in the future, am not currently). You say this as though anyone at Google (or Microsoft or whatever) can go in and search for 'falcolas' and look through your GPS history.

I'm honestly not sure if there is a single individual at the company who had that power. I honestly think that the best thing Google could to is publicize their internal training and documents on personal information, because the regulations and such made me a lot more comfortable with giving Google the sort of amorphous entity my data, because no person is going to be looking at that data.

>, not Google (stories abound of individual GMail accounts being hacked).

One of these is not like the others, unless you're talking about something I'm not aware of. Hacking an individual GMail account requires guessing/taking someone's password, which is not an attack on Google's infrastructure (Unlike the yahoo, sony, apple, etc. examples), its an attack on a bad password.

> [...] which is not an attack on Google's infrastructure

This strikes me as a matter of semantics; does it really matter if I'm targeted whether they hacked my account or hacked Google?

> I'm honestly not sure if there is a single individual at the company who had that power.

Think harder. Who has the root access to the servers holding the data? Could the existing infrastructure and data segregation ever change? How many external checks and balances are in play that can't be manipulated by internal forces (i.e. is there anything stopping Google, or holding Google accountable if their data protection policies change)?

I'm sorry but if you think that far ahead, then how do you do anything?

Do you go out in public? because if you do, some company could be recording you on CCTV, and the company that makes the CCTV equipment could sell the business to Google who could update it to use the CCTV footage in AI learning, which means that someone could eventually lookup your face and see you were at a smut store 6 years ago.

At some point you need to draw the line, there is no perfect privacy.

You are, of course, correct. Especially in this day and age, perfect privacy is nearly impossible.

That said, you can limit your exposure. Adding all of these Google implements creates a far greater surface to lose privacy through than not using all of these Google implements.

People routinely underestimate how much can be gleaned about your from correlating such "incidental" data. Thus I feel it's important to remind them of what it can cost them.

Is the benefit worth the cost? To some, yes. To me, no. And that's why I posted this, an explanation of why I don't find this level of information gathering and correlation by a private and profit driven company acceptable.

  > Who has the root access to the servers holding the data?

I'd be surprised if such a thing existed in any large ‘cloud’ system. A data center machine is a small and fungible unit of computation and/or storage, and there's no reason for anyone to be able to log in to one.

>This strikes me as a matter of semantics; does it really matter if I'm targeted whether they hacked my account or hacked Google?

I think is incredibly important. If your information is put at risk due to bad practices by Google/Yahoo/Apple/Facebook/whomever that's a problem to be taken up with the company. If you use insecure passwords and someone is able to access your information that way, then the problem is with your passwords, not with the platform.

>Think harder. Who has the root access to the servers holding the data?

As far as I'm aware, no one. Like I said, from my experience, accessing personal data and user information as an engineer required a lot of red tape and approval from 'the powers that be', and violating those rules would get you fired faster than anything else.

>Could the existing infrastructure and data segregation ever change? How many external checks and balances are in play that can't be manipulated by internal forces (i.e. is there anything stopping Google, or holding Google accountable if their data protection policies change)?

Here I agree with you, probably not (or very little). They obviously have public privacy policies, but you have no proof that they abide by those, and I don't know (and doubt that) they get audited or whatnot to make sure that those policies are followed. Which is why being an employee made me more comfortable. If nothing else, it meant I'd know ;)

How about the government? Isn't this exactly the access that Snowden (a contractor) had? And there are/were countless tales of people using the system to track ex-girlfriends/celebrities. Now imagine that not only do they have phone/email access, but every action the person takes in their home and potentially every single thing they say in their home (the microphone is always on).

In what way is this not exactly the nightmare scenario in 1984? You can argue you don't need to install this, but 10 years ago you didn't "need" a cellphone either. The risk is the consolidation of information and the potential for misuse/control. And not so much potential, but the inevitability.

Maybe today, but what about tomorrow?

Even if Google is perfectly secure from bad-actors today, they might not be tomorrow. And if they themselves suddenly switch to being a bad-actor, they aren't going to throw all that data away and start from scratch first.

"<...> comfortable with giving Google the sort of amorphous entity my data, because no person is going to be looking at that data."

How do you explain this then?

http://www.theweek.co.uk/google/11581/google-worker-fired-fo...

Sure, there are some trust issues, but just regarding your two first points:

A very limited number of Google employees have access to private user data (only when it's vital to their work) and they have strict policies in place (data does not leave the data centers etc.). Which third parties are you referring to? As far as I know, Google does not give their users' private data to a third party.

> they have strict policies in place

What guarantee do you have that these policies will never change in the future? Or are you simply assuming that risks never change?

I think the strongest guarantee is that the sustainability of their business very much depends on that. Billions of incentives to make not a single ex-employee able to say "I managed to hack my way to user personal data".

That's not much of a guarantee. First, you're relying on everyone acting rationally. I hope they would, but humans often act irrationally, especially if grudges or money is involved.

More important is your assumption that the decision would even be made by Google. Outside forces such as governments may force Google's hand.

> able to say

It doesn't matter what is said. If Google had sufficient deniability (perhaps an NSL gag order? or a sufficiently high purchase price?), they can say user personal data is secure while sending it outside their control.

The only guarantee that would be believable is if they indemnified their users against any future damages derived from their data collection, and there is no way Google (or any company) would willingly accept that kind of liability.

Sorry, the policies the parent commenter was talking about were about employees' (lack of) access to user data.

Being ordered by a judge to do something obviously supersedes any policy, but that's the case for any person and business.

> was talking about were about employees' (lack of) access to user data

Which we have to take their word on and hope that never changes in the future, even though Google might not be the party with the authority to make that decision. Even when they are, business plans change and a pile of potentially profitable user data is a very powerful temptation towards moral hazard. Only a fool would claim that this wasn't a risk.

> that's the case for any person and business.

Only if you deliberately ignore the entire point that the data shouldn't be stored at all by 3rd parties. A business that sold a real product (instead of a service masquerading as a product) would run locally and no data would be put at risk.

If a judge orders me personally to reveal something, they probably need a warrant and there is a process by which I can challenge that order. If, however, that data is stored on Google's servers then I don't have standing to challenge any interaction between Google and the government.

Lots of reference to a user's private data - but what is private? Is my zipcode, gender, and birthdate private? Those three factors can be used to uniquely identify greater than 80% of the US population. Are the GPS locations I visit private? If so, why does information about them show up on lock screens?

Third parties get my voice recordings for "improving the voice recognition service" - what if my name is mentioned in the background of one of those recordings? What if I'm not a savvy user and add private data to those recordings?

You're also talking about what's in place today. If I give Google my data, that data is probably going to stay with Google as long as they are a business (and potentially after, if Google were ever liquidated and their assets sold off). What measures are in place to protect me then?

Yes, if data can be used to potentially locate somebody, like a combination of zipcode, birthdate and first name, it is considered PII (Personally identifiable information) and those strict policies would apply.

I'm responding to a comment that said trusting Google == trusting ALL Google employees, which is not true. Trusting Google with your data is believing that having some convenience (a mail service like Gmail, an intelligent assistant, etc.) is worth the risks you are talking about: Google drastically changing their policy, or being bankrupt and acquired by less scrupulous owners, etc.

Let's not just act like anybody at Google can look at your data and play with it, or a disgruntled employee will suddenly click a button and release all users' data on pastebin...

> Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you?

You forgot: every single state which Google is subject to.

Yup. If any of my data is stored on servers in, say Canada, what is to stop the Canadian government from siezing Google's servers in an effort to stop my maple syrup smuggling ring?

Certainly one of the main weaknesses of the current system, but the more power Google accumulates, the faster it can displace those states.

There are three levels of discomfort some people feel with this situation:

1. Concern that a single, third-party entity (Google, in this case) might peer into every aspect of our lives, and/or reverse-engineer an exhaustive catalog of our entire lives, by virtue of data collection.

2. Concern that many consumers will unwittingly opt into such control, unaware of the privacy they're relinquishing, and unable to make informed decisions about the possible applications and consequences of the tradeoff.

3. Concern that the custodian of all this personal data (Google) might use, sell, transmit, or turn over the data in ways we had not anticipated or believed we'd consented to.

Personally speaking, I understand these concerns but also understand the potential upside. I'm not 100% sure where I stand just yet. The aforementioned bullet points are presented without editorial comment; just trying my best to articulate what I believe to be the crux of people's concerns here.

The way you phrased the question sounds to me like "what are the shortcomings, leaving aside these terrible shortcomings?"

Having said that: Google is not in the business of making your life easier, but in the business of selling you ads. The data that Google collects about you is incredibly powerful, allowing them to go from a "simple" manipulation to sell you stuff you wouldn't otherwise buy, to full-scale blackmailing you if they see it fit (not saying that this is happening, but if they wanted to, who would stop them?).

It's putting too much power in the hands of a single, amoral entity (like all corporations). That's not good.

> but if they wanted to, who would stop them?

The law, and the economic interest of all the rich shareholders that care about the company's reputation.

Not an ignorant question, but a reasonable one. I think the skeptics believe that society overall does not yet understand the tradeoff. And I agree with them there. I think on a deeper level, people are creeped out that data paints such a deterministic picture of our lives. Sure, I leave home the same time every day, and I leave work about the same time every day. That Google knows where I live and work without me ever explicitly saying shouldn't be a surprise, but not everyone enjoys seeing how their daily lives are so easily circumscribed, even with just passively-collected data.

I wouldn't mind an artificial assistant.

I don't want a company that employs tens of thousands of people, along with a government in a foreign country, along with all the governments on the data route in between, and their employees, civil servants and assorted snoopers of all shades, to have access to the artificial assistant's communications and thoughts relating to me.

All these organisations are made out of people. People with power are inherently untrustworthy; they need enforcement mechanisms to be kept in line, and enforcement mechanisms need to be activated every now and then to stay in working order. That is, occasional abuses are required to keep abuse in line. The thin blue line wavers like a pendulum: it's how we know it's working.

Part of what I fear is that Totalitarian Surveillance is only a "bit of a reach in the west" because we put such a high value on privacy (and personal liberty) that we're willing to defend it. When that goes away then the "reach" will be far easier.

Edit: Interesting comment in another thread: https://news.ycombinator.com/item?id=12639530

> incredibly useful

Sure it would be useful. Sell the assistant as a locally-installed app that guarantees personal data never leaves the LAN and will sell.

> sure they'll collect your info

Only if you let them. Demand better behavior from their software and business practices.

> Totalitarian Surveillance or Data Breach Concerns

What you seem to be missing is that the concern isn't about today's level of surveillance or today's data breach risk. Data generally persists indefinitely once it makes its way into a database or logfile.

To make a claim that these are low risk requires that at no time in the future will surveillance risk increase or data breaches become more common, ... or that the company will run into financial trouble and need to sell your data, ... or that a breach will be forced by a government (not necessarily your's or Google's), ... or that your data will be aggregated into other databases, increasing the "predictive" power and attack surface, ... or any of the other unknown ways your data could be used in the future.

Humans are already known to be terrible at assessing risk, especially when there is a very large separation between the cause and effect. Smoking today giving you cancer many years later is a traditional example. We already know data breaches happen, well meaning employees make mistakes or succumb to corruption, and external powers such as governments or organized crime occasionally take away your agency. Do you really want to claim that none of these risks will ever happen? Because that's the actual wager you're making when when you use Google's products.

For most of history, most humans lived under tyranny or domination if not outright slavery. It's only been the past few hundred years that this mostly stopped in some places.

Maybe we've turned a corner and will never go back to that. But I don't have confidence yet.

>Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful?

No, not really. Restaurant recommendations and traffic reports are simply not that hard for me to find on Yelp or Waze myself. The "anticipation" here doesn't really help me in any material way.

Reply on news.ycombinator.com