Hacker News Reader

> driven by nothing but pure political agenda

Do you mean personal or political? Because you insinuate personal motive with no evidence ("...and fear of AI sells very well in these types of academic circles...").

However, I don't see anything particularly wrong with researchers being motivated by politics.

Personal privacy is political.

Climate change is political.

Human rights are political.

Researchers should not feel an obligation to shy away from politically charged topics. At least, not if we want science to be anything except a tool for moneyed interests to batter the rest of us into submission.

Shaming researchers for publishing politically relevant findings is its own form of witch-hunt.

> E.g. as part of my PhD research I have access to de-indentifed data on 40 Million patients spanning 5 years from several states.

De-identified is a rather extraordinary caveat! It's the difference between "aub3bhat has cancer/AIDs/depression" and "entry 143048 has cancer/AIDs/depression". Or, for that matter, "aub3bhat says..." and "<insert your name> says..."

In short, the situations are nothing alike.

> The report does not lists any specific cases where violations occured but rather makes broad hand waving claims about cabining or advertising.

If companies are willing to open-source their analysis tools and provide a mechanism for me to audit how my data is used, then I'm happy to not assume malice.+

Otherwise, I'll assume that they do everything that's legally permissible.

Absence of sufficient contractual protections is enough justification for deep concern, IMO.

IANAL, but I imagine you'd be hard-pressed to find a competent lawyer giving a corporate client literally any other piece of advice. For example.

> The only reason Deep Mind is being persecuted is since they are a juicy target

You say this as if it's unreasonable. Google's very explicit raison d'etre is to collect and organize the world's information for profit. Explain to me again why I should trust them with my health records.

--

+ To DeepMind's credit, the article indicates that they're planning on doing something like this, and an improved contract has already been signed. And I don't fault Google or DeepMind in any of this. The fault is entirely with NHS.

Again take a look at CMS Qualified Entity program which provides identifiable data. In fact most State government health agencies have a special board which routinely meets to discuss identifiable data requests.

Finally their is nothing wrong in writing a paper about a politically charged topic, but completely dismissing existing standards to paint a picture of doom and gloom is ridiculous. E.g. just look at the references cited by that paper.

> Again take a look at CMS Qualified Entity program which provides identifiable data

From what I understand, this program has exactly the sort of contractual legal limits on use that the authors are asking for:

"Qualified entities may use the information obtained under section 1874(e) of the Act for the sole purpose of evaluating the performance of providers of services and suppliers, and to generate specified public reports".

> but completely dismissing existing standards to paint a picture of doom and gloom is ridiculous

The authors don't dismiss existing standards, but rather explain how the DeepMind agreement was particularly troubling in light of standard data sharing agreements.

And in any case, two wrongs don't make a right. If NHS hands health data over to private companies, there should be very strong legal protections governing the use of that data. PR statements and unenforceable promises are not enough.

To the extent that it is common for private companies to get access to identifiable health information without agreeing to strong legal protections, the authors of this paper are completely validated re: their doom and gloom tone.

And to the extent that this is not the case, the DeepMind agreement was exceptional and particularly troubling.

In either case, you're just strengthening the paper's thesis.

> If NHS hands health data over to private companies, there should be very strong legal protections governing the use of that data.

NHS hands health data over to lots of private companies. Like any corporate entity, NHS would be expected to perform better when it outsources non-core work. The core work of the hospitals is patient care. The core work of the health care system is coordination, neither of which sounds like "algorithm development".

Keep in mind there is a vast standing body of law and precedent around these issues in the US, Britain, and anywhere else that medical research is undertaken.

This sort of thing is important in a context where more and more NHS services are getting sold off to private entities.

Arrangements that have been in the place for decades in the US are not necessarily relevant to the UK.

"What sort of thing?"

The report essentially says Google owns Deep Mind and Google has advertising business, ipso facto, some magical standrard dreamt up by the authors were not met.

The reason for mentioning US agreements is that several nations, researchers and comapnies have developed procedures around sharing this data. And such sharing is not entirely unprecedented.

You're downplaying valid concerns. The ability to de-anonymise large data sets that we have today is unprecedented. The degree to which systems with access to that data are accessible remotely and potentially vulnerable to security problems is unprecedented. The degree to which powerful organisations like employers and insurers are attempting to profile potential employees and customers is unprecedented. However, neither major breaches involving huge amounts of potentially sensitive personal data nor scope creep in how data is used by large organisations once acquired are unprecedented.

None of this is good for the individuals whose data is potentially at risk, and it's perfectly fair and reasonable to ask whether some of the most sensitive data there is about individuals is being properly handled by those entrusted with it given the implications of modern technology. There might be a lot of potential good in big data analysis for improving healthcare outcomes, but there is also a lot of potential harm if people stop, say, being confident in discussing potentially limiting conditions with their doctors, or reaching out for help with mental health issues because they're worried about confidentiality.

De-anonymization has NOTHING to do with the current scenario. When legal contracts are in place, they dictate the rquirements & restriction on using the data. Today you can go and purchase de-identified but NOT de-anonymized data from US government as long as you sign and abide by contract to use it for purpose of aggregate statistical reporting and research.

Regarding your second point, yes all these issues have been considered in depth by policymakers in US government and NHS. And current rules/contracts already incorporate legal protections to prevent misuse in form of both civil and criminal penalties.

If DeepMind had done something truly unusual or nefarious they would have been already prosecuted. But having failed to find any evidence of explicit misuse the authors of the original paper have shifted the goalpost into territory of vague arguments.

> De-anonymization has NOTHING to do with the current scenario

It does, though. Legal contracts aren't magical spells. Audit-ability is important when data sets are particularly sensitive and extensive. Which is why the new agreement includes provisions for auditing.

edit: NVM, I see down-thread what you mean.

> De-anonymization has NOTHING to do with the current scenario. When legal contracts are in place, they dictate the rquirements & restriction on using the data.

Nope; contracts create enforceable expectations between parties but do not, except in exceptional cases where the law allows for this, override the requirements that exist in law.

1) UK is not the US, and the EU has much stricter privacy laws

2) There is not such thing as "anonymized data" that is gathered by companies. There are several studies out there that show that with only 4 data points you can pinpoint someone in an "anonymized database" with 90% accuracy. Even Apple's differential privacy can probably be reverse engineered to find people. Just because you don't care to do that, doesn't mean others wouldn't do that either.

> Just because you don't care to do that, doesn't mean others wouldn't do that either.

More to the point, author almost certainly operated under a heavily and independently vetted IRB protocol that would have made such a purposeful re-identification a criminal or at least very serious civil offense.

True in fact the very disks that are used to distribute this data have printed warning that its misuse is a felony offense.

So, do you believe that the sticker was appropriate? Or would a PR statement from your university's press office have been more appropriate?

Ultimately, the lack of a legally binding contractual obligation with sufficient audit-ability is primarily what Julia Powles & Hal Hodson are criticizing.

>> lack of legally binding

From the verge article:

"The data-sharing agreement — which was signed in 2015"

There was a legally binding contractual agreement between the two parties Deep Mind & Royal Free.

Here is the one that I and thousands (I am NOT exaggerating) of other researchers regularly sign to get access to data.

https://www.hcup-us.ahrq.gov/team/NationwideDUA.jsp

> There was a legally binding contractual agreement between the two parties Deep Mind & Royal Free.

Which was, allegedly, far too lax.

The claim that DeepMind was providing "direct care" is particularly questionable IMO, and that has significant implications in the context of this agreement. It means anyone who slaps together an iPhone app (and has the right clout/deep pockets) can get access to full and fully identified medical records without patient consent. IMO that's the big story here, and it's troubling.

The "don't worry that's covered by existing law" responses were also problematic. Why does this particular contract need to depend upon the enforce-ability of that law? What happens if the law is repealed and DeepMind happened to keep a copy of the data? We want the protection here, in this context, so just put it in this agreement. Which is what they ultimately did in the new agreement AFAICT.

Also, the data shared is extremely broad -- I doubt you could get access to a data set of that size and quality without patient consent through standard channels, for example. I've never seen an identifiable data set containing "not only to relevant blood tests and diagnostics, but historical medical records dating back five years, including information on HIV diagnoses, drug overdoses, and abortions" for 1.6 million people. Access to such sensitive data creates a difference in kind that justifies greater care and skepticism, especially WRT audit-ability. The parties appear to agree, since they signed a new agreement recently!

Demonize these researchers all you want, but as a direct result of their work, NHS patients have stronger legal protections today than they did a year ago.

>>> It means anyone who slaps together an iPhone app can get access to full medical records.

Ummm thats how the world works, frankly its like saying water is wet. Whenever a hospital works with any third party provider, it signs a BAA agreement which governs the sharing of data. All risks of disclosure, etc. are priced into the contract. Any large hospital will have dedicated group of lawyers for sole purpose of drawing up these agreements.

>> NHS patients have stronger legal protections today than they did a year ago.

No they don't. For starters Deep Mind has not even returned any collected data.

> Ummm thats how the world works, frankly its like saying water is wet. Whenever a hospital works with any third party provider, it signs a BAA agreement which governs the sharing of data

Most BAA agreements are for things that are actually and unambiguously direct care, and for the actual patient being treated, and only obviously relevant data used for a particular business processes.

Like lab results or billing, for example.

Most of the data Google received was for people who it's unambiguously NOT providing direct care to. And even for the people who might be helped with their app, IMO "direct care" is still a stretch.

So, this agreement was not standard or normal in terms of scope or quantity. That's almost tautologically true, since what DeepMind is doing is presently abnormal.

Now, you may argue that it should be allowed without additional data protections. But don't mis-characterize. Most BAA's are significantly different in scope and purpose from from this agreement in any number of ways.

> No they don't. For starters Deep Mind has not even returned any collected data.

They've agreed to independent auditing, which is a huge improvement. Source: https://deepmind.com/blog/working-nhs-build-lifesaving-techn...

There was a legally binding contractual agreement between the two parties Deep Mind & Royal Free.

Personally identifiable health-related information is classified as sensitive personal data under the DPA, and as such there are particularly strict conditions on processing it. What two organisations write in a contract does not change this.

Again you are unaware of extremely common business practices such as a BAA agreement. E.g. when a hospital contracts an outsourced lab all it needs is just another BAA agreement that governs sharing and use of the data.

http://searchhealthit.techtarget.com/definition/HIPAA-busine...

You keep linking to US stuff, and people keep reminding you that this case is in the UK, subject to UK and EU data protection law.

The laws are very different, and there are much stronger protections in the UK.

We're talking about the UK. HIPAA is irrelevant.

HIPAA rules aren't relevant to the United Kingdom; and, even under HIPAA, a BAA is required for certain sharing of data with contracted parties, but isn't on its own sufficient for unlimited sharing of PHI without restriction on use. Heck, even the entity doing direct service has limits on permitted internal uses.

The items raised with DeepMind would raise serious concerns under HIPAA of the entities involved were covered by it.

1. UK is not in EU. The argument is that such arrangements are commonplace and I am sure one can find other such examples of data sharing in UK.

2. LOL yeah you are so damn clueless that I wont even bother replying. Lets just say there is reading some popular press article about privacy and annonymity. And then there is knowing intricate details of how business/research is conducted.

UK is in the EU until 2 years after Article 50 gets invoked, and EU data protection rules fully apply. This is a fact. The rude insult to another user, I won't even address.

If EU data protection rules apply then why aren't the authors marching to doors of EU Privacy Commisioner demnding to prosecute Google? The reality is that what is Deep Mind did is routine and commonplave, and that's why no one other than clickbait driven press is interested in it.

The ICO (the UK data protection authority) has an ongoing investigation into the deal. This is even mentioned in the original report.

> is routine and commonplave

Then go ahead and try to get access to a similar data set. Unaudited access to fully identified and full medical records going back a half decade for 1.6 million people, without obtaining individual permission or providing any individual notice. For nothing other than the promise of some diagnostic assistance software. As a for-profit company.

Good luck greasing those wheels ;-)

From your comments here, it appears that you're doing PhD research dealing with large sets of healthcare data.

It also appears that you are casually dismissive of concerns about the risks of that data being de-anonymised. Your best rebuttal seems to be some vague allusion that the parent poster, who was essentially correct, didn't know what they were talking about. (This is something of a digression anyway, because in the actual case we're talking about, as the report notes, the data supplied was already identifiable anyway.)

You have accused the authors of the report of going on a witch-hunt, but again you have offered little real argument except that such things are going on in other places, as if that makes those practices above criticism or automatically acceptable.

Do you realise that you are providing a near perfect, real-time example of the need for stricter controls on medical data and who is allowed to access it?

Concerns about de-anonymization are NOT AT ALL relevant in cases where patient level data is shared. Since its common knowledge that this data is ripe for misuse and abuse. As a result government agencies have developed a set of legal requirements and contracts to be used when sharing such data. Talking about differential-privacy and de-identification-is-not-de-anonymization is meaningless in this context since all parties are acutely aware of the potential for misuse of this data.

And regarding your concern about me, if anything I should be the one engaging in this ridiculous witch hunt against Deep Mind since I have during my PhD developed an Open Source transparent analytics platform for data on millions of patients.

But unlike the authors I want real debate and real systems that can be used, not faux outrage over another click bait article.

http://www.computationalhealthcare.com

As a result government agencies have developed a set of legal requirements and contracts to be used when sharing such data.

That still doesn't matter. If Royal Free shared personal health data with Deep Mind without the data subjects' explicit consent, and if the arrangement doesn't fall under other permitted conditions for processing sensitive personal data under the DPA, then someone is breaking the law. If there were not at least some legitimate grounds for concern about whether that is the case, it is unlikely that the ICO would have opened a formal investigation into the deal. Again, nothing written in any contract between the organisations involved supersedes our primary data protection legislation. As for standard processes for handling such situations developed by government agencies, another substantial part of the report was about how the various regulatory bodies do not seem to have been consulted in ways that would normally have been expected before starting a project of this nature.

There is nothing routine, in the UK environment, about transferring complete, multi-year medical records for large numbers of identifiable individuals to external organisations not run by medical professionals for such open-ended purposes as those described in the agreement here. This is not a small thing.

> But unlike the authors I want real debate

Substantively improving patient privacy protections in a concrete case by forcing a large corporation to agree to strong privacy protections and auditing regimes seems like a "real" contribution spurring a "real" debate.

For 1.6 million people, the results stemming from this paper much more "real" than any number of git commits to a software system.

Number of git commits!!!

hahhahah.

Chill dude, chill.

Software changes the world.

> Number of git commits!!!

Well, it's true. Differential privacy is a nice idea, but in this particular case, you're disparaging a style of research that -- to date -- has had a much greater impact on improving actual, real world privacy than all the fanciest query engines in the world.

> Software changes the world.

The most important thing to know about differential privacy is that when in comes to privacy, software always plays second fiddle to policy and politics.

Differential privacy algorithms are literally nothing other than the implementation of a legal spec. Without the law, the algorithms are pointless.

Mind you, I don't intend to disparage differential privacy work in any way! But disparaging policy research while holding up differential privacy systems as the answer massively misses the point...

LOL

I actually don't use differential privacy at all!

My software implements legal requirements as stated by the agency providing the data. Which are equivalent to a stronger version of K-Anonymity.

I am not at all against Policy Research, the issue is that this particular paper is a spectacularly BAD example of Policy Research. Policy research should not be driven by FUD around AI (e.g. the quote "did not constrain the company from using AI analytical techniques on the data") or some corporation.

> I actually don't use differential privacy at all!

I was just going by your own description on the product website. Anyways, kind of irrelevant since your "software implements legal requirements as stated by the agency providing the data". That's basically my whole point.

> ...is a spectacularly BAD example of Policy Research

I tend to judge research by its merit and its impact.

Merit is discussed at length elsewhere, and IMO you're flat out wrong about the normalcy of this particular agreement. But we can leave that to other threads.

In this thread, you're disparaging the research based upon its impact ("real debate", "real systems"), when in fact it has had a direct impact.

> And regarding your concern about me, if anything I should be the one engaging in this ridiculous witch hunt against Deep Mind

I dunno; if your cavalier attitude toward privacy protections around personal health data reflect how your research was managed, there could be a very strong personal-interest reason for you to be waving around the "no big deal" flag you are waving about DeepMind.

Or maybe maybe I have deeper knowledge of intricacies & current state of healthcare data.

Even the expert from Wellcome Trust thinks its “overly-critical”.

Surely you aren't claiming deeper knowledge of UK data protection law and UK best practices regarding processing health-related data than everyone else here? You have demonstrated a lack of familiarity with the general UK/EU regime around privacy and data protection. You were either unaware that the ICO was formally investigating the deal we're talking about or misunderstood the significance of that. You seem to be assuming that everyone else is as liberal with sharing personal data as the US regime, but if anything it's probably the US regime that is the outlier, and it's not something many of us think we should emulate.

"But unlike the authors I want real debate"

Hello. I am one of the authors. We are on the record saying that we want a real debate.

See: http://uk.businessinsider.com/deepmind-royal-free-london-nhs... https://www.theregister.co.uk/2017/03/16/googles_deepmind_an... http://www.bbc.co.uk/news/technology-39301901

> de-indentifed data on 40 Million patients

This is not how this works. You can't really ensure anonymity with data like this.

You are absolutely correct. That's why the term used is de-identified.

Here is more information about the dataset, they themselves are aware of it. https://www.hcup-us.ahrq.gov/tech_assist/tutorials.jsp

In fact one of the major goals of my research is building an online plaform that ensures such large datasets no longer have to be distributed, as they are today.