Hacker News Reader

Comment by jimmar | Hacker News Reader

It is interesting in the study web page (https://privacystudy.cs.princeton.edu/) that they consistently mention contacting "websites" instead of "people." As if a website is some autonomous thing that can communicate with a researcher.

I wouldn't be sleeping well if I were involved in this study. There is no way an IRB could determine that this is not human subjects research if you're emailing people and asking them anything.

If I want to ask random people about the weather and publish the results in a journal, that qualifies as human subjects research and IRB protocols must be followed. Emailing people asking about privacy policies is definitely human subjects research. Either they misrepresented the study in their IRB application, or the IRB didn't do due diligence reviewing the application.

View on HN

In my understanding (from a french cultural context), asking people questions as part of a field study is not human subjects research. Ethical questions arise when you ask people to take specific actions in order to measure their reactions, not when you're asking about the status quo.

Lying about who you are and pretending that you are allowed a certain thing… I mean legally it’s not fraud but it sure feels like it!

Imagine someone showing up to your office building pretending to have an interview , walking into the office waiting room, then walking out saying “oh, just an experiment!”

“It’s just an email” the ease of the mode of communication here is not super relevant to the action, right?

> Lying about who you are and pretending that you are allowed a certain thing… I mean legally it’s not fraud but it sure feels like it!

I'm not aware of the details, but maybe there's an actual Victor Coutant from Nice on the research team. If not, what does it change? The point is precisely to study how a random person trying to defend their rights to privacy will be received/treated by hosts. You can't study that if you sign your emails with "Princeton privacy researcher".

> the mode of communication here is not super relevant to the action, right?

Exactly. Researchers will declare themselves as such before conducting interviews, but they'll rarely hesitate to take notes or ask a simple question as part of a field study.

I mean if you're not happy this researcher is doing their job to investigate how corporate America mistreats people's data privacy rights, i'd be happy to be the person sending tons of pseudonymous automated emails and handing them over to a researcher. Would that change something for you?!

> Ethical questions arise when you ask people to take specific actions in order to measure their reactions

"Answer my questions within 45 days or I will sue you." That seems to read like a demand for a specific action.

Nonono you don't understand, it was actually "Answer my questions per my extremely benign request including the exact statute requiring your response".

Totally different. Obviously. /s

That's not what the message said. The message asked specific questions about data processing in regards to privacy regulations. Anyone could have sent this message. Hell, i have been on both ends of this message (with CNIL not CCPA) and as an honest person taking part in non-profits i can assure you there's nothing to feel threatened about.

Maybe in your Silicon Valley culture where lawsuits are more easily triggered than private data requests things are different, though. The point is, if the author acknowledged they're running a study in the original email (not the follow-up) they would have skewed their data because corporate assholes don't treat college researchers and common people the same.

> That's not what the message said.

The message said:

> I look forward to your reply without undue delay and at most within 45 days of this email, as required by Section 1798.130 of the California Civil Code.

This is clearly a threat of legal action. Your experiences with CNIL might be misleading. There are few risks of getting bankrupt because of a frivolous lawsuit in Europe, but this is a very real risk in the US. Even if you are in the right and did nothing wrong.

The email ended with a "looking forward to your response within 45 days". Immediately in-house council was under the impression it was someone trying to entrap the website owners into a lawsuit for not being compliant.

There was a Twitter thread of a number of in-house forwarding to external council which costs time and money. This study incurred monetary damages.

Guidelines depend on jurisdiction. Research conducted in the United States may have different requirements than other jurisdictions.

The underlying ethos is that researchers should respect the people who are participating in their studies. People should know that you are conducting a study, the aims of the study, and choose whether or not they want to participate.

> I wouldn't be sleeping well if I were involved in this study. There is no way an IRB could determine that this is not human subjects research if you're emailing people and asking them anything.

Do you have a citation for this? What I'm seeing from random Googling is that you have to be obtaining information about the person for it to count.

If I were researching, say, price trends in some commodity and I called up several companies' sales lines and asked for their current price that looks like it would not be human subject research despite the fact that I'm talking to a human to get each company's price.

If I were researching pay trends at those companies and called up their sales lines and asked the people who answered how much they were paid it would be human subject research.

Here are questions sent to individuals in the study: Would you process a CCPA data access request from me even though I am not a resident of California? Do you process CCPA data access requests via email, a website, or telephone? If via a website, what is the URL I should go to? What personal information do I have to submit for you to verify and process a CCPA data access request? What information do you provide in response to a CCPA data access request?

The word "you" appears in every question.

To quote my IRB training (citiprogram.org):

"Most research in the social and behavioral sciences involves gathering information about individuals. However, some research that involves interactions with people does not meet the regulatory definition of research with human subjects because the focus of the investigation is not the opinions, characteristics, or behavior of the individual. In other words, the information being elicited is not about the individual ("whom"), but rather is about "what." For example, if a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter, that inquiry would not meet the definition of research with human subjects because the information requested is not "about" the director. If the researcher interviewed the director about her training, experience, and how she defines the problem of battering, then the inquiry becomes about her - and therefore "about whom."

The current example is similar, in my opinion, to "how she defines the problem of battering" which the IRB training identifies as human subjects research. The people receiving the researchers' email in the current study are being ask to define the way they interpret and comply with a legal statute.

I can accept that some people don't see the information requested as being "about whom" and therefore is not human subjects research. But the fact that people who have received this email have panicked indicates that the recipients, at least, felt that the questions were more than merely recording impersonal data about their websites.

> I can accept that some people don't see the information requested as being "about whom" and therefore is not human subjects research. But the fact that people who have received this email have panicked indicates that the recipients, at least, felt that the questions were more than merely recording impersonal data about their websites.

I guess the distinction is whether you see a website as an organization, even when that 'organization' is as small as a sole proprietorship or DBA, or even a personal site or blog.

But at the end of the day a "website" -- whether it's a large organization, a small business or a sole proprietor-- is still maintained by people. There might be a process in place if it's a large enough company, it might even get directly sent to the lawyers to deal with.

But at the end of the day someone has to look at the email and respond, and in many cases that can cost money.

To wave off the whole thing as "we're contacting a website not a person" just shirks all the responsibility of ethically experimenting on people, which is what the study actually does.

Of course it costs money! Even my own "huh, this is weird, anyway, we don't collect any data, so let's tell them to eff off" cost me time, and therefore money, so, invoice time it is.

They're sending requests to websites to see how they behave, not to request information.

If you called up companies' sales lines to ask for prices, that's just gathering facts.

If you called up companies' sales lines to see what happens when you ask them for prices for things they don't sell, or to see if they're willing to accept a bribe, or to see if they respond with different prices when you lie to them about who you are, you're researching human behavior.

In this case, they are testing what procedures, if any, companies have in place for handling CCPA and GDPR law, by posing as nonexistent customers and making potentially bogus and misleading requests under the terms of those laws.

This is like performing research on retail refund practices by going into a bunch of shops and seeing how they handle being asked for a refund for an item you didn't buy from there in the first place.

There's a more ethical way to do that study, though, which is to actually buy something from the store first, then go back and try to refund it.

Similarly, there's a more ethical way to discover how websites handle CCPA/GDPR requests, which is to use the website first, and determine in the course of that what possible information about you the website should have; then, within the terms of your rights under CCPA/GDPR, to contact them and make reasonable and legitimate requests to see if/how they are able to handle them.

This is incorrect. It's only human subjects research if the researcher is obtaining data about a human. This is the "about whom" requirement. A classic example is calling a business and asking someone about the products and prices they offer. That's not human subjects research.

Who is the subject of the emails sent to personal domains?

Not sure I follow your question.

An example of something that's not human subjects research would be emailing people who have websites and asking about their privacy policy.

An example of something that is human subjects research would be emailing people who have websites and asking what inspired them to start a website.

I realize that may seem like a subtle difference, but it's an important distinction from an IRB perspective. For reference, and because a lot of people seem confused on this thread, here's what the human subjects research training at my university says about this...

"...some research that involves interactions with people does not meet the regulatory definition of research with human subjects because the focus of the investigation is not the opinions, characteristics, or behavior of the individual. In other words, the information being elicited is not about the individual ('whom'), but rather is about 'what.' For example, if a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter, that inquiry would not meet the definition of research with human subjects because the information requested is not 'about' the director. If the researcher interviewed the director about her training, experience, and how she defines the problem of battering, then the inquiry becomes about her - and therefore 'about whom.'"

I don't buy that interpretation. That is, I'm willing to believe that's how your university interprets the regulations, but I personally think it's perverse and unethical when applied to this situation.

When you deliberately deceive someone in order to obtain information that you think they would be otherwise unwilling to give you, the response you get back is as much "about" their behavior in response to your deception as it is about the subject of your inquiry. (And if the researchers in this case didn't think the deception would make their targets more willing to cooperate, why the threatening language?)

That doesn't necessarily mean this kind of research should never be allowed, but it should definitely go through an IRB's oversight.

I think one problem is that with small websites run by a single person or small group, a person can feel the website is an extension of herself. So a question about the website in some way becomes a question about the person.

More critically, it may actually be an extension of themselves in terms of legal liability.

> An example of something that's not human subjects research would be emailing people who have websites and asking about their privacy policy.

No, that's an example of human subjects research that may be exempt from the regulations due to specific reasons, such as by only interacting with subjects through surveys and interviews (while adhering to further restrictions, that this research probably runs afoul of since it's not anonymous).

> For example, if a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter, that inquiry would not meet the definition of research with human subjects because the information requested is not 'about' the director.

What a terrible example. They've only demonstrated that the director does not qualify as a human subject, while ignoring the question of whether the women staying at the shelter would qualify as human subjects!

You misunderstand the research in question. To quote from the researchers website

> When the system has even higher confidence, it sends up to several emails that simulate real user inquiries about GDPR or CCPA processes. This research method is analogous to the audit and “secret shopper” methods that are common in academic research, enabling realistic evaluation of business practices. Simulating user inquiries also enables the study to better understand how websites respond to users from different locations.

They are not just asking for the existing privacy policy, they are actively attempting to put the subjects into a realistic environment and seeing how they respond. The focus is the behavior of the individual. This should also be evident from the fact that they felt the need to lie to and threaten them...

https://privacystudy.cs.princeton.edu/

He understands perfectly well. What's relevant is whether the response is a property of the individual or the organization, and it's arguable, and controversial, but you'll find a lot of studies performed using this technique that were not considered human subjects research.

As to whether it's deceptive and threatening (the latter of which I find pretty hyperbolic, this is a pretty boilerplate request), that has no relevance as to whether it's human subjects research.

Maybe they should have limited the scope to larger organizations.

Someone looking up the exact statute and quoting it, while not a direct legal threat, certainly carries a lot of implied threats. People don't just look up legal statutes for shits and giggles.

Epistemologically, using a fake name and a threat of legal action to elicit a response from whoever's picking up the phone is no different from dressing up as a cop and harassing someone on the street. The question of whether the content of your accusation stems from their own or their employer's action is peanuts compared to the ethical boundary you crossed when you decided to impersonate authority to witness their reaction.

This feels like it creates a massive ethical loophole.

There are different ways to gather pure factual information, too. In particular where the factual information you are trying to gather is information about the extent to which someone complies with the law, there's some real danger in being able to fall back on a 'we're just gathering facts' defense.

Take this example: "a researcher calls the director of a shelter for battered women and asks her for the average length of stay of the women who use the shelter"

What are the regulatory requirements shelters need to comply with? Do any of them concern length of stay? Are there any liabilities a shelter might expose itself to if it were known that it had women staying there for longer than a certain period? Or individual liabilities if it were discovered that they restrict how long people can stay? Would they potentially expose any of their clients to danger if the length of stay information were revealed to a particular person?

If so, then providing the answer to that question is something the shelter needs to give some thought to. And the manner of their response might be different if that question were posed to them by:

- a woman enquiring about staying at the shelter

- a government inspector

- their landlord

- a random man phoning them

- a journalist

- an academic researcher identifying themselves and the nature of the study they are conducting

So if as an academic you ask a 'just gathering information' question, but conceal your identity, don't share whether the information will be aggregated or identifiable, and don't explain what you're gathering the information for, you are not just collecting a fact - you are forcing the person you are asking to make an evaluation of what information to provide; in other words, you are creating a human behavior, and what you are studying will be the outcome of that.

You realize that the internal regulation is wrong, right?

Like the semantic distinction doesnt matter because nobody gives a fuck about Princeton’s organizational policy.

This is not Princeton's organizational policy or internal regulation, this is the regulatory definition of human subjects research as set by the government. Its semantic interpretation and the "about whom" requirement is exactly how you go about making a determination about whether your research is human subjects research.

In that case I care, and would say that’s an inadequate way to prevent trolling by researchers

Any research which involves human subjects is human subject research.

Nobody can disagree with that.

The US federal government does, as do many western governments. Research that involves humans usually comes under many delineations and sub-delineations with precise names that reflect specific ways in which the research takes place and the corresponding laws and regulations which the researchers must follow.

Determining which category a specific research project comes under usually involves checking specific criteria, in the US they have flowcharts, in Europe they have tables. Either way you can be sure a lot of people are going to be looking at it, most of whom have had to undertake ethics training as part of their career, and some of whom have spent their entire life studying these questions and seen them put to the test over hundreds of trials.

In that light, whether this category of research has got "human" in its name is not going to get you far wrt understanding the problem at hand.

source: I've undertaken interventionist medical research in the U.S and Europe.

If you say "I am a researcher studying X, can you please answer the following questions" then you might be studying a "what", depending on the specific questions.

When you lie about who you are, what your purposes are, and use scary legal language in an attempt to elicit a response, that is absolutely human research. You may be able do those things ethically as scientist but you absolutely need IRB review becausr it is definitely human research.

My guess is that the IRB in this case was not informed of the deceptive nature of some of the emails as lieing is absolutely a red flag that you are doing human research and not just information gathering. Indeed, evaluating such lies for potential harm is an important part of why we have IRBs for psychological and sociological research.

Reply on news.ycombinator.com