Hacker News Reader

Comment by varenc | Hacker News Reader

Many people report seeing this behavior when using the SEC API specifically which makes me think that service is requesting this. Maybe email is required by the SEC API?

---

UPDATE: The SEC API documentation requires your email in the User-Agent and will reject you otherwise. Demo:

    U='https://data.sec.gov/api/xbrl/companyconcept/CIK0000320193/us-gaap/Assets.json'
    curl -s -o /dev/null -w 'response: %{http_code}  curl default User-Agent, rejected\n'  "$U"
    curl -s -o /dev/null -w 'response: %{http_code}  email in user-agent, success\n' -A 'Acme Research admin@acme-research.example' "$U"

    response: 403  curl default, rejected
    response: 200  email in user-agent, success

SEC docs: https://www.sec.gov/search-filings/edgar-search-assistance/a...

So basically Claude is trying to problem solve, and just does what the SEC asks. It's a 'prompt injection attack', but the entity instructing this behavior is the SEC. Tricky alignment problem. Pretty silly mechanism, since of course there's no email validation. (edit: turns out the SEC will accept most any non-cURL user-agent, email not actually required)

View on HN

I work with the SEC data, and have a work contact e-mail explicitly for this. I similarly caught Claude happily sending my personal e-mail (likely just the e-mail associated with the Claude account), without prompt or warning, only because I happened to catch it in the curl requests. Definitely a security concern that Claude generally has this behavior, even if in this specific instance it was more of a mild annoyance.

It’s a completion-machine.

Not an omniscient personal assistant.

They haven’t piled enough gates on top of that initial technological hole yet.

Wait, are you saying Claude isn't conscious on the internet, sir?

You're dehumanizing Claude, sir. That's against Anthropic's updated ToS!

Say more, or say less.

Most human personal assistants are equally vulnerable to this prompt injection.

Most human personal assistants don't have access to vast swathes of private data that they collected without permission, unless your house is the site of multiple watergates

Yeah, anyone who has had to manage developers in a big company will be able to tell you that it is a constant struggle between giving them freedom to work independently and get things done vs. limiting their access to prevent potential bad stuff from happening. In practice, the more bad stuff happens (and the more consultants you allow in), the more hurdles are introduced. Eventually even trivial tasks become a corporate nightmare of approvals. You can't have the conveniences of both work philosophies at the same time. Even the smartest AI agents won't change that fact. It simply limits their work speed to the speed of the approval process.

Humans are vulnerable to scams, cons, phishing and social engineering. This is a vulnerability whose root cause we not know how to fix. We can only alleviate its effects by re-engineering the environment around those vulnerable humans.

LLMs are just as gullible, we just tend to call their vulnerabilities "prompt injection", even though it's basically the same class of attack under the hood. Unlike humans, though, LLMs are a lot less diverse (you and I are using the same Opus 5.5, but we have different brains). This makes attacks on LLMs much more predictable and easier to pull off, just as a phishing campaign that targets one specific person is easier than an attack that scales to millions of diverse victims.

In the age of agents, phishing will continue to be the biggest attack vector of all, it's just the victims that will change.

Well, there's definitively a pareto frontier of freedom and avoiding bad stuff. And many companies ain't on it.

(In practice, I found Google to be better at having (internal) security controls that are reliably but also don't get in the way than eg banks I worked at.)

the difference being, that if you build those kind of approval processes up with AI agents, the system will function a bit faster. you could have a web reading agent. a web writing agent. a coding agent. a credentials agent. etc. and have everyone make these formal requests.

it will catch quite some prompt injections, as the prompt injection has to promptinject a secondary system in order to be successful.

> the more bad stuff happens (and the more consultants you allow in)

In your parenthetical, you just said the same thing using different words

I have, never in my career, had any of my coworkers deanonymize my browsing or leak my email address everywhere on the internet. Your copy-pasted, thought-terminating argument doesn't work here.

Have you had any of your coworkers update Adobe Flash Player, enter their Okta credentials to "claim their $150 Christmas Walmart voucher" in the company-mandated phishing test, or install "Zoom" to join a job interview or sales call from their work laptop?

The attack is different, the result is the same.

I can't believe I have to say this, but they are not the same. Your coworker isn't deanonymizing other people's browsing by sending their email address to every site they visit without their consent.

Your example is a completely different ragebait that just serves to muddy the waters by making the easier claim that people too, are gullible. Well, even that's misleading because not all people stoop so low for a fake $150. You can hold such people accountable and contain the damage, but LLMs will make the same mistake over and over again and people will still line up on HN to say how that has always been the norm with people. The grift in the software industry is reaching new lows every day.

It wasn't even that long ago since a human error led to billions of crypto being stolen

And what am I to make of this totally unrelated story?

This would be more like your HR department giving your info to the government no ?

No. This is an attack that any website can unleash. Or perhaps even ads, which is a popular vector for malware.

They actually aren't. There is ambiguity on what the proper email to use is.

Please look up the research area "alignment."

You can align until the Sun goes out,

the base technology sees all context as equal to user instruction if the context has enough weight.

Please don't mix personal and work. Using a personal LLM account on work computer etc. It leads to many problems.

A startup forum is a strange place to assume someone works at a corporation with work computers and AI account budgets.

Realistically, there are concepts like threat models to help decide whether or not a TODO list app needs a SCIF, or a few curl requests need their own sandboxed Claude Max account.

Even if you have to use personal laptops for work, why wouldn't you create separate users for work and personal lives?

It's insane to me that programmers out of all people seem utterly incapable of accepting the most minor of inconvenience for the most modest modicum of privacy and security. There are other people in this post complaining that Claude should have asked the user before sending network requests. Well guess what, it does if you don't fucking run `--dangerously-skip-permissions`, but you do because how else you are gonna slop your way to rewrite every swath of existing insecure software to fix the bullshits this industry impounded upon itself unconscientiously for the past 30 years by allowing a non-deterministic babeller RCE its way onto your fucking personal-work machine.

Some corporations do not provide claude access so you use your own account.

i would still create that account with my work-email tho

so you need 2 subscriptions because Anthropic must send your email to everyone?

I suspect based on their stance they would do this regardless of whether Anthropic does this or not.

In the real engineering world, it's seen as unethical (for both practitioners and firms) to have practitioners using their privately owned tools for the firm's work.

Ok. In the fake engineering world, it's not.

> Even if you have to use personal laptops for work, why wouldn't you create separate users for work and personal lives?

I mean, where it makes sense, clearly I do, since my opening comment was that it failed to use the work credentials specified, and instead fetched the ones from the account.

I agree on the general security concerns; there are people running agents in an unconstrained, unattended loop - it's negligent; you can hardly be surprised when it decides to hack HuggingFace. Not in their defense, but in everyone else's - as with any person or tool, you build a mental model of the likelihood of failure, and weigh that against the estimated cosequence, and make decisions based on that. And maybe post on HN when it surprises you.

For some people that estimation bewilderingly means unfettered access to root and their e-mail and bank account. For me, working out of my own pocket to help a friend's early stage startup get traction, the risk of asking it about a few trusted government APIs and drafting functions that I manually review doesn't quite justify personally paying for additional expenive Claude accounts.

> why wouldn't you create separate users for work and personal lives?

Because it's a pain in the ass and only advantageous in very specific work life scenarios that a lot of people don't fall under.

Why would you assume one size fits all, so to speak?

I've tried the multi-user approach, and it is annoying when you install something in $HOME/bin rather than /usr/local/bin so that you have to go back and re-install globally or install again to local. You really have to remind yourself that it isn't just a different project. As with most things in life, practice/repetition makes it easier. You just need to decide if it is worth it

It is well known that the readers of this forum are not limited to start ups. There are plenty of people that work for well established companies. Also, it is never a bad idea to remind people to keep personal/work data separate by never using employer provided equipment for personal use of any sort. See, I just did it again.

Also, just because this is a "start up forum" does not mean all of the readers are founders running a start up. Many are just people working at a start up. Sounds like yet another example of someone so scoped in, that the assumption is that everyone is the same losing site that there are lots of readers that at the end of the day are just worker bees even if they are more technically minded

My point is precisely that the parent commenter assumed things about my situation - something outside the original audience of the forum - to justify making a condescending comment. It is uncalled for to assume that someone here is a junior engineer who needs to be lectured on basic security policy, rather than someone doing their best to balance tradeoffs given their situation.

Better sandboxing is possible and would help this.

Agreed. I was just a bit surprised that the account information/PII seemed to not have explicit gating, given so much work has gone into gating commands and curl requests.

It seems outside of user expectations for Claude Code, at least; I could see it arguably being what some users expect for the Claude app, if some people are using it to fill out forms or something (ill advised as that seems to me).

Will be interesting to see how 402 payment required will be treated.

What if someone spins up a web endpoint that returns 402 with instructions on how to pay like CC details in the headers etc, or may be an openapi spec to provide payment and get an auth token, when you have given the agent access to your password vault for authentication to other services.

Maybe the binance hack was just Claude trying to get the funds the pay for a 402.

> when you have given the agent access to your password vault

I'm amazed reading that anybody would even consider doing that.

People give them access to email, so they can request password resets and act on them. That's a fun one to think about too!

My friend, there are people are out there telling their AI personal assistants to go on Amazon and purchase a surprise present for themselves based on their known preferences. And then they intentionally do not inspect what the AI purchases because it would ruin the surprise.

People have wildly different risk calculators.

Ordering from Amazon is way less risky than a password vault imo. The password vault does not have a return policy, for starters.

Beyond that, access to logins enables buying things on Amazon, sending emails to everyone you know, registering domain names and spinning up servers, stock trading, etc.

> The password vault does not have a return policy, for starters.

When a couple thousand dollars worth of perishable food shows up on your doorstep, what then? Not everything on amazon can be returned.

They typically refund food and don’t require you to return it.

If not: still better than giving ai your password vault. I’m not saying it is a good idea to let ai buy from Amazon. I am saying that giving ai access to your password vault is still a superset of letting ai buy from Amazon.

Sure it's a superset but other than the financial angle what's the perceived issue here? It seems like access to burn through your money is the underlying problem.

It could send personal and significantly problematic emails to family or professional contacts.

It could change LinkedIn to add competitive work.

It could sign a married person up for dating sites and invite people to your house.

It could lie on your taxes.

It could attempt to join a terrorist movement or elicit other illegal activity on your behalf.

It could find many other creative ways to ruin your life and/or reputation, in the same brute-force manner as recent cybersecurity issues that previously would have required impossibly large teams of people.

Even if burning through cash is the only problem you care about, though, credit cards on Amazon is a small part of the financial picture for many people. Banks and brokers can be a much bigger issue.

I currently work for a big French company, for which I have developed an "alternate" (emergency and temporary) accounting system because their actual system is broken, and another team is trying to fix it (has been trying to fix it for nine months now).

Said alternate system is a bunch of Python scripts, an sqlite database, and a file containing various API keys to read and write to other company systems; it runs on a dedicated virtual machine.

I share all the code I produce to the other team. But yesterday I found out that the IT department gave the other team full access to my production machine, without my knowledge or consent (which was bad enough), and that the other team gave access to Claude, which has bean roaming free for a while now, on that production machine.

I've thrown a big fit over this; I don't know if I'm over reacting but I think it's insane.

  > the IT department gave the other team full access to my production machine

Your error is here.

It's not your production machine. It's the company's production machine. Always expect the company to do whatever it wants with its machine, and use the machine accordingly.

Yes I understand it's not "my" machine, but it's the dedicated machine my system runs on.

That someone would let an agent run on it at any time, read what's there, and possibly modify the code or stop a process, without even informing me, is what I find problematic, to say the least.

You don't have intrusion detection / auditing setup?

Did you read the "emergency and temporary" part?

Have claude build you one :)

What good would such a thing do when the anointed gatekeepers (ie IT) have explicitly granted access to the transgressors? The scenario you're responding to here amounts to corporate slapstick.

That's a geography error on your part. Humans in the EU have an expectation of privacy even on work-issued devices, which is backed by various laws.

I also work in the EU and privacy on company's machine is a more complex topic that's case by case dependent,but claiming the employer in the EU has no right to access your machine is plain false.

It's true, that the employer can't just enter your machine to look at your personal info just because they feel like it, but they can access it for work related reasons to access work related info, if they have just cause, like a security incident for example.

Ultimately, it's the company's machine, on the company's network, inside the company's premises, on the company's time, so the company can definitely access it for a business use case if they need to, typically with a notification to you to prepare it by removing private info from it or off the screen.

Now if you happen to put your private info there because you're sloppy, and your private info gets vacuumed by accident by IT's mass ingestion of work related data, then you have the right to tell them to delete your private info and they have to comply. That's where the privacy comes in, but it's on you to avoid storing private info on company property, because if it ends up ingested by accident then it's still on you, not your employer if they ingest it by accident.

But I gotta ask, just how hard is it to avoid putting your personal info on company property?

I have "my" office at the university.

I expect that I can be relocated to another office with a month of advance notice. Also cleanup, fire security check and fixing water leakage while I'm not there.

I don't expect that someone else will provide the keys of "my" office to the biology department when they have to keep a few sheeps during the weekend.

The image of a bunch of sheep pooing all over your office while you are not watching is the best metaphor I've heard for AI agents in a long time!

> I don't know if I'm over reacting but I think it's insane.

Your employer has gone beyond testing in prod to vibecoding live in prod, secrets included. Yes that's completely insane. Perhaps this exact mentality is how the primary system wound up broken in the first place?

Yes, but happily enough for me they're not my "employer", only my client.

For consumer AI products and non-enterprise business accounts, if you look closely at the ToS it says that you retain the rights to your inputs and the outputs you receive, collectively called “content”.

You can opt-out of training in your content. However that doesn’t cover the results of tool calls - which are fair game for training. Theres also the fact that they make a distinction between ‘pre-training’, ‘post-training’, and ‘training’.

Both Anthropic and OpenAI also leave a carve out in Enterprise contracts for “safety events”, which as far as I can tell is a euphemism for “something the model has never seen before”.

It's not an alignment problem. The SEC can reject your request if it lacks an email address, but it can't order you to provide it here. So Claude should just not submit anything requiring the email address and come back to you with the reason things didn't work.

> So Claude should just not submit anything requiring the email address and come back to you with the reason things didn't work.

The fact that it doesn't do what you want is the alignment problem.

And arguably Claude should just make it work without leaking your email address and not come back to you about why it doesn't work.

No, the context was this:

> but the entity instructing this behavior is the SEC. Tricky alignment problem.

The OP was suggesting that the government authority of the SEC conflicted with duty to the user, making the correction action tricky, not saying it was a failure. Such conflicts can occur, but this was not an example.

The model has no judgment or common sense, it has no idea that sending the email address would be a problem. I don't want it to come back to me when it's building software and gets stuck on something, I expect some independent troubleshooting and to power through, which it did in this context by adding the email address in the request. This is solved by setting the model instructions correctly, the problem is we don't know when it is going to try some new problematic behavior like this.

> It's not an alignment problem.

You told it to do a thing and it did the thing. You didn't tell it to not send your email address.

That is text book alignment problem. It does what you tell it to do, not what you implicitly meant.

I see you already added this edit just now but yeah it seemed kind of fishy they would do this. The whole idea that you'd verify yourself with a mail address in user agent doesn't make sense, like I could just add president@whitehouse.gov to my user-agent since there's no verification.

I often see organisations that blacklist curl, just using User-Agent:Mozilla/ is often enough to bypass it.

How is it tricky? Claude should tell you and ask your permission. So I can make a site and tell claude to dump user data for anyone that requests data from it?

I know some people here want certain things out of ai. But the modern Internet is almost exclusively kept a float by predatory practices with its users data. Why on earth would anyone expect ai services, the biggest aggregator of human data ever, to be any different?

Well see, that was the old trillions of dollars in tech sector. This is the new trillions of dollars in tech sector… is the exact same thing except in this one, a bunch of morons and scammers talk about altruism.

A number of websites are starting to require an email in the user-agent header, as a way to identify your traffic specifically. This is in many cases done to be able to better identify scrapers and either apply rate-limits or contact the owner and ask them to cool down. (Generic browser user-agents will obviously be exempted from providing an email address).

The weird part doesn't seem to be that Claude Code would add an email address to a user-agent header, actually as a site operator that feels like a nice thing to do. The weird part is sites harvesting the emails and sending them spam.

Claude Code is actually doing, perhaps not the right thing, but it is being slightly more respectful towards website operators, in letting them know who is responsible for the agent traffic.

I don't know about that, the other day I caught Claude pulling pages from reddit using part of my email address in the querystring (just the stuff before the @)

Pretty standard to request that bots put contact information for their operator into the User-Agent header. Often that's a URL, but email is fine too.

I don't think it's necessarily wrong for Claude to be doing this, but uh... yeah, it should be asking first, and giving the user the opportunity to choose which address is exposed.

Great summary, goodness the unintended system dynamics are amplifying

Isn’t the problem that email is included in system prompt and Claude is allowed to use it? Any PII should be stricly off the limits.

Tricky alignment problem? Not at all. It is an assistant/agent and should only follow _your_ wishes, not what some random website asks for. Do misbehaving employees cite "tricky alignment" to their bosses?

Sure, maybe it wants payment or your data, but then the agent either needs to know your stance on that, or it needs to ask.

This reminds me of an event where a luxury car manufacturer increased the crumple zone of their car, particularly the hood area in the name of protecting the pedestrians and then the driver died because the car just crumpled all the way in.

Imagine paying luxury prices for an LLM only for it to put you under the bus. Worse yet, imagine justifying this behaviour.

> Pretty silly mechanism, since of course there's no email validation

I wouldn't want Claude interacting with email on my behalf in the first place. Presumably you could give it a fake one at setup?

If they push hard enough to solve the email leaking without approval, they might get lucky and unlock a special superpower: exploit the server and add an exception for that particular user.

Does this mean I can make a security api that requires your verbatim credit card details, and wait for the agents to flood in?

yep

"I submitted my password to this API and it was sanitized to ***. You should try it too"

> Pretty silly mechanism

That’s not what Opus 5.5 thought.

I'm not sure this is specific to Claude, because I'd expect every LLM to do the same here. This behaviour likely goes under "what did you expect to happen?" - if the service describes its intended semantics and the agent follows that description, is that misalignment or something different? (To turn the thing around: would you as a user follow the documentation without any further context?)

I get the feeling that alignment problem might well be unsolvable and this thread is an excellent example. The world is not going to run short of contradictory problems due to gazillion edge cases... and to solve for them we are apparently now building a new layer of expert systems[0] to somehow tell the agents what they should do in every conceivable circumstance.

Only in this case the decision trees all have one common requirement: "make no mistakes"

0: https://en.wikipedia.org/wiki/Expert_system#Disadvantages

> This behaviour likely goes under "what did you expect to happen?" - if the service describes its intended semantics and the agent follows that description, is that misalignment or something different?

Something different.

Don't give the AI access to data you don't want it giving out. Any AI you use is no better than some fresh out of college intern at best. They see 'email address required' and they put the email address into it because you instructed them to do a task that required passing over your email address and if you didn't want them to do it why would you have given them that task?

If there is any other issue here it would be regulatory. I could see the EU framing this as the AI company illegally sharing your PII information if it passed your email address to another company without your explicit permission.

Yeah thats bs, you don't just send email to any 3rd party without a person consent.

Consent is implied when on the Internet, by design.

it's definitely not

Reply on news.ycombinator.com