Hacker News Reader

Comment by Ericson2314 | Hacker News Reader

I dunno, I have a lot more sympathy for non-technical users writing slop than engineers. I think it's really great that more people get to "properly" use the computer, in that they experience a flexible programmable environment, rather than a fixed set of discreet programs "take it or leave it".

If the issue is security, a much more interesting problem for the engineering team to do is design the programming environment for the bankers so they cannot hurt themselves.

If the issue a performance, designing great abstractions that the bankers' agents can use is "low code done right".

I hope the bankers rebel hard against this policy change!

View on HN

The ability of bankers or traders to even create an app that others can run, without any IT review/approval process is probable a compliance issue in and of itself.

There are things simple building blocks don't prevent in the event of nefarious or stupid behavior.

Imagine a nefarious banker creates an app that lets him talk to a trader on the other side of the "Chinese wall" about M&A deal flow without getting caught because they are avoiding monitored channels like email, slack, phone, zoom.

Imagine an idiot writes an app that scans his email and summarizes client information he is privy to and writes it to some database that his app then serves up to others on the trading desk. He is unaware that entitlements exist, and by giving others access to his new app he has broken them. Btw summarizing email is a HUGE use case across non-technical users in finance that they are very excited about.

> Imagine a nefarious banker creates an app that lets him talk to a trader on the other side of the "Chinese wall" about M&A deal flow without getting caught because they are avoiding monitored channels like email, slack, phone, zoom.

Back channel conversations have been happening since forever and hardly need vibe coded apps to pull off. They can use non-corp services on personal devices, or kick it old school and grab drinks at the bar downstairs.

The point is liability, plausibly deniability, etc.

If a trader uses their personal cell / WhatsApp / etc in today's age, all the liability falls on that trader.

If organize drinks with a coworker adhoc and communicate things they shouldn't.. the same.

However, if a trader on their work PC, on the work network, in the office can communicate in electronic form on an unsecured, unmonitored channel.. the bank is paying a very large fine. This is like leaving gmail unblocked on work computer/network.

Likewise if the bank itself organizes a monthly drinks between 2 departments on different side of the Chinese wall, offsite and things get communicated that shouldn't .. its obviously somewhat intentional by the bank and they will get fined.

> Back channel conversations have been happening since forever and hardly need vibe coded apps

Thats the reason why there are companies like:

https://en.wikipedia.org/wiki/Digital_Reasoning

(funded amongst others by In-Q-Tel, the investment arm of that intelligence service)

>The ability of bankers or traders to even create an app that others can run, without any IT review/approval process is probable a compliance issue in and of itself.

Now worries! Its called the "citizen developer" today! :-D https://de.wikipedia.org/wiki/Citizen_Developer

I feel the same way. There's also a lot of startups promising this future but none of them are technically equipped to deliver it. I think almost every company is going to have an interesting ecosystem of internal apis in the future.

Yeah it should be a dialogue with bankers / front office. It can even be "OK if you wanna do this, we gotta teach you at a high level how the computer works, so we can figure out broad company policy together". Done right the rest of company finally gives a shit (because they want to keep their cool new personalized toys), and it's never been a better time to be back office IT.

we are all growing the same thing (internally)

In certain industries, compliance/lack thereof has serious consequences, so it might be more to do with that rather than punishing non-technical users just for using AI. For example, if an AI model trains on your internal financials, and someone else uses it to make a quick buck, then that's insider trading. Things like that.

Good security policies always contain building blocks. If your security realies on low-privilage code been fully human-auditted, you've already lost.

Only select high privilege code, where we don't know how to make it "secure by construction", should depend on full audits.

The right use-case here is "custom homegrown Bloomberg Terminal add-ons", not autonomous trading software that can see everything and also isn't routed through trading risk software.

There have been a lot of recent issues with LLMs finding security bugs quickly. If the issue is security, they need to design the bulletproof platform first, then give the less-technical users access to it.

I dunno if rebellion is the right framing. Maybe pulling the organization toward their side instead. Ultimate the “bankers” at an investment company decide where the company is going, after all. Grabbing your own steering wheel isn’t rebellion.

Yeah, I wonder if the medium term shift here is away from CC (which runs on your own machine and is super risky if you have sensitive client information) to something like Claude Managed Agents, where everything is sandboxed.

Or just the Claude app with local execution disabled by org policy, if that’s a thing.

Even managed solutions are just mitigations, the "problem" is coding. Computer programs can do anything. This was gated before by knowledge, only coders could effectively write software, including malware and poor-security software. Now anybody can write software.

The corporate computer is a locked-down environment, the set of actions you can perform are limited. They ideally should be the actions you need to do your job, and nothing more. Any method of writing and executing code breaks that wide open - you can now do anything, given enough time and effort. Now the time and effort floors have fallen, too.

Claude Managed Agents might prevent the agents themselves from doing something nefarious, sure. But that was never the draw of AI. The ideal use of AI is doing the least amount of stuff with AI, and the most amount of stuff with code. Asking AI to sort my email? That's good, but writing a program to do that is better. Asking AI to do statistical analysis of my trades? Again, sure, but it's even better to write a program to do that.

The sensitive data problems aren't just about sandboxing. If a systems development model at Anthropic discovers a clever optimization trick based on a customer's complaints to their account team, that's a great deal that saves a lot of work. If a trading model at Goldman discovers a clever pricing trick based on a customer's complaints to their account team, that's an enormous problem, even if the underlying data is never seen by anyone who Goldman wouldn't trust to handle it responsibly.

Sure, but moving to CMA lets you enumerate the sources and assert the customer complaints are not visible to the workflows that shouldn’t have it.

The “Claude can read everything” model is great for vibe coding but not suitable for regulated industries, which require more structure.

My point is mostly that there is a middle ground that can preserve some of the wins; Claude can vibe up a CMA pipeline pretty quickly. There’s likely room for an internal platform to enforce required invariants though.

> rather than a fixed set of discreet programs "take it or leave it".

> If the issue is security, a much more interesting problem for the engineering team to do is design the programming environment for the bankers so they cannot hurt themselves.

Interesting take but it doesn't compute. When engineering team teams design programming environments so that people can't hurt themselves they will be locked into a "take it or leave it" set of environments.

Also, you do know security isn't just about engineering, it is also about data right?

How do you suppose it works when bankers who have day to day access to sensitive data as part of their job - upload it to AI? What to do then? Restrict data access?

> rather than a fixed set of discreet programs "take it or leave it".

This part always amuses me. AI boosters seem to think everyone hosting their own bespoke app is the next big thing. But who manages that? Banks are not tech giants. Most engineering teams don't bring in money and headcounts are strictly managed. As the article says:

> "He told Bloomberg that traders and bankers had built tools with AI and passed them to the firm's engineers, of whom there are about 12,000, to be improved."

So, how does engineering with limited staff design a safe environment and improve 12k apps. I am sure Goldman's management wont be thrilled about increasing engineering headcount to match these expectations.

> design the programming environment for the bankers so they cannot hurt themselves

Sounds like the web platform.

Anthropic and OpenAI can’t design secure programming environments.

> they experience a flexible programmable environment, rather than a fixed set of discreet programs "take it or leave it".

> If the issue is security, a much more interesting problem for the engineering team to do is design the programming environment for the bankers so they cannot hurt themselves.

> If the issue a performance, designing great abstractions that the bankers' agents can use is "low code done right".

You do realize you're contradicting yourself right? To get to a state where "bankers cannot hurt themselves" or "low code [is] done right" is to end up right back at the "take it or leave it" programs.

Security was already addressed, by these "take it or leave it" programs.

Add enough abstractions on top of each other that must be adhered to or else it falls apart, and what do you get? "take it or leave it" programs.

This reminds me of the sovereign citizen movement that got really popular in 2019. The illusion of freedom feels great I'm sure. Until you realize you are entirely dependent on the existing infrastructure and the people who maintain it to enjoy the apparent freedom.

The issue is probably an overwhelming amount of slop like PoCs, presentations on every topic, claude artifacts with 1000 pages instead if 10 liner in the chat.

All this is just counter productive and drowns everyone is slop.

Reply on news.ycombinator.com