Comment by majormajor | Hacker News Reader

Comment by majormajor | Hacker News Reader

Parent Full thread

Credit history, social media footprint, payment history, etc. Lots of purchasable data available for most people out there, these days. Someone could easily write a prompt today like:

"""Here is a person's power usage history:

{{ power usage by month }}

Here is their bill payment history: amount, date bill sent, date bill due, date payment received, if any:

{{ history }}

Here is their credit history for the last five years:

{{ credit history }}

Here is the location of their home and some overall information about the grid:

{{ grid info }}

Please give me a strategy for maximizing profit from this customer, options include "disconnection", "encourage them to use power at different times", "encourage them to buy solar", "move them to variable usage-based billing".

"""

Power companies in most places are probably too regulated to get too sneaky, but I'm sure there's shady stuff that could be done, especially if you can similarly tailor the marketing to each user e.g. "we want this person to get solar since the grid going towards them is near its max capacity and we don't want to invest in upgrading it, tailor the messaging towards them based on their credit history and social media profiles."

If you imagine a less regulated industry there's even more room for price discrimination and such.

The issues, I think, are at least three-fold:

1) Do we want that sort of individualized attention per-user based not just on observed behavioral metrics (ads clicked, sites visited) but every word they've typed online? too?

2) Who is responsible if this model then makes decisions that harm people? The people who trained it? The people who used it? The CEO? It's a wonderful tool for bureaucracy to avoid there having to be a "decision maker" and just have people follow what the tool says to do.

3) And just the practical: the model also is going to spit stuff out, but is it really going to be "optimal" for something closer to a maximization exercise vs just text generation? Possibly not, but I've seen people try stuff like this anyway.

Are your concerns about using AI for this? Or doing this in general?

Planning coercive disconnections in order to maximize profit from a customer based on credit history seems like the problematic thing in this example. Asking an LLM for recommendations on doing seems unrelated.

If they asked a human consultant for it instead, it would be just as bad. And just because the human's recommendations would be more explainable, it doesn't make the human consultant's recommendations any less problematic.

That's what I mean by the first of the three ways I'd be concerned here.

Asking a human consultant to take a human-intelligence-using look at every single customer (or every single visitor to a website, or whatever) isn't realistic at scale. A company will pay a bit for a background check for a prospective employee, they won't hire a PI to read every single piece of social media correspondence that person has ever written.

So if LLMs get cheaper and cheaper, there's a major scale difference due to the existence of the tool.

Technological developments leading to re-thinking laws and regulations is nothing new, of course, and "we should outlaw employers hiring PIs to read every piece of text a human has ever posted online" is something that never would've been a realistic thing to pitch to a politician 25 years ago.

Now, the technology has changed, so we need to talk about it.

You seem to be agreeing that it would be bad, so like the original poster here proposed, let's focus on figuring out what "bad" use cases this tech makes far cheaper and more realistic today and then trying to convince others that we should regulate those use cases.

(The second thing I brought up is a difference not in scale but in deniability and ass-covering: it's illegal already to take a retaliatory action against an employee who complains about harassment, for instance. Figure out a sneaky-but-very-plausible way to make that employee just-happen-to-come-up-bad in some LLM-mediated part of a performance review process? Now you'd have a much harder time getting caught.)

Gotta fight fire with fire. For every public facing process we should perform bias analysis with specialised tools. These tools should elicit edge cases of all sorts, do automated "red teaming".

Putting on my Republican hat: it's too expensive. If we just focus on "was there bias in the output" the amount of time and effort required to analyze all the things is going to be super high. A completely-zero-trust economy would have far too many parasitical drains on productivity as we constantly have to prove everything to everyone every time.[0]

Taking off my Republican hat: which is why up-front regulation on specific actions and methods is what we need instead. Much easier for other people to spot, much more of a bright line "you did this thing, we said don't do this thing." Not at the granularity of "don't use ChatGPT specifically" but more like "these are the things we will allow and won't allow in how you process job application background checks" (we already do this for discrimination, I just think we should update it to reflect that we don't want a centralization/standardization of process so that people become de facto unemployable based on what some tool used by some company thinks about them).

[0] not to mention that looking for bias in outputs in a mechanical way is also gonna false-positive a bunch; p-hacking, but for accidentally getting sent to jail :(

I don't understand your concerns. Many utilities already perform credit checks on new accounts, and customers without good credit may be required to pre-pay or post a deposit rather than post-pay (which is effectively a loan). They also already have programs to encourage customers to shift load to off-peak hours through rate differentials and rebates. And they aren't legally allowed to disconnect customers unless the bill is months overdue; they have zero financial incentive to ever disconnect a customer who pays on time. None of this requires AI, or would even benefit much from it.

Thank you for the reply.

The power companies are going to buy access to this data. This government regulated utility is going to be allowed to charge its subscribers based on social metrics.

This sounds very far fetched to me.

I think you'd be surprised by how many companies buy access to that data.

Would it surprise you if your utility pulled your credit score? Cause that already happens.

Why is it far-fetched that they wouldn't take another step or two along that path?

And let's say it isn't the power company. Let's say it's your employer. Feel good about that?

I'm not surprised about how data is shared. I know we freely share it to companies without concern. Much of this is within our own control.

Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism.

> Regulations would prevent this type of abuse to power consumers in the United States. The rest of this argument is whataboutism.

So... you agree with the original post now that there's a specific risk with the current potential uses of these models?

I'm not sure what you're questioning at this point.

("what if it was your employer" is also not "whatabouttism," it's another facet of the same concern. Another party who pulls publicly available data today, and could potentially pull even more, giving us as the public the need to decide if we want to allow that to happen. Should someone be able to be made unemployable if an LLM decides they're too much of an asshole on Twitter? Let's figure it out.)

>So... you agree with the original post now that there's a specific risk with the current potential uses of these models

No. I do not believe the US government would allow public utilities to set prices based on social metrics bought from 3rd parties.

>"what if it was your employer" is also not "whatabouttism,"

I do not believe a public utility and my employer correlate. Maybe if I was a government employee. Even then, I don't believe the government would have the right to use that data against me either.

I have a hard time believing these underhanded tactics are overlooked and allowed.

I don't understand your fixation on power companies.

Imagine you'd never read the original comment and instead you read a comment from me, saying, "I have a specific concern about how LLMs like ChatGPT will let companies do far more intrusive background checks against every applicant than they do today. I don't want a world where the standard process for getting a new job includes a background check that runs an LLM across everything you've posted on the internet. It wasn't practical for them to do this for every single applicant in the old world, they just did cursory background checks since doing more would cost too much, but as LLMs get cheaper it will be easy and cheap."

Do you agree that that's a concern introduced by the development of LLMs?

AI is not introducing any more concern. Companies already incorporate this practice. Are you against the practice or against AI?

>I have a hard time believing these underhanded tactics are overlooked and allowed.

Really, we're looking at the wrong place talking about the power company... where you should be looking is rental properties.

https://www.propublica.org/article/yieldstar-rent-increase-r...

My friend, do I have a rant lined up for you..:

When I worked at a utility company, we had a purchased dataset on customers with things like inane, ridiculously specific features such as "average number of game consoles per household", on street level.

Now,

a. the data quality was absolute dog shit, and

b. the idiotic amount of features made it super easy to overfit, leak or otherwise do untoward shit when training a model, and

c. often the resulting models (churn damping, targeted marketing) didn't perform significantly better than random sampling...

But the business users/POs ate up that shit like fine cuisine italian sandwiches, because "we have comprehensive 360' multichannel whateverthefuck" insight into our customer base and we make meaningful business decisions based on this.

And this is in GDPR-crazy europe.

Yeah... I've seen similar in the US.

I see LLMs rapidly increasing the data quality of those sorts of datasets by enabling full-text crawling of all sorts of other publicaly-available or purchasable streams of text. You said on HN once that you had 5 consoles? Well, we matched your hn username to your username on this other site, and there was a breach that matched usernames to emails over there, and there was a different breach that let us match emails to full names, and bam, now we have an accurate number.

>now we have an accurate number.

No, now we THINK we have an accurate number because everyone in the entire chain is bullshitting about how good machine learning and their data is.

That's so much worse.

People lie online. I have 5 Ferraris. I am 7 ft tall. I once scored 4 touchdowns in a single game. Now the power company AI can use those "facts" to target their marketing message to me, one of their wealthiest and most attractive customers.

Reply on news.ycombinator.com