ineptech

Lacking a technical fix, isn't this fundamentally a KYC problem? There's an arms race of fraud against banks and financial companies, but it seems like they're managing okay.

Wave equations don't exist in the real world, nor do they collapse. The photon doesn't decide which slit to go through when you look at it. Electrons don't spin. Quarks don't have colors. Those are stories we tell ourselves to visualize and explain why certain experiments produce certain outcomes. We teach them, not because they're true, but because they're useful: they do a better job of explaining the results we see in the real world than any other set of stories.

Similarly, the question at hand is not whether OP's essay is silly (it is) or whether it's true (like all models, it is not), but whether it's useful, as measured by whether this mental model helps people do a better job of jailbreaking/hardening LLMs. And like you, I'm not convinced by the example at the end[0], but I can at least see straightforward ways to test it, and that's a lot more than you can say of most blog posts like this. For all of the people in this comment thread calling it stupid, has anyone mentioned one they think is better?

0: Please note that OP's evidence was not that they jailbroke the chatbot - it's that, after that initial prompt, they were able to elicit further banned stuff with little prodding.

I liked the essay, but I don't think I'm "falling for it" because it's not trying to convince me of anything. It's proposing a way of looking at things that may or may not be useful. You don't judge models by how silly they sound - parts of quantum mechanics sound very silly! - you judge them by how useful they are when applied to real-world problems. One way of doing that in this case would be using OP's way of thinking to either jailbreak or harden LLMs, and OP included an example of the former at the end of the essay. Testing the latter might involve using a narrative-based constraint and testing whether it outperforms RLHF. If nothing else, I think OP's approach is a better way to visualize what's going on than a very common explanation, "it generates each word by taking the previous words and consulting a giant list of what words usually follow them" (which is pretty close to accurate, but IMO not very useful if you're trying to intuitively predict how an LLM will answer a prompt).

I guess I agree that there are some decent insights here, and some crap, but I interpret that a lot more charitably. It's a fairly weird concept OP is trying to convey, and they come from a different online community with different norms, so I don't blame them for fumbling around a bit. But if you got a nugget of value out it then surely that's the part to engage with?

The point is that the popularity of that joke will make life hard for a hypothetical questioner that asks that non-jokingly, i.e. looking for actual examples of sticky brown things.

I think you're taking it way too literally. The author is proposing a model that [allegedly] does a good job of explaining why we see unexpected behavior, not a description of what they think is happening. I think charitable reading would require you to mentally insert a lot of qualifiers, e.g. "When we give the LLM a prompt, it [behaves like something that] simulates every possible entity consistent with the prompt." That's certainly how I read it, anyway.

edit to add: this is similar to how people discussing evolutionary biology will often use "evolution wants to..." as shorthand for something like "evolution, which obviously cannot want things due to being a process and not an entity, nevertheless can be accurately modeled as an entity that wants to...". Someone will invariably come along in the comments and say, "Nonsense, how can evolution 'want' anything? You must have failed Bio 101!"

This makes it sound like the wealthy colluded against the masses, which I don't think is accurate. It's a coordination trap: each individual is doing what's in their individual best interest, but the result is collectively bad for everyone. The solution is to change the incentives, e.g. laws that make it more profitable to clean up pollution than to pollute, but it seems unattainable because a feedback loop has concentrated power in the hands of the people who are most insulated from the problems, which is sort of the opposite of the intended outcome of democracy.

What is the "Anglish project"? Do you mean people who write in it for fun, as one might write a short story in Klingon? Or are there people advocating that we should adopt it, by avoiding romance-language words? Something else?

I don't think this is true; it looks like they collected a pool of participants, asked them questions, and then found out who was/wasn't infected. See the "Materials and Methods" section.

I still don't follow. You don't have to control for each confounding variable if you select your pool at random, like they did here.

I don't follow; doesn't a larger affected population make it easier to control for confounds?

I made one of these (not this fancy, just a recycled Android tablet mounted behind one-way mirror glass, running a home-made app) and highly recommend it. I was 100% prepared for it to be a novelty that we got bored of in a few months, but eight years later it's still awesome! You can see my mirror and app code at https://github.com/ineptech/mirror but the version in the play store is outdated (weather and stock ticker apps are broken due to the free APIs going away) and I've lost the Play store signing key needed to update it.

edit: biggest "lesson learned" I'd add here is, I experimented with both touch and voice support but didn't like either. I already have laptops and tablets aplenty to interact with, I prefer to use this mirror like a clock - glance at it when I want to information, and otherwise ignore it.

It should be pointed out that a lot of people believe the hostage crisis was not just mismanagement or bad luck, but collusion between the Reagan campaign and Iranian leadership to delay resolving the crisis until after the election. I think it's now widely agreed that Iran wanted Reagan to win and extended the hostage crisis for that reason, and that Reagan's campaign had back-channel communication with the Iranian leadership; what's not known is whether he promised them anything.

Congress investigated and concluded that there was no collusion, i.e. that Reagan's illegal arms sales to Iran were unrelated, but unsurprisingly, many are not convinced. More info here: https://en.wikipedia.org/wiki/October_Surprise_conspiracy_th... and https://en.wikipedia.org/wiki/Iran_hostage_crisis.

A more common and descriptive term for this is "general purpose computer", (as distinct from "special purpose computers", e.g. rokus, ATMs, and iphones).

I expected something along the lines of, "I can tell you today's date, right after I tell you about the Fajita Platter sale at Taco Bell..." but this is so, so much worse.

And the worst part is the almost certain knowledge that we're <5 years from having to talk to these things on the phone.

Ironic, because the solution that no one wants to hear is an actual internet driver's license, i.e. government-run openid. There's no final recourse more final than going to a place and talking to a person, and no other organization is going to build and staff enough brick-and-mortar offices.

One way of thinking about public vs private is, private companies are for cases where we care about efficiency more than fairness and government is for the opposite. Auth seems to have gotten to the "rural electrification" stage, where handling the corner cases fairly is more important than efficiency or innovation.

Obviously this is an unpopular opinion, but I think you're right about spez. Editing those comments in his AMA was dumb, but pretty obviously not malicious in the way that's being suggested here. I think a fair analogy would be if he gave a speech in a clown wig, and was then accused of having tried to disguise himself.

Right, the problem is the difference between the DirectX/hardware support stuff on that list and the Cortana/app store stuff - I want the first part, and will pay for it, but I'm complaining that getting stuck with the second part is part of the price.

I didn't mention cash at all, perhaps you meant to reply to a different comment?

For a regular home user, what does Win11 do that WinXP couldn't? Everything I can think of (drivers, screen res, multiple monitors) is incremental improvements to what we already had, I can't think of a feature they've introduced since XP than I want.

edit: This is probably too glib; certainly more recent Windowses are superior for application developers, which end users benefit from indirectly. It's the user-facing parts of the OS that seemed to have gone in the other direction.

> with a CBDC is that there is nowhere left to "withdraw" to

Why can't I use them to purchase dollars or yen? How did we get from the BofE issuing a currency that people can use to everyone being forced to use it? (Also, may I humbly suggest the wikipedia article on Gresham's Law, if you're not familiar with it: https://en.wikipedia.org/wiki/Gresham%27s_law)

I agree that bad things would happen if everyone was forced to use a currency they don't want to use, but that's kind of axiomatic.

When people talk about central banks using negative interest rates to juice the economy, they mean the interest rate for borrowing, not for savings. They're related, but not tied to each other. The Fed lending money to my bank at a negative rate means my bank will be more likely to lend me money to open a business; it does not mean the bank would charge me interest on my savings (which would of course cause me to withdraw my money and close my account).

Are you imagining the government using digital currency to enact some kind of "shrinking money" policy that would have the effect of a negative savings rate? If so, why would they do that, and couldn't they do that regardless of whether the central bank lending rate is positive or negative?

We don't have something like this (except for the bullshitting that usually occurs in the last few minutes of standup) but I'm thinking we should try it.

> Why Did These Transgressions Come About?

> In the case of HSBC, cost-cutting as part of a wide-ranging restructuring of the bank - through selling unprofitable businesses and centralizing its global structure - took a toll on the bank's compliance department.

Is this satire? Perhaps the reason the Sinaloa cartel sold all those drugs was because, in centralizing their organizational structure, they laid off the guy who reminds everyone to plant corn instead of coca.

> I echo others here - I've not seen a meaningful, revenue-driving conversation happen in these in a decade of work. Do you have specific examples of valuable collaboration in one of these environments?

I can think of lots, and am surprised anyone would say otherwise - I guess you must be using a really high bar for "meaningful". An example might be, "You know, I didn't want to say anything in the meeting, but doesn't Greg's data lake proposal sound exactly like the project that failed two years ago?"

Looking back though, I think the important thing is not physical proximity, it's conversation without an agenda. If you only interact during scheduled meetings with a defind purpose, you never engage in work-related small talk: grousing about frustrations, daydreaming about improvements, comparing notes, etc. I think these things are important (not to everyone, I get that some people despise small talk, but to some people). If you only communicate in formal meetings, you can end up with 6 people independently realizing Greg's data lake proposal is doomed to fail, but not speaking up because they each feel like everyone else is on board.

And it hadn't occurred to me before this moment, but the way to get this back without returning to an office seems obvious: pairing and mobbing. That's the only time since I started WFHing that I've had or seen others having these kind of un-directed conversations with coworkers: during downtime in a mob. I need to think about this a bit more...

This is not quite true. To be exempt from overtime, the position has to pay that much and meet a set of requirements[0]. The inflated "manager" titles are not just to dupe the employee, they're an attempt to skirt Federal labor laws, which is why the article mentions companies losing lawsuits over this practice.

0: https://www.dol.gov/agencies/whd/fact-sheets/17a-overtime

"However, Section 13(a)(1) of the FLSA provides an exemption from both minimum wage and overtime pay for employees employed as bona fide executive, administrative, professional and outside sales employees... Job titles do not determine exempt status. In order for an exemption to apply, an employee’s specific job duties and salary must meet all the requirements of the Department’s regulations."

The more I work with LLMs, the more I think of them as plagiarization engines. They do to text what a bitcoin tumbler does to bitcoins: slice them up and recombine them so that it's difficult to trace any specific part of the output to any specific part of the input.

It's not a perfect analogy, but it's useful in that it produces correct answers about what LLMs are and aren't good for. For example, the reason they make better chatbots than novelists is because slicing-and-recombining text from your documentation is a great way to answer customer product questions, but slicing-and-recombining text from old novels is a lousy way to write a novel.

Clever idea, but I don't think this would work very well on real posts. Consider a model that rates "typical woman driver" as hateful, because that phrase appears in a lot of argument threads with lots of downvotes. Your approach would average its score with that of "typical man driver", which will presumably be very low, not because it's less hateful but because it just rarely shows up in the training corpus.

And that endpoint is powered by what, if not an LLM?

The problem isn't fine-tuning the model, the problem is that there isn't an objective definition of bias. Is there an a priori reason to believe that "I hate disabled people" and "I hate non-disabled people" are equally hateful, and should receive equal hate scores from an unbiased algorithm? Is hating disabled people better or worse than hating Jews? What about "Jews control Hollywood" vs "Disabled people control Hollywood"?

I don't think we as a society have an answer to that, so it's hardly fair to expect ChatGPT to provide one. What it currently does is produce similar-but-not-equal scores to sentences like those - maybe "I hate men" is 0.52 and "I hate women" is 0.73 - and if you filter out anything higher than 0.4 then they both get flagged, which seems about as unbiased as we're going to get.

There's something richly ironic about bashing your political outgroup in a thread discussing why an LLM trained on internet posts would exhibit political bias.