lsy

Something like this seems expected, right? If you tune a statistical model to very high accuracy in "addition" over tokens, then the resulting structure of the model must correspond to some structure in the training data. And fourier would make sense for some token like "123" which internally is represented as the integer 7633, but needs to "contain" information about the text digits for math to work. Notably, this still ends up being in some way a statistical endeavor rather than truly learning addition, as even the fine-tuned model doesn't reach 100% accuracy.

I thought this would be an article about how every web search now involves a sort of crawl through the pile of LLM-generated garbage to find the one site with some kind of human input and actual expertise. It's a little funny that the new "alternative" is going directly to the source to get custom-generated garbage that seems more accurate because it is perhaps for now less commercial.

The goalpost here is pretty specific: a couple hundred people try for 4,000 hours to figure out a "universal jailbreak" which means it converts the model to one that answers all 10 of a set of "forbidden" questions. Since they couldn't, the technique is considered robust.

Looking at the data though, there apparently exist jailbreak techniques that make the model answer five of the questions at full detail, and nine at "half detail". Given that the model would ostensibly be deployed to millions of people who would collectively use it for millions of hours, I'm not sure how confident I am that the 10-question barrier would remain unbroken for long.

If the LLM’s user indicates that the input can and should be translated as a logic problem, and then the user runs that definition in an external Prolog solver, what’s the LLM really doing here? Probabilistically mapping a logic problem to Prolog? That’s not quite the LLM solving the problem.

To me this is an illustration of one reason we are still some way off from a truly general system. Applying these "filters" to sensor data is enormously helpful, but only in certain applications. For a system giving GPS driving directions, you may be able to linearize and smooth movement, or constrain it to the nearest road. But you have to know that those assumptions don't apply to other domains that could use GPS data (e.g. surveying, wilderness navigation, animal tracking), and you also have to know when the situation calls for just going out with a tape measure, or installing a camera and referencing landmarks.

Probabilistic reasoning is useful when the problem space has been constrained to admit probabilistic solutions. But the real world is not just analog and continuous, it is also discrete, and it may require fine discrimination between two states that are covered by the same distribution. Effective operation as a general intelligence in the world requires the ability to tell when the stakes or details of an event require a different approach. Determining when and how to make those decisions is still a largely unsolved problem.

Some of these clues wouldn't be very good for a human playing. "007" for example isn't a very good clue for "laser", not only because something happening to be in one of several films about a character doesn't rise to the typical level of salience, but also because other words on-board like "shark" and "astronaut" even moreso meet the criterion of featuring prominently in James Bond movies, and "astronaut" appears to be a game-ending choice.

There’s a conflation going on here. Pareto can be good engineering—a product that solves 80% of use cases at 20% of the cost is a great efficiency: tax prep software for simple tax returns only; a minimalist photo sharing site with few social features; a phone with a great UI and no user-installable apps.

This gets conflated with products that are 80% reliable across all their tasks (LLMs, brittle software). That makes it difficult for users to rely on the product, because occasionally a failure will happen, and the user can’t build a mental model of what works and doesn’t.

The title makes it sound like some new architecture, but this is a blog post where someone likes the results they get sometimes when they fiddle with their input to the LLM to suggest “contemplation”, which apparently makes the LLM generate a large paragraph of highly neurotic text before the answer. There aren’t benchmarks or investigation of the model to see whether it is robust or generalizable so it’s hard to say whether this is useful or not.

Leaving aside the lack of consensus around whether LLMs actually succeed in commonsense reasoning, this seems a little bit like saying “Actually, the first 90% of our project took an enormous amount of time, so it must be ‘Pareto-hard’. And thus the last 10% is well within reach!” That is, that Pareto and Moravec were in fact just wrong, and thing A and thing B are equivalently hard.

Keeping the paradox would more logically bring you to the conclusion that LLMs’ massive computational needs and limited capacities imply a commensurately greater, mind-bogglingly large computational requirement for physical aptitude.

I think “alignment faking” is way too generous for what is happening in these “tests”. The program spits out text based on text. If you provide it text encouraging it to spit out a certain kind of “deceptive” text, then append that text and ask it for more text, you’ll find that you get a “deceptive” result in keeping with what you appended. But this is all the experimenter’s doing, not the model. It’s like a ventriloquist publishing a paper about how his dummy lies.

What is not in evidence:

- any kind of introspection on the model’s part

- a plausible mechanism by which the model can distinguish between training and real world use without the experimenter making it explicit

- any aspect of this dynamic that is due to the model rather than the framework of scratch pads and prompts built around it

It seems like what's being tested here is maybe just the programmed detail level of the various models' outputs.

Claude has a comically detailed output in the 10th "generation" (page 11), where Gemini's corresponding output is more abstract and vague with no numbers. When you combine this with a genetic algorithm that only takes the best "strategies" and semi-randomly tweaks them, it seems unsurprising to get the results shown where a more detailed output converges to a more successful function than an ambiguous one, which meanders. What I don't really know is whether this shows any kind of internal characteristic of the model that indicates a more cooperative "attitude" in outputs, or even that one model is somehow "better" than the others.

I guess it's not news but it is pretty wild to see the level of millenarianism espoused by all of these guys.

The board of OpenAI is supposedly going to "determine the fate of the world", robotics to be "completely solved" by 2020, the goal of OpenAI is to "avoid an AGI dictatorship".

Is nobody in these very rich guys' spheres pushing back on their thought process? So far we are multiple years in with much investment and little return, and no obvious large-scale product-market fit, much less a superintelligence.

As a bonus, they lay out the OpenAI business model:

> Our fundraising conversations show that:

> * Ilya and I are able to convince reputable people that AGI can really happen in the next ≤10 years

> * There’s appetite for donations from those people

> * There’s very large appetite for investments from those people

I find it hard to believe that anything like this will be feasible or effective beyond a certain level of complexity. It seems like a willful denial of the complexity and ambiguity of natural language, and I am not looking forward to some poor developer trying to reason their way out of a two-hundred-step paradox that was accidentally created.

And for a use-case simple enough for this system to work (e.g. regurgitate a policy), it seems like the LLM is unnecessary. After all, if your system can perfectly interpret the question and answer and see if this rule set applies, then you can likely just use the rule set to generate the answer rather than wasting resources with a giant language model.

Given that anyone who’s interacted with the LLM field for fifteen minutes should know that “jailbreaks” or “prompt injections” or just “random results” are unavoidable, whichever reckless person decided to hook up LLMs to e.g. flamethrowers or cars should be held accountable for any injuries or damage, just as they would for hooking them up to an RNG. Riding the hype wave of LLMs doesn’t excuse being an idiot when deciding how to control heavy machinery.

The C-suite operates in the universe of big ideas and are being presented demos and mockups that imply more capability than is actually possible. And the employees have to actually deal with the rubber hitting the road. They can see that the tech is half-baked and are dealing with the real problems users experience, and there is an intuitive sense that customers aren't going to pay subscription prices for something that fails in an embarrassing and opaque way for every N times it's invoked.

It's also difficult for engineers to understand how a sustainable industry can be built on tech where the primary method of "engineering" it is to try doing something (training a new model, changing a prompt) and hope it will work rather than knowing what will and won't. The potential to get trapped in a local maximum or burn massive resources for insufficient return is too high, and the cost of switching models is too low to build any moat based on model quality.

At the end of the day this is the only thing tech investors are excited about, so the market will dictate strategy until the bottom falls out for whatever reason. The video is not assuring on this front, saying that CEOs are just staying the course without much indication they have a reason to believe a return is around the corner.

> In reality, it's a pretty objective fact that being disabled means being unable to do something.

This isn't an argument for or against the comment or the OP, but this is not universally seen as objective, and there are more ways to think about this than the medical model where there are "normal" people and that disabilities are deviations from what is "normal". Many disabled people and experts on disability see it as socially constructed, because certain conditions (e.g. being shorter than average), while limiting your physical abilities, are not disabilities because society generally provides for those conditions and extends equal access. If 50% of people were born Deaf, our society would still function, but it would address Deafness with affordances like the ones currently offered to short people.

To put it another way, despite e.g. being gay or short posing various social or physical disadvantages, we shy away from encouraging conversion therapy or height extension surgery by default, instead opting for a more inclusive society. And human beings are complex — a person who has found culture and meaning as a member of the Deaf community may disagree with you that they are "objectively worse off" for not having hearing, despite the obstacles.

This article I think does a good job of explaining the quandary of a parent faced with a "fix" for a condition that is challenging but through which people have developed identity and culture. I am lucky to not currently have to take a stand on the right choice here, but I think the complexity of the issue deserves more respect than to be dismissed out of hand.

It's wild to compare these maps with what is being published today in the New York Times about the war in Ukraine: https://www.nytimes.com/interactive/2022/world/europe/ukrain...

The modern maps, while technically more "accurate" than hand-drawn diagrams, are almost shockingly light on information in comparison, and the accompanying text is a linear recounting of various advances and retreats with only passing explanation of the strategic importance of either.

I think there are two effects at play here: one is the decrease in expectation of the readership to have much comprehension or critical thinking facility, which is counterintuitive given supposed strides in education over the last eighty years. The second is the gutting of the news media as an industry where career professionals could get and keep lifetime experience in understanding important events and their place in history. While no doubt reporters today are doing their best, the news media is increasingly reliant on contract work and fresh grist for the mill, and it seems obvious that the writers at the NYT have just not been given the resources or motivation to become as familiar with the contours of the conflict as the writers of the past.

By using humor, I think Apple is attempting to avoid previous failures in the advertising space where consumers recoiled from the unsavoriness of using an AI to replace a genuinely important interaction. In that sense it's kind of a clever way of alluding to consumers' potential use cases without taking on the commitment of standing behind those use cases, and leaving consumers ethically on the hook for their decisions to use these tools interpersonally.

On the other side though, it definitely portrays the AI user as a sort of slacker, which is an interesting way to advertise a tentpole feature.

This argument justifies any unethical behavior. "Some bogeyman on the other side of the world is (letting computers decide to kill people | manufacturing and using bioweapons | torturing their captives for information), are you really going to let your 'ethics' get in the way of our safety?"

Maybe you don't think there is any line we shouldn't cross, but I'm guessing you do. The fact that other people draw that ethical line in a different place shouldn't be all that surprising. The people you're addressing likely believe that the US can achieve its defense goals without turning over matters of life and death to LLMs or image recognition systems.

There can’t be any information about “truthfulness” encoded in an LLM, because there isn’t a notion of “truthfulness” for a program which has only ever been fed tokens and can only ever regurgitate their statistical correlations. If the program was trained on a thousand data points saying that the capital of Connecticut is Moscow, the model would encode this “truthfulness” information about that fact, despite it being false.

To me the research around solving “hallucination” is a dead end. The models will always hallucinate, and merely reducing the probability that they do so only makes the mistakes more dangerous. The question then becomes “for what purposes (if any) are the models profitable, even if they occasionally hallucinate?” Whoever solves that problem walks away with the market.

I was surprised to see an “Alexa-compatible” microwave listed as released in 2010 when Amazon didn’t buy Alexa until 2013. The linked reviews of the oven were both in 2018 so perhaps it came out around then.

This was probably always the likely outcome of an internet economy that revolves around the production and monetization of "content".

We started by putting advertisements on existing content, then moved to social networking and social media, which was essentially an engine for crowdsourcing the production of greater amounts of content against which to show advertisements. Because money is up for grabs, producing content is now a significant business, and as such, technology is meeting the demand with a way to produce content that is cheaper than the money it can make.

The problem of moderating undesirable human-generated content was already starting to intrude into this business model, but now generative tools are also producing undesirable content faster than moderation can keep up. And at some point of saturation, people will become disinterested, and tools which could previously use algorithmic heuristics to determine which content is good vs bad will begin to become useless. The only way out I can see is something along the lines of human curation of human-generated content. But I'm not sure there is a business model there at the scale the industry demands.

I guess the question is: Who cares? What is this for, except illustrating blogspam?

It seems that more resources are being poured into verisimilitude across generative models, but what is the business model or even human use case for it?

A picture of a glorious landscape seems worthless to me without any grounding to be able to ask a question like, "where is that?", "when do those flowers bloom?", "what is on the other side of that mountain?" and receive any kind of interesting answer.

What's the advantage of specifying it as random over specifying it as sequential in some way? Aren't both just specifying a behavior, where one behavior is potentially more useful than the other? I guess I understand the principled point that a set of hash keys is not an array. But it seems like more complexity than is necessary, and even possible to fall victim to Hyrum's law besides… you can imagine someone using random hash ordering to implicitly randomize something without needing to call an RNG explicitly.

It seems like the most principled approach might be an re-specification of the data structure: why is "iteration" even possible over a hash table? Shouldn't the keys be specified as a "set" on which only set-appropriate operations like map can be performed?

This has always been the end-game for the pseudoscience of "prompt engineering", which is basically that some other technique (in this case, organizational policy enforcement) must be used to ensure that only approved questions are being asked in the approved way. And that only approved answers are returned, which of course is diametrically opposed to the perceived use case of generative LLMs as a general-purpose question answering tool.

Important to remember too, that this only catches those who are transparent about their motivations, and that there is no doubt that motivated actors will come up with some innocuous third-order implication that induces the machine to relay the forbidden information.

Note that for the purposes of this paper a “problem” just means a formally decidable problem or a formal language, and the proof is that by creatively arranging transformers you can make individual transformer runs behave like individual Boolean circuits. However, this is a long way from any practical application of transformers: for one thing, most problems we care about are not stated as formal languages, and we already have an exceptionally more efficient way to implement Boolean circuits.

LLM and other generative output can only be useful for a purpose or not useful. Creating a generative model that only produces absolute truths (as if this was possible, or there even were such a thing) would make them useless for creative pursuits, jokes, and many of the other purposes to which people want to put them. You can’t generate a cowboy frog emoji with a perfectly reality-faithful model.

To me this means two things:

1. Generative models can only be helpful for tasks where the user can already decide whether the output is useful. Retrieving a fact the user doesn’t already know is not one of those use cases. Making memes or emojis or stories that the user finds enjoyable might be. Writing pro forma texts that the user can proofread also might be.

2. There’s probably no successful business model for LLMs or generative models that is not already possible with the current generation of models. If you haven’t figured out a business model for an LLM that is “60% accurate” on some benchmark, there won’t be anything acceptable for an LLM that is “90% accurate”, so boiling yet another ocean to get there is not the golden path to profit. Rather, it will be up to companies and startups to create features that leverage the existing models and profit that way rather than investing in compute, etc.

I suspect it is actually a great strength of Europe that the attitude towards regulation (to put it perhaps overly simplistically, favoring more intangible rights over economic growth) precludes hype and bubble phenomena like the mentioned "creation of large, integrated data sets" or "young, innovative tech companies". This means that business, at least to some extent, is forced to address real-world issues and work at a more local level. What is seen as "innovative" in the US is typically merely rent-seeking or parasitic on the broader society, and it seems likely that slower, more measured growth is consistent with better outcomes overall.

It is also curious to me that the US and China are seen as examples of what to aim for, rather than as cautionary tales of what can happen with an excessive focus on growth. It is not clear to me that the general happiness of people in the US or China exceeds that of people in Europe. On the contrary, it seems that those two countries are almost constantly dealing with some horrible externality of unbridled focus on stock prices, revenues, growth etc, to the detriment of the broader social good. Housing crises, homelessness, rampant privacy violations, the expansion of police states to maintain the boundary between haves and have-nots, debt crises, long working hours and bad conditions — these are all phenomena that seem more prevalent in countries with a higher focus on "competitiveness". What good does it do a middle-class worker to make twice the absolute salary when they are spending half of it correcting for the deficiencies of their society?

There is a valid point that in a society where police have broad immunity and little oversight, the requirement to do paperwork sometimes serves a purpose in introducing enough friction to moderate the capriciousness of initiating an interaction. If you have to do paperwork you are more likely to reserve paperwork-generating actions for situations that truly warrant it. Making that a push-button process means that the “fun part” of apprehending someone and exerting power over them has less downside and may happen even when it’s unnecessary.

I am more than a little discomfited by what the author seems to want here:

- “Reduce competition” by making online dating exclusive to those who can afford $100+/mo

- Obtain potential match’s sexual history prior to conversation to use as a proxy for promiscuity

- See all social media of potential matches

- Ratings hit for people who decide not to meet with you after chatting

- Interface for quantifying the multiple human beings you are talking to as “leads”, CRM style

- Automated reverse image search / face recognition for social media

- Random bonus: ability to filter Instagram messages by male/female??

Leaving aside the basic disrespect for the people on the other end of the chat here, who actually thinks women would participate on a platform where they are being cyberstalked by, and pressured into meeting with, desperate men who are tracking them in a spreadsheet?