thorum

For adults, a better solution is to give users more active controls over the feed and its addictive elements. For example, the ability to remove kinds of content (“not interested”), put hard limits on minutes per day (Screen Time), hide Like counts, reduce notifications etc. Lots of other ideas have been proposed for putting users in control of their feeds and limiting dark patterns that promote addiction. If we’re regulating, I’d like more regulation requiring social media platforms to implement these.

I wonder does it work in real time or does it add latency after each sentence?

The information was tucked inside a blog post about the new safety board apparently:

https://openai.com/index/openai-board-forms-safety-and-secur...

Wow - the most impressive thing about this is the control options. I’m not aware of any other TTS systems with the same balance of control, quality and language support. Looking forward to testing this out…

There’s still a Sky option but the actual voice has been changed.

I don’t understand why self-driving systems don’t have any kind of fallback protections. Like however hard full self driving is to get right, surely a hard rule for “don’t drive at full speed into solid objects, no matter how good an idea it seems at the time to the AI” is not that hard to implement?

You might be interested in how CEV, one framework proposed for superalignment, addresses that concern:

https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...

> our coherent extrapolated volition is "our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted (…) The appeal to an objective through contingent human nature (perhaps expressed, for mathematical purposes, in the form of a utility function or other decision-theoretic formalism), as providing the ultimate criterion of "Friendliness", is an answer to the meta-ethical problem of defining an objective morality; extrapolated volition is intended to be what humanity objectively would want, all things considered, but it can only be defined relative to the psychological and cognitive qualities of present-day, unextrapolated humanity.

CEV is one possible answer to this question that has been proposed. Wikipedia has a good short explanation here:

https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...

And here is a more detailed explanation:

https://intelligence.org/files/CEV.pdf

The superalignment team was not focused on that kind of “safety” AFAIK. According to the blog post announcing the team,

https://openai.com/index/introducing-superalignment/

> Superintelligence will be the most impactful technology humanity has ever invented, and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous, and could lead to the disempowerment of humanity or even human extinction.

> While superintelligence seems far off now, we believe it could arrive this decade.

> Managing these risks will require, among other things, new institutions for governance and solving the problem of superintelligence alignment:

> How do we ensure AI systems much smarter than humans follow human intent?

> Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us, and so our current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs.

Extra respect is due to Jan Leike, then:

https://x.com/janleike/status/1791498174659715494

Maybe? My definition of an art style would be something distinct and recognizably different like say, Van Gogh, Picasso, synthwave, or the aesthetic of one of Wes Anderson’s movies. I’ve seen AI blend, remix and do slight variations on these styles, but I haven’t seen AI make something that I’d consider a new style in that same sense. Can you share an example?

The needle in the haystack test gives a very limited view of the model’s actual long context capabilities. It’s mostly used because early models were terrible at it and it’s easy to test. In fact, most recent models now do pretty good at this one task, but in practice, their ability to do anything complex drops off hugely after 32K tokens.

RULER is a much better test:

https://github.com/hsiehjackson/RULER

> Despite achieving nearly perfect performance on the vanilla needle-in-a-haystack (NIAH) test, all models (except for Gemini-1.5-pro) exhibit large degradation on tasks in RULER as sequence length increases.

> While all models claim context size of 32k tokens or greater (except for Llama3), only half of them can effectively handle sequence length of 32K by exceeding a qualitative threshold, Llama2-7b performance at 4K (85.6%). The performance exceeding the threshold is underlined.

Large context models offer some hope for this. Gemini had a demo where the model was able to accurately translate a (human) language it didn’t know by putting an entire dictionary and grammar guide into the prompt. So maybe in the future you’ll be able to link Copilot to your new language’s documentation and it will just work.

A creative prompt can do new things with existing art styles. It can’t make new art styles.

Many artists now view the tech industry as a credible threat to their work and livelihood, because of AI. If you want them to buy your products, it’s probably a good idea to show some sensitivity to that concern.

It didn’t just not appeal to some audiences. It actively alienated one of the primary audiences of their product. Of course that requires a response.

Discussion from the community: https://meta.stackexchange.com/questions/399619/our-partners...

It’s an impressive model, but why would OpenAI need to do that?

The text says “in a way that would be significantly more difficult to cause without access to a covered model” and in another place mentions “damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human” so that probably doesn’t count. Though it might be open to future misinterpretation.

The bill only applies to new models which meet these criteria:

(1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations.

(2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models.

…and have the following:

“Hazardous capability” means the capability of a covered model to be used to enable any of the following harms in a way that would be significantly more difficult to cause without access to a covered model:

(A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties.

(B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents.

(C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human.

(D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive.

…In which case the organization creating the model must apply for one of these:

“Limited duty exemption” means an exemption, pursuant to subdivision (a) or (c) of Section 22603, with respect to a covered model that is not a derivative model that a developer can reasonably exclude the possibility that a covered model has a hazardous capability or may come close to possessing a hazardous capability when accounting for a reasonable margin for safety and the possibility of posttraining modifications.

One form this takes is that, even when you forget the details, you retain the general shape of the subject matter. You might not remember all the details about XYZ, but now at least you know XYZ exists: your internal map of the world is expanded and corrected, even if parts are a bit fuzzy.

The increase from 2019-2021 (+2 years) was higher than the increase from 2012-2019 (+7 years). I wouldn't want to make any specific claims about the cause, but clearly the pandemic or the response to the pandemic was involved somehow.

I found the graphs in the original linked article pretty helpful in understanding the trend.

https://cdn.jamanetwork.com/ama/content_public/journal/cardi...

I wonder how much a truly high-quality book recommendation service that actually worked and people trusted would change the landscape of the book publishing industry. My own experience as an avid reader is that it's quite hard to find books I actually like and that are worth spending money on. But when you hear these numbers about how many books are published and few books make it big, it makes me wonder how many amazing books there could be that I've never heard of because the publishing industry never marketed them.

GPT-4 and Opus are better for complex/precision tasks, and Haiku is cheaper for everything else.

Good 7B/8B models are still really useful but let’s not be hyperbolic.

I agree wholeheartedly with this:

> In biology class, biology wasn’t presented as a quest for the secrets of life. The textbooks wrung out the questing. We were nowhere acquainted with real biologists, the real questions they had, the real experiments they did to answer them. We were just given their conclusions.

So - not just for biology - what are some good books or other learning resources that encourage questing, curiosity and wonder?

The first one that comes to mind is Feynman’s Lectures.

Can embeddings be used to capture stylistic features of text, rather than semantic? Like writing style?

For most kinds of human systems, evolution selects for resilience, flexibility and ease of communication instead of logic and efficiency. So we end up with things that are kind of weird but somehow remain dominant for thousands of years, while your perfectly designed alternative never gets off the ground. Metric is surely a great leap forward than what we had before.

> And here’s the thing – we all know that GPT-3 was vastly better than GPT-2. And we all know that GPT-4 (released thirteen months ago) was vastly better than GPT-3. But what has happened since?

Other models have gradually caught up to GPT-4 while OpenAI has declined to discuss or release anything regarding their internal research and work on GPT-5. Given how far ahead they were with GPT-4, it seems odd to assume no further progress has been made and just not published yet.

The author’s assumption is based on the claim that GPT-4 Turbo “was a failed attempt at GPT-5” but I don’t see any good reason to believe this unless you take the claim that GPT-5 is impossible as a given and reason backwards.

As alternative perspective in terms of power dynamics: The Kagi CEO is a somewhat powerful figure as the CEO of a well-known tech company. The blog author is a random person from outside the tech startup culture.

The internet levels the playing field so the random person has the power to post criticism of the more powerful person and be heard. It doesn’t make sense to compare with the offline world because this wouldn’t be able to happen outside the internet.

In response the CEO is attempting to force them into a different context where he once again has power. The author recognizes this and therefore refuses.