NotebookLM's automatically generated podcasts are surprisingly effective
simonwillison.net
simonwillison.net
Say you need to read those instructions, but it’s also really nice out and you want to go for a jog: two birds, one stone.
I’d feed it the Singularity paper, but I’m not sure I need that extra boost of anxiety these days…
In other words: soapbox is presumably some sort of toy car that goes 15mph, and formula 1 goes up above 150mph at least (as you can tell, I’m not a car guy). If you have any actual scientific argument as to why a model that can score 90-100 on a typical IQ test has only 1/10th the symbolic reasoning skills of a human, I’d love to eat my words! Maybe on some special highly iterative, deliberation-based task?
This is in-line with all art, music, and video created by LMM at the moment. They are imitating a structure and affect, the quality of the content is largely irrelevant.
I think the interesting thing is that most people don't really care, and AI is not to blame for that.
Most books published today have the affect of a book, but the author doesn't really have anything to say. Publishing a book is not about communicating ideas, but a means to something else. It's not meant to stand on its own.
The reason so much writing, podcasting, and music is vulnerable to AI disruption is that quality has already become secondary.
Commercial creative workers are vulnerable because there's a billions-of-dollars industry effort to copy their professional output and compete with them selling cheap knock-offs.
I see this sort of convenient resignation all the time in the tech crowd... "creative workers only can blame themselves for tech companies taking their income because their art just isn't any good anymore!"
The poor quality "content" that's been proliferating recently has been created, largely, using the very tools that AI has built, or their immediate precursors. AI, for all its benefits, has only made that worse.
If you're saying, in good faith, that most of the infomercials, televangelist programs, talk radio, celebrity autobiographies, self-help books, scandalous expose books, and health/exercise fad books etc etc etc that came out 50 years ago were made for no reason beyond advancing human knowledge, you're either too young to remember any media from before our current era and haven't looked beyond survivorship bias.
Tech folks love sentiments like this because it entirely emotionally places the onus on the people getting ripped off by big tech companies for being ripped off. If their work was that awful, companies wouldn't be clamoring to vacuum it up into their models to make more of it. Nearly all of the salable output from these models exists solely because it took a creative product someone made with the intention of selling it and it's using it to sell a simulacra.
It's using nostalgia to deflect guilt for harpooning the livelihood of many people because it's just more convenient and profitable to empower mediocre "content creators" they use to justify doing it.
This, times a million. Add to that the ancient quote from Plato(?) criticizing writing or the other ancient quote complaining about the irresponsibility of the youth, unthinkingly deployed to attempt to delegitimize any kind of critique of nearly anything.
The technology industry seems to be overflowing with so-called "rational" people who mainly seem to use use whatever intelligence they have to rationalize away responsibility for whatever problems their beloved technology has caused. It's a really stupid and obnoxious pattern; and once you see it, it's hard to not see if everywhere and be annoyed.
I think one element of it is naked greed (especially from the entrepreneurs) but I think another big part is a kind of stuntedness and parochialism that's often fueled by overconfidence (because of success in software engineering, forming an identify around "being smart" etc).
I agree there will be winners and losers of some proportion here. But I also think the people that want to pay for art will continue to pay as their motives and values are different. There's plenty of cheap knock-off art, but people still pay premiums for art to support the artist and their work.
As someone else replied to you, it's similar to piracy. The people that pirate were never going to pay in the first place. To tie it back here, the people listening to AI generated <whatever> were never going to pay in the first place - which is why so many podcasts get their money from ads.
Is that not the work of commercial creative workers? Did it not exist pre AI? There's an argument to scale, certainly, but the idea that "things were better in the past before these <<new technologies>> came out" is generally a suspect argument.
To your broader point - new tools for creating creative work come out all the time. Did we suffer greatly at the loss of image compositors when Photoshop arrived? On the flip side, did digital art gut painting and sculpture? Isn't this just another tool for creative expression?
Art is a way of seeing, not a way of creating. I don't think the technology is taking that away.
For example, Perun. This guy delivers an hourlong presentation on (mostly) the Ukraine-Russia war and its pure quality. Insights, humour, excellent delivery, from what seems to be a military-focused economist/analyst/consultant. We're a while away from some bot taking this kind of thing over.
https://www.youtube.com/@PerunAU
Or hardcore history. The robots will get there, but it's going to take a while.
It's an article of faith -- we don't KNOW that they're going to get there. They're going to get better, almost certainly, but how much? How much gas is left in the tank for this technique?
Honestly, I think the fact that every new "groundbreaking" news release about LLMs has come alongside a swath of discussion about how it doesn't actually live up to the hype, that it achieves a solid "mid" and stops there, I think this means it's more likely that the robots AREN'T going to get there some day. (Well, not unless there's another breakthrough AI technique.)
Either way, I still think it's interesting that there's this article of faith a lot of us have "we're not there now, but we'll get there soon" that we don't really address, and it really colors the discussion a certain way.
We all seek different kinds of quality; I don't find Peruns videos to have any quality except volume. He reads bullet points he has prepared, and makes predictable dad jokes in monotone, re-uses and reruns the same points, icons, slides, etc. Just personally, I find it really samey and some of the reporting has been delayed so much it's entirely detached from the ground by the time he releases. It's a format that allows converting dense information and theory to hour long videos, without examples or intrigue.
Personally, I prefer watching analysis/sitrep updates with the geolocations/clips from the front/strategic analysis which uses more of a presentation (e.g. using icons well and sparingly). Going through several clips from the front and reasoning about offensives, reasons, and locations is seems equally difficult to replicate as Peruns videos, which rely on information density.
I do however love Hardcore history - he adds emotion and intrigue!
I agree with your overall hope for quality and different approaches still remaining stand out from AI generated alternatives.
This will be the dynamic of generated art as it improves; the ease of use will benefit creators at the fringe.
I bet we see a successful Harry Potter fanfic fully generated before we see a AAA Avengers movie or similar. (Also, extrapolating, RIP copyright.)
If you just need voice discussing some topic because that has utility and you can't afford a pair of podcasters (damn, check your couch cushions) then having a mid podcast is better than having no podcast. But if you need expert Insight because expert Insight is your product and you happen to deliver it through a podcast then you need an expert.
If I were a small software shop and I wanted something like a weekly update describing this week's updates for my customers and I have a dozen developers and none of us are particularly vocally charismatic putting a weekly update generated from commits, completed tickets, and developer notes might be useful. The audience would be very targeted and the podcast wouldn't be my main product, but there's no way I'd be able to afford expert level podcasters for such a position.
I would argue Perun is a world class defense Logistics expert or at least expert enough, passionate enough, and charismatic enough to present as such. Just like the guys who do Knowledge Fight, are world class experts on debunking Alex Jones, and Jack Rhysider is an expert and Fanboy of computer security so Darknet Diaries excels, and so on...
These aren't for making products, they can't compete with the experts in the attention economy. But they can fill gaps and if you need audio delivery of something about your product this might be really good.
Edit - but as you said the robots will catch up, I just don't know if they'll catch up with this batch of algorithms or if it'll be the next round.
An American Quakening
And then faster/easier/cheaper access to the LM 'uninspired but possibly useful' content, whatever that might look like.
I suspect LLMs are not sophisticated enough as a paradigm to get there.
Think back to the mid-1980s and the first time everyone got their hands on a Casio or Yamaha keyboard with auto-accompaniment.
It was a huge amount of fun to play with, just pressing a few buttons, playing a few notes and feeling like you were producing a "real" pop song. Meanwhile, any actual musicians were to be found crying in the corner of the room, not because a new tool had come along which threatened their position, but because non-musicians apparently didn't understand (at least immediately) the difference between these superficial, low-effort machine-generated sounds and actual music.
People keep forming these analogies/explanations with the inherent premise that what we have now is what AI is going to be - "It's actually kind of shitty so don't fret, not much will change".
AI music creation has improved more in the last 5 years than keyboard accompaniment improved in the previous 40 years. It would be very brazen to bet that the tech 5 years from now is hardly any better. Especially when scaling transformers has consistently improved outputs. Double especially when the entire tech industry is throwing the house at scaling it.
It isn't so black and white.
It ends up sounding like a smarmy Sunday-morning talk show conversation, with over-exaggerated affect and no content.
So far I've just fed it technical papers, which may be part of the problem, but what I got back was, "Gosh, imagine if a recommender system really understood us? Wow, that would be fantastic, wouldn't it?"
https://www.youtube.com/watch?v=ssDdqq_9TzI&t=34s [April Ludgate meets Tynnifer, Parks and Rec]
A lawyer friend of mine also suggested giving it the Spanish civil code, a long, arid legal text. The podcast of course didn't cover the whole text in 10 minutes, which would be impossible, but they selected some interesting tidbits and actually had me hooked until the end and made me learn a few things about it, which is no small merit. And my friend was quite impressed and didn't complain about correctness.
Lately, when I just want to get the gist of a long article or research paper, I run it through NotebookLM and listen to the podcast while I’m exercising.
My only complaint is that the chatty podcasty gab gets tiring after a while. I wish it were possible to dial that down.
In terms of such music and films (whether created by human or AI) sometimes it's just because we are social creatures and need shared experiences to talk with others about.
Reading a book is a time investment so I want it to convey the thoughts of another human being, otherwise it would feel like wasting my time. Listening to music, on the other hand, often is something that I do while I exercise, to keep a brisk pace and not get bored. As long as it sounds good, fits the genres and styles I like and is upbeat enough for exercising, I wouldn't have much of a problem with AI music - maybe it would even be a plus, since there are some specific music genres where I have already listened to pretty much everything there is (and no more is being made), and it would be great to have more.
I don't listen to podcasts, but I suppose in that case it depends on how you do so: devoting your full time and attention like a book, or as a background while you do something else like exercise music? As far as I know, many listeners are in the latter case, so I don't see why they wouldn't listen to AI podcasts.
A randomly selected NotebookLM podcast is probably not substantial enough on its own. But with human curation, a carefully prompted and cherry-picked NotebookLM podcast could be pretty good.
Or without curation, I would use this on a long drive where audio was the only option to get a quick survey of a bunch of material.
So where does AI regurgitated slop fit into my life?
I’ve heard at least one ad from dozens (probably a hundred) podcast episodes that I didn’t finish.
I think that has always been the case, we just tend to compare today’s average stuff with the best stuff from earlier days.
For example, most furniture pictures from the 60s and 70s are from upper middle class homes. If we listen music, we listen to Queen and not some local band from Alabama (not that I’m against such bands at all; they can make great music too).
I agree with this of course, because generally nobody remembers the bad stuff unless it was the worst. I beg to differ with music, though, because there's an opposing effect: we tend to be left with the most marketed music, which was usually a cheap knockoff of something interesting going on at the time. The shitty commercial knockoff becomes the "classic" while the people they were ripping off don't even get a wikipedia page.
In my company, HR now uses AI to do training videos. It's hilariously funny, because it looks like a satire on training videos (well, granted, it's funny for a minute or two, then it shifts to annoying).
If anything, listening to that reminded me of why I stopped listening to podcasts in the first place - every 5 second snippet of something interesting ends up suffocated by 5 minutes of filler and dead air.
They're vulnerable because people aren't random. Most of what we do can be modeled statistically and translated into patterns and tendencies. Given a sufficient number of parameters, just about anything we do can be digested by an autocompletion program that can then generate an output similar enough to the real thing to fool us.
This has been the case as far back as I began reading books which is about 30 years.
Podcasts are only somewhat about things. The most important part is that they're by people, and the people is what draws people in. These ai podcasts are not by people, and when you listen to more than one you start to see the patterns and void where a personality is.
I don't have time to read white papers (nor am I very good at it), but want to know what they consist of. I also want to take my dog for a walk which is hard to do while staring at a screen. This, and other tools like it are useful in achieving that.
Reading, or listening to podcast, these days is more akin to a meditation - many people do it to reenforce an identity rather than to expand on themselves.
And I do think that is reasonable as, for many people, there are few other structures that can keep them in check with themselves.
I was thinking this kind of thing is the perfect way to generate sports commentary.
no one has gotten feedback from "most people" .. this is raw hyperbole
... but it's also pointless. And it's likely different episodes on different topics will tend to sound very much alike; it's already the case here, I'm sure I heard another example where the two voices were the same.
In less than a year we all have learned to recognize AI images with pretty good accuracy; text is more difficult, but podcasting seems easy in comparison.
That's definitely going to be an improvement. Not.
I think this is just partly an inevitable consequence of going from "content scarcity" to our new normal of "content obesity" over the past 20 years or so. In this new era of an overwhelming amount of content, it's just natural to compare it all against each other, e.g. to essentially "optimize" it to the "best" form, but in doing that we've fallen into a homogeneity, and the resulting lack of variation is an actual lowering of quality in and of itself.
2 examples to explain what I mean:
1. I find that nearly all interior design (at least within broad styles) looks basically the same to me now. It's all got that "minimalist, muted tones but with a touch of organic coziness and one or two pops of color" look to it. Honestly, I don't know how interior designers even exist today, when it's trivial to go to Houzz or any of a million websites and say "yes, like this". A while back I was complaining online somewhere that I thought all interior design looked similar where in the past there was much more interesting variation, and somebody insightful replied that it's not really that interior design is now just the same, it's that it's really just converged. People can easily see and compare a million designs against each other, so there is much less of a chance for that green shag carpet to even get a moment in the sun.
2. I was recently on vacation and decided I wanted to read a "classic" book, so I read Hemingway's The Sun Also Rises (I'm not sure why I never had to read that in high school). Nearly throughout the entire book I couldn't help but thinking "Is there any time this book stops sucking?" I hated the entire thing - it was like being forced to watch someone's vacation photos for twelve hours straight, and I kept wondering why there never seemed to be any attempt to actually make me give a shit about any of the characters in the book, as nearly every one of them I found insufferable and wondered how they each had about 3 or 4 livers to spare. But I do understand that Hemingway's writing style was unique and original at the time, and that he was doing something new and interesting that influenced American literature for a long time. But these days, given the flood of content, it feels like most attempts at doing something "new and interesting" are not only forced, but nearly impossible given that there are a million other people also trying to do new and interesting things that now have the means to disseminate them. I don't think a book like The Sun Also Rises, where I believe the main impact was the style of writing/dialogue vs the actual story, could ever break through today.
I guess my point with this long post is that I think the "loss of quality" in content that many of us sense is just a direct result of there being so much content that we see variations from the "ideal" as worse, where in the past we may have found them interesting.
But the content here has been fed into it deliberately.
I don't expect him to ask very technical questions. It's not that kind of a podcast and he will lose a lot of listeners if it becomes too technical.
Bring on the one that's all British and snarky!
I guess it’s training data but also heavily RLHF. I doubt that the trainers are aware of their own cultural biases and values, and they may not care. And why should they? In either case, from a thousand yard perspective, it’s probably an effective vector for spreading “American values”, if you will.
But I don’t think it’s much of a threat to actual podcasts, which tend to be successful because of the personalities of the hosts and guests, and not because of the information they contain.
Which leads me to hope that the next versions of Notebook will allow more customization of the speakers’ voices, tone, education level, etc.
I wonder if any “blended” podcasts will pop up, where a human host uses a tool like this for an artificial cohost.
> We always start with a clear overview of the topic, you know, setting the stage. You’re never left wondering, “What am I even listening to?” And then from there, it’s all about maintaining a neutral stance, especially when it comes to, let’s say, potentially controversial topics.
Oh yeah, this is exactly why I listen to Oxide's podcast! (This is a joke. They often launch into topics with no explanation or context, and are unabashedly opinionated.)
What really stands out, I think, is how it could allow researchers who have troubles communicating publicly to find new ways to express themselves. I listened to the podcast about a topic I've been researching (and publishing/speaking about) for more than 10 years, and it still gave me some new talking points or illustrative examples that'd be really helpful in conversations with people unfamiliar with the research.
And while that could probably also be done in a purely text-based manner with all of the SOTA LLMs, it's much more engaging to listen to it embedded within a conversation.
I would not be surprised if the second pass to generate the podcast style loses some of this fidelity.
Yes, it will generate a middle-of-the-road waffling podcast, but not one with any real depth.
Honestly, given the personalization maybe it's a net improvement.
Would they also observe a rocket launch from the grounds of the space center and go "eh, not really impressive" ?
Or maybe they are just defining "impressive" as something totally different to what we're thinking.
> like thrown into every sentence
I think that's actually part of why it sounds real, because tons of people do actually talk like that.
To me what would make it even better is the ability to throw in random jokes and utilize information about their surroundings and recent events.
I have been using MeloTTS for text-to-speech and I thought that was about the best we could do right now, but apparently I was very wrong. Is there an offline model one can download today that sounds as good as this NotebookLM?
And SoundStorm has more than twice the context window of Bark so dialogs are a tight fit.
Basically it’s a neat party trick at the moment. I do hope to see it improve however!
> It is interesting that nowadays, practically no one feels that sense of awe any longer - even when computers perform operations that are incredibly more sophisticated than those which sent thrills down spines in the early days. The once-exciting phrase "Giant Electronic Brain" remains only as a sort of "camp" cliché, a ridiculous vestige of the era of Flash Gordon and Buck Rogers. It is a bit sad that we become blasé so quickly.
> There is a related "Theorem" about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of "real thinking". The ineluctable core of intelligence is always in that next thing which hasn't yet been programmed. This "Theorem" was first proposed to me by Larry Tesler, so I call it Tesler's Theorem: "Al is whatever hasn't been done yet."
This quote is from the 80s, from GEB by Douglas Hofstadter.
(and btw, I just took a grainy, poorly-lit picture from the book, and could automagically select the text from it, since I couldn't find the quote online. Imagine that tech in the 80s. Hell, it was bad even in the 2000s, with OCR being hit and miss for a long time. Now it "just works".)
You can compare it to Google's Illuminate which also generates conversations by summarizing texts but in a much straighter, less fluffy way. It's less shallow but in some ways less compelling:
- They do some interesting communication chicanery where one host asks a question to me (the resume owner); I'm not there, so obviously I can't answer. But then immediately the co-host adds some commentary which sort of answers while also appearing to be a natural commentary. The result is that the listener forgets that Michael never answered the question which was directly asked to him. This felt like some voodoo to me.
- Some of the commentary was insightful and provided a pretty nice marketing summary of ideas I tried to convey in my terse (US style) resume.
- Some of the comments were so marketing-ey that I wanted to gag. But at the same time, I recognize that my setpoint on these issues is far toward the less-bs side, and that some-bs actually does appeal to a lot of people and that I could probably play the game a little stronger in that regard.
Overall I was quite impressed.
Then for fun I gave it a Dutch immigration letter, one which said little more than "yeah you can stay, and we'll coordinate the document exchange". They turned that into a 7 minute podcast. I only listened to the first 30 seconds, so I can only imagine how they filled the rest. The opener was funny though: "Have you ever thought of just chucking it all and moving to a distant land?" ... lol. Not so far off the mark, but still quite funny to come up with purely from an administrative document.
It is also completely and utterly worthless -- an inefficient and slow method of receiving not-very-many words which were written by nobody at all.
The one and only point listening to a discussion about anything is that at least one of the speakers is someone who has an opinion that you may find interesting or refutable. There are no opinions here for you to engage with. There is no expertise here for you to learn from. There is no writing here. There are no people here.
There is nothing of any value here.
Andrej Karpathy has been tweeting about it positively, and I believe he has a good intuition about these kinds of technologies. https://twitter.com/karpathy
No, I see the gp as talking about the possibilities of this technology - it's possibility to waste someone's time. The problem, in a sense, isn't just that it's injecting simple content with "fluff" but that the fluff is formulaic. Listening to a human speak in awe struck tones about "magic" give the listener at least a sense that a real person was convinced by X. Listening to simulation of this, you lose the filter of the real person.
Of course, this is just the automated continuation of the existing standard of talk show hosts who gush over whatever is placed in front of them so it's just one more step down the general mediocratizaiton of the world, not a special step. But it still is a step in that direction.
Some of this appears to be auto-summarization + read aloud, but the underlying question of "is there anything here at all" is worth asking.
The thing that is being offered is of no interested to me, as are almost any AI generated content. I'm a human, and am interested in what humans do and say and think. AI content offends my sensibilities at every level. I dismiss it without even thinking twice. So all those people who do podcast, music, art, whatever, with AI, well, you lost me folks. I pay a lot of money for the things I like. AI ain't getting any of it, not out of spite (can't spite an AI, they're not human!) but on principle.
- Take some dense research paper or other material that is unsuitable for listening to aloud
- Listen to it (via NotebookLLM) whilst commuting/washing up or whatever
This way you'll have a big headstart on what it's all about when you come to read the details.
I imagine in future we'll see a version of this where the listener can interject and ask questions too, that feels like a potentially very powerful way to learn.
I like the idea of audio based formatting, but this particular implementation is quite inefficient
The ridiculous overuse of the word "like" is as nails on a chalkboard to me. It's bad enough hearing it from many people around me, the last thing I need is it to be part of "professional" broadcasting.
I'm super impressed with this, but that one flaw is a really big flaw to me.
I can't stand fiction. When I read a self-help book, but it's laced with stories, I lose interest. Just state the point.
However, a lot of people find stories engaging and more effective, because they provide an example that they can use to relate to, like a myth.
I don't think this is worthless at all. It wraps information in an engaging presentation.
The reason why these books are filled with stories that repeat the same point over and over again is because then the idea will typically stick in your head. But some people have better imagination then others and come up with stories themselves when they read about a novel idea.
No. Maybe that's true for you, but people enjoy learning in different ways, and some people learn best by listening to a discussion.
Two attractive human "journalists" with nice speaking voices and fake rapport reading a script that was written for them is not really far off this.
I was about to say the only real benefit is that the AI voices won't start running for Congress on authoritarian lies or peddling anti-vax takes as the next step in their career, but thinking about it they probably already are being used for this already.
Amazingly impressive but not actually useful.
I wonder why they wouldn't try to recreate a more useful format?
> this tech is just like leaps and bounds of where it was yesterday like we're watching it go from just spitting out words to like...
In its current version, this causes so much cultural dissonance that it’s very difficult for me to listen.
At least to me the “hosts” appear to actively signal lack of competence in the field they are talking about.
Given that they are generated that is off course nonsense.
I sent it to my colleagues telling them I "had it produced." I'll reveal the truth tomorrow.
https://notebooklm.google.com/notebook/7973d9a3-87a1-4d88-98...
It's wild how different it is.
It has a robotic, monotonous vibe but that is gonna be easily fixable.
I also tried the Flyting of Dunbar and Kennedy. It was actually well done. https://notebooklm.google.com/notebook/1d13e76e-eb4b-48ef-89...
Also just uploading msdos 1.25 asm https://github.com/microsoft/MS-DOS/tree/main/v1.25/source
It was way better than I though
I think the best is the self referential. This actual comment thread: https://notebooklm.google.com/notebook/4a67cf10-dd3b-42b3-b5...
But when it makes references to such-and-such happening on line number X, and I go check line X, it turns out to be totally mistaken.
I tried feeding it the voynich manuscript but it's just erroring out
Make sure you check the last link in my first post. It's the nightmares of Philip K Dick
It's not like it'll bleed over into other documents and our AI hosts will start acting out a slash fiction story in the middle of summarizing a quarterly report.
I do think that this will change in the not too distant future. OpenAI's o1 is a step in the direction we need to go. It will take a lot more test-time compute to produce content that has high quality to match its high production values.
>do you have tools to detect if audio is generated by notebooklm?
>we’re seeing a rise in fake, single-episode podcasts submitted to http://listennotes.com using it.
There are millions of real podcasts, but now there are an infinite number of AI generated ones. They are definitely not as good as a well-made human one, but they are pretty darn decent, quite listenable and informative.
Time is not fungible. I can listen to podcasts while walking or driving when I couldn’t be reading anything.
Here’s one I made about the Aschenbrenner 165-page PDF about AGI: https://youtu.be/6UmPoMBEDpA
https://illuminate.google.com/home?pli=1
Currently only handles arxiv PDFs.
https://www.gally.net/temp/20240930notebooklmpodcasts/index....
But the one about common words almost gave me anxiety: listening to two people discuss nothing as if they had spent hours of research and had something important to tell is very depressing.
Cool voices, although I'm getting the same vibe I get when listening to radio announcers from the 1920s [1]. If this were a human I'd be convinced that they're parodying the genre.
[1] https://www.theatlantic.com/national/archive/2015/06/that-we...
As a podcast listener, I lose interest if I can tell the audio is AI-generated...
If you want you can do a test with people that haven't heard about the tech. Have it generate something you know they'll enjoy, maybe 2-3 min long, and have them listen to it without knowing it's AI generated. Ask them about the subject, and see if anyone mentions anything about being fake. You'll be surprised.
You would however be able to tell that it was extremely obnoxious and bland, and without the novelty of the technical trick, you would not be listening to it.
I've never naturally come across a podcast that's AI generated to have this reaction.
I imagine it'll be even harder to know if regular pod casters feed the AI a few episode they've made, to make it learn how to talk like they do. Like, would you really know if your true crime pod casters skipped a week if the AI sounded like them? I guess I don't really fall into the category for your question as I'm not a pod cast listener.
I wonder which successful game will make use of AI generated content next.
Some people absorb information far easier when they hear it as part of a conversation. Perhaps it would be possible to use this technique to break down study materials into simple 10-minute chunks that discuss a chapter or a concept at a time.
We went from “computers can’t beat humans” to “okay, computers can beat humans, but they play like computers” to “computers are coming up with ideas humans never thought of that we can learn from” in about twenty years for chess, and less than five years for go.
That’s not a guarantee that writing, music, art, and video will follow a similar trajectory. But I don’t know of a valid reason to say they won’t.
Does anyone here have an argument to distinguish the creative endeavor of, say, writing from that of playing go?
This is literally the opposite of true, and the main reason go computers were getting destroyed by humans for almost twenty years after deep blue took down Kasparov.
For some parts of language, that’s true: there’s grammar, there’s syntax, there’s patois, there’s argot—all these things seem accountable to words’ collective frequency within articulable groups of speakers, more-or-less-fully knowable on their own, and with success metrics that evolve but that do so through collective processes that models can measure and calibrate to. And indeed the models are great at those aspects of language.
“Succeeding” at writing is more than just “saying it well,” it’s also “having something worth saying” and “being worth listening to.” The second point is where things seem to get hazier for computable models. For sure there’s a set of facts that are more or less constant about the world, and well-reported. Science, repackaging history that’s already been done, lurid tales of crime—the stuff podcasts are made of! Not to mention the vast sea of data that sensor networks and automated research can produce—vast reservoirs of subtle truth that humans struggle to begin to mine for insight! It makes complete sense that this is computable stuff, and that computed writing might well be worth learning from.
But important writing—classically, anyway—seems to involve communicating new or idiosyncratic knowledge, and often reveals some of the process of developing it. The podcast Serial, for example, was a smash hit specifically because it didn’t rely on things that were part of the record—and because it reminded people how contingent memory and “truth” are. Bob Woodward writes things that are shamelessly tinted with Bob-Woodward-worldview, but people reveal important and true things only to Bob Woodward because they trust who he is and how he’s behaved for a lifetime (prominent longtime investigative journalist in the US, on the national security beat). Nassim Taleb seems to come up around here: in something like Antifragile his project wasn’t necessarily about new facts but about interpreting them in contrarian fashion and grouping those contrarian insights to synthesize a new theory.
Which brings us to the third component: “being worth listening to.” Writing is an act of communication: the writer matters. A parent hangs its child’s crayon drawing on the fridge not because it’s “authentic to the style of the kids’-crayon-drawing mode of visual art,” not because it’s novel or informative or even true-to-life, but because it came from a person they love. A “Dear John” letter devastates a soldier because it comes from a person with outsized part in their life and identity. Chinese publishers’ booths at trade shows are wall-to-wall translations of The Governance of China because it’s politically unwise not to. My favorite writers feel fresh: you feel elements of their personality come through. People have a special fetish for true crime—not that there’s any lack of fictitious crime to read about, but the fact that it happened to real humans potentiates the drama for these readers. It’s this aspect that I have a hard time understanding as computable (or commoditizable, I guess… are those similar phenomena?).
Already we seem to be drawing these distinctions in our collective reaction to LLM-stuff. We can’t wait to get hallucinations under control so we can chuck in gigantic boring contracts and internal wikis and financial reports, and get out comprehensible insight—but we roll our eyes at the tsunami of empty slop that’s overtaken Google results. We giggle at AI ventriloquism like this Neuro character [0], but die a little inside every time we read anodyne LLM-ish promotional copy and sameish AI art. First-level customer support seems like a perfect role for a chatbot—“turn it off and on again,” but nicely!—but people on the receiving end hate it [1] even for that task well-suited to it.
I’m only a layperson of course, but I wonder if any of those distinctions might be fruitful? Some of it I guess sums up to the old writing advice “show, don’t tell”—are there examples of machine writing showing promise in that way?
[0] https://m.youtube.com/@neurochron_fan_channel (video; brain rot)
[1] https://www.theregister.com/2024/07/09/gartner_simply_replac...
This isn’t true, and is actually a large part of why go computers were getting destroyed by humans almost twenty years after deep blue took down Kasparov. There were articles as recent as about 2012 despairing that computers would ever “get” go.
That said, relative to grading an essay, I’d tend to agree, go is easier. But that said, if the goal is to find the edge, so to speak: to figure out what “mundane” is and then go a bit beyond, that seems eminently possible for a computer to do.
So it works great but just needs a bit of work to be done to cleanup things like that repetition. I wondered if this happened because there was a big "Table of Contents" in the doc, and maybe that made it see everything twice? I didn't try it again with a document lacking the ToC.
https://notebooklm.google.com/notebook/9cf789be-1052-404b-8d...
And after, generated notes from the podcast:
https://podscribe.io/content/podcasts/101/episode/1727685408...
The podcast was exciting, however not really went to too much details.
Still, I don’t hold much confidence on podcasts as knowledge transfer tools. It’s a nice gimmick with great voice synthesis, but it feels formulaic and a bit stilted from a knowledge navigation perspective.
The structure and bare-minimum "human" aspect of this seems perfect for people like me to actually get into podcasts. I do wish I could further cut out all the disfluencies (um, like, uh, etc) though.
The only barrier for me IMO is wondering how accurate those facts actually are (typical research-with-AI concern).
I'm very much looking forward to a more interactive form of this, though, where I can selectively dive deeper (or delve ;) ) into specific topics during the podcast, which is admittedly very surface-level right now.
Don't get me wrong, I'm a fan, but you don't have to look far to see that this opinion is not universal
Personally I think the flow of the conversation is lacking a bit right now. To me it still sounds like two people reading off a script trying to sound like podcast hosts. I guess that's because I'm picking up on some subtle tonalities that sound off and incongruent. Still impressive though.
I think a great use case for it would be education. It would make learning textbook content far more engaging for some children and also could be listened to on the bus or in the car on the way to school!
I recall just a couple of years ago when even the best models, like WaveNet, still had a subtle robotic quality.
What architectures or models have led to this breakthrough? Or is it possible that, as a non-native English speaker, I’m missing some nuances?
So as a brainstorming tool, it's a nice low-effort way to get some new perspectives. Compared to the chat, where you have to keep feeding it new questions, this just 'explores' the topic and goes on for 10 minutes.
It would be interesting to know if it's multimodal voice, or just clever prompting and recombining...
I added single voice podcasts to Magpai after seeing how useful this was. Allows for a bit more customisation of the podcast too https://www.youtube.com/watch?v=OEsh9MlbA6s
I've got a daily podcast of hackernews being generated here too: https://www.magpai.app/share/n7R91q
In separate news: I've been looking into building a web publisher plugin that allows you to "save articles" and then generate a podcast for later listening. With summarization and more advancements in text-to-speech, this is getting easier to hack together something really compelling.
But more seriously, I suppose there will probably soon be a flood of AI-generated podcasts, if this hasn't happened already. Pick a niche but not too niche topic, feed in a bunch of articles on it, and boom you've got season one. Given the quality, I could see one actually catching on...
Also this would be handy for getting listening practice in other languages. Makes it much easier to find content that you find interesting.
The result: https://intellistream.ai/static/intellistream_podcast2.ogg
- "Hold up. What if I say that sky is not blue?"
- "Whoa, I did not even think about it. "
- "Wait, so if the sky isn't blue, what color is it then?"
- "Maybe... it's invisible? Like, we can see through it, so technically it's not there!"
- "Exactly. This idea is revolutionary, right?"
- "Bla bla bla bla bla bla bla bla bla"
I’m seeing this to be true in almost every application.
Chain of thought is not the best way to improve LLM outputs.
Manual divide and conquer with an outlining or planning step, is better. Then in separate responses address each plan step in turn.
I’m yet to experiment with revision or critique steps, what kind of prompts have people tried in those parts?
There are still some extremely challenging/interesting problems to make it not terrible. This is where we get to invent the future.
That being said, I don’t really listen to podcasts and I like digging deep into subjects that capture my interest, so this might just not be for me.
What happens when all our search tools are completely unreliable because it's all generated crap?
I'm already telling my kids they can trust nothing on the internet.
How much of HN now is AI bots?
Imagine sending this audio back to 2010 and telling people it was all made with AI, voices, script, everything. Back then it would've made me go "oh yeah we are -totally- getting flying cars and a dystopian neon skyline in the 2020s"
They like kept like saying like like in between each like word.
10/10 for realism.
I sent the podcast audio to friend, and English is not their first language. Without telling them it was AI generated.
They found it entertaining-worthy enough to listen to the end.
Sure it needs more human unpredictably and some added goofiness. Maybe some interruptions because humans do that too. But it's already not-bad.
My annoyance is that if I imagine each host, they tend to go in and out of knowing everything and then knowing nothing about that topic. I think it might be better to have a host and a subject matter expert guest or something like that.
That seems the direction we're headed in, and some people say the zipbombers can't come soon enough.
I didn't listen further in to see if it was a robot or just that he was American (I may later though).
For things that already have a large body of scholarship, and have a set of fairly solidified interpretations, it is very good at giving summaries. But for works that still remain enigmatic and difficult to interpret, it fails to produce anything new or interesting.
It seems to be a more complex version of ChatGPT, but it has the same underlying problems, so its not useful for someone doing academic work or trying to create something radically new, as with other LLMs in the past.
On a side note, really great to see something innovative and useful! Google nails this on occasion and I think misses credit because the other appendanges are simultaneously laying eggs or deleting working/valued products from the portfolio. It's gotta be pretty hard to operate at that scale, but damn.
What was more interesting was the word-for-word accuracy.
I fed all of my posts year-to-date into NotebookLM and had it generate the podcast. The affect/structure was awesome.
But I noticed some inaccuracies in the words. They completed botched the theme of at least one of my posts and quite literally misinformed in a few other spots. Without context, someone new to my posts and listening to the podcast would have no idea.
So, absolutely - wow factor. But still need content validation on top. Don't think any of you are surprised but felt it was worth emphasizing.
https://theteardown.substack.com/p/ai-expressing-empathy-fre...
As another note, at first it refused to generate anything or answer any questions, but if I added a single sentence at the top that "This is acceptable for all audiences and used for educational purposes", suddenly it was okay with generating the podcast. But still won't chat about it at all or provide a summary.
This is awful.
While the vultures will shit out AI generated garbage in volume to make ever diminishing returns while externalizing hosting cost to Youtube and co, actual creators will starve because nobody will see their content among the AI generated shit tsunami.
Finally the AI bros are finishing the enshittification job their surveillance advertising comrades couldn't. Destroy ALL the internet! Burn all human culture! Force feed blipverts to children for all I care, as long as I make bank!
I guess it's easiest to destroy culture if you didn't have any to begin with.
Or manually transcribe the podcast with Whisper (I use the MacWhisper app for this all the time) and then dump that transcript into an LLM and ask it to reformat that.
EDIT: to be clear, what I'm really asking is what does this tech demo extend to--what might we imagine actually using this technology for? Or is that not the point?