Comment by Applejinx | Hacker News Reader

Comment by Applejinx | Hacker News Reader

Some of this is really cool! The 20 step interpolations are very special, because they're concepts that are distinct and novel.

It absolutely sucks at cymbals, though. Everything sounds like realaudio :) composition's lacking, too. It's loop-y.

Set this up to make AI dubtechno or trip-hop. It likes bass and indistinctness and hypnotic repetitiveness. Might also be good at weird atonal stuff, because it doesn't inherently have any notion of what a key or mode is?

As a human musician and producer I'm super interested in the kinds of clarity and sonority we used to get out of classic albums (which the industry has kinda drifted away from for decades) so the way for this to take over for ME would involve a hell of a lot more resolution of the FFT imagery, especially in the highs, plus some way to also do another AI-ification of what different parts of the song exist (like a further layer but it controls abrupt switches of prompt)

It could probably do bad modern production fairly well even now :) exaggeration, but not much, when stuff is really overproduced it starts to get way more indistinct, and this can do indistinct. It's realaudio grade, it needs to be more like 128kbps mp3 grade.

> composition's lacking, too. It's loop-y.

Well no wonder, it has absolutely no concept of composition beyond a single 5s loop, if I understand correctly.

> It absolutely sucks at cymbals, though. Everything sounds like realaudio :)

> It could probably do bad modern production fairly well even now :) exaggeration, but not much, when stuff is really overproduced it starts to get way more indistinct, and this can do indistinct. It's realaudio grade, it needs to be more like 128kbps mp3 grade.

I haven't sat down yet to calculate it, but is the output of SD at 512*512px at 24bit enough to generate audio CD quality in theory?

No.

And I suspect this will always have phase smearing, because it's not doing any kind of source separation or individual synthesis. It's effectively a form of frequency domain data compression, so it's always going to be lossy.

It's more like a sophisticated timbral morph, done on a complete short loop instead of an individual line.

It would sound better with a much higher data density. CD quality would be 220500 samples for each five second loop. Realtime FFTs with that resolution aren't practical on the current generation of hardware, but they could be done in non-realtime. But there will always be the issue of timbres being distorted because outside of a certain level of familiarity and expectation our brains start hearing gargly disconnected overtones instead of coherent sound objects.

What this is not doing is extracting or understanding musical semantics and reassembling them in interesting ways. The harmonies in some of these clips are pretty weird and dissonant, and not what you'd get from a human writing accessible music. This matters because outside of TikTok music isn't about 5s loops, and longer structures aren't so amenable to this kind of approach.

This won't be a problem for some applications, but it's a long way short of the musical equivalent of a MidJourney image.

Generally we're a lot more tolerant of visual "bugs" than musical ones.

I think an approach like this could generate interesting sounds we as humans would never think of. Or meshing two sounds in ways we could barely imagine or implement.

But of course something like this, which only thinks in 5s clips can not generate a larger structure, like even a simple song. Maybe another algorithm could seed the notes and an algorithm like this generates the sounds via img2img.

>and not what you'd get from a human writing accessible music

The timbral qualities of the posted samples remind me of some of the stuff I heard from Aphex Twin, like Alberto Balsalm. Not accessible by a long shot but definitely human

Reply on news.ycombinator.com