Training neural nets on MIDI files seems to allow them to pick up these structures (better than unpicking them from raw audio) but it's a lossy format which misses all the unique instrument sounds and nuances of playing and sounds of the human voice.
The Bon Jovi / Chopin crossover here isn't particularly musically interesting, but mainly it doesn't fit the prompt because MuseNet has no idea what Bon Jovi's voice or guitar tone sounds like https://openai.com/blog/musenet/ And yes, it's also unimpressive because a toy script could add drums and bass and variations to the piano hook...
By analogy, I guess it's like training an AI image generator on polygons and textures. You'd end up with coherent geometry and the ability to look at the asset from different perspectives which might be useful to generate video as well as static assets, but the images would look like late 90s computer games, not photographs and wouldn't be difficult to procedurally generate in other ways.
Training on raw audio could pick up the nuances in tone and timing but lose the differentiation between instruments and notes in the mix which allows the music to be rearranged. I guess an idealised training set would include track by track raw audio and notation: but good luck getting the music industry to licence that as a training set...
Though at the same time, it seems like Dall-E2 is really good at faces [2], which is an example where there is a lot of structure, and viewers are very good at finding mistakes (the "uncanny valley"). Dall-E2 is of course a much bigger model with more training than Dall-E mini. Perhaps with the same amount of training, music AI can get to the same level.
[1]: https://news.ycombinator.com/item?id=31787743
[2]: https://www.vice.com/en/article/g5vbx9/dall-e-is-now-generat...
Regardless of opinion on audio files, it's the case that many AI approaches to music have focused on symbolic representations of music, which works, as do non-NN ways of procedurally generating music using the same symbols. What AI can theoretically add to a procedurally generated midi track is expressive performance, but that's not exactly its strength and probably needs better curation than just giving it lots of audio files.
And even the "pure audio" approaches like OpenJukebox have to break out vocals and treat them separately (because it's something of an understatement to say that reflecting a static image is not a more complex problem than generating four minutes of singing with appropriate pitch, intonation and emotion and the impression of a consistent genre-appropriate vocalist), and the background music is... kind of muddy sounding and compositionally uneven.
The main thrust of my argument wasn't the complexity of music though, the main thrust is comparative difficulty of compiling the training sets. More work to annotate and more licensing complications than just scooping up public domain images especially if you want a well rounded model. For another example take OpenAI's jukebox. It's clear the "Heavy Metal In The Style of Rage" vocals use a model trained on metal bands, but it's also clear that Rage Against The Machine wasn't prominent amongst them. Also probably the case there's simply not enough Rage Against The Machine albums to get your digital Zach de la Rocha vocals out of the uncanny valley even with the relevant permissions
(The answer to any motivational questions will probably always be, "strongest signal". An artistic motivation will probably never be to just express the strongest signal in a common cultural context, rather, it will be contrast and interpretation. Otherwise, we should start worrying about that given society.)
An image is likewise limited by representation in that we define against a resolution, color depth, gamma, etc. Images define a large number of sample sources, but without variation over time. And in most circumstances, like with audio, we also currently limit the sources to fixed areas, rather than generating surround images.
What we're relying on in using these formats successfully is knowing the limits of our sensory perception, and then what our brain does to process it. It says nothing about "A is easier to represent symbolically than B", because of course you can come up with symbolic compression schemes; those are how humans generate interesting images and audio. But the transmission protocol doesn't need to know them.
https://magenta.tensorflow.org/perceiver-ar
Here is an older project that generated audio from lyrics:
https://openai.com/blog/jukebox/
Music has two key difficulties: the long, linear context makes it hard for most models to generate coherent works and reason about long-term higher level structure, and the global structure being learned is much more function of human preference, rather than of physical constraints.
That said, I don't think the gap is all that sizeable. Jukebox is easily comparable in quality to the image synthesis from arbitrary text of its time. Perceiver AR is not as impressive as DALL-E 2, sure, but DALL-E 2 is new, and it's plenty of evidence that music synthesis will continue to work.
As an enthusiastic amateur producer of music using desktop PC MIDI software and instrument plugins, I agree. The samples you linked already seem coherent enough to at least be useful in helping untrained amateurs like myself come up with ideas for subsequent musical phrases by proposing variations based on my existing midi chords, BPM and perhaps a genre selection. It would only need to suggest plausible continuations to be useful because if the suggestions are in MIDI, it's easy for me to riff and revise any that sound interesting. Even a 'hit rate' of one in five suggestions would be compelling for the 'hobbyist-assist' use case.
It seems like the DeepMind PerceiverAR that you linked generates entire piano pieces all at once, instead of generating token by token like OpenAI GPT or Jukebox. So that should take care of long-term coherence right? Since all parts of the song are processed and generated at once
Except we haven't had as much development in AI music yet. It's still at the "dall e mini" equivalent.
You can literally just play three chords and have a pop song. Play the same string, hell, the same note, on an electric guitar and bass and you have a metal song, as long as you get a good rhythm. The bar for algorithmic music is low.
Except AI doesn't use algorithms. It imitates patterns that it sees, and probably doesn't get the pattern as well as someone who knows a little music theory.
Is DALL-E 2's work remarkable if you saw it on art station? No, it's just remarkable because of who made it.
Also, consider the difference in challenges. I've never heard of someone instructing an AI by "a short series of bars in the style of Beethoven with a fire siren in it", and then going, "yeah, this is a fire siren, this is great music!" There are certainly more criteria than this to be satisfied for this being considered great art. (In theory, this applies to visual arts quite the same, but we tend to be satisfied at a much lower level, like, "yes, it's a fire engine!")
(That said, there has been this long-nursed idea in art theory that music would be the highest of all arts, for being both the most abstract one and also the most direct one. One may assume that a higher degree of abstraction would favor computer generated content, the emotional dimension, however, may give it away more easily.)
I'm sorry to break it to you but this just sounds like you're not a music person. The answer for me is close to 95% I find most music I hear interesting e.g. most music I listen to on Spotify can be classified as impressive as I derived adequate amount of artistic enjoyment out of it to be impressed.
My gut says it's mostly the first bit, though. We probably just have a lower entertainment threshold for randomized images.
This is true even when no machines are involved. Music of the same standard as your average fan art on the internet is nothing anyone would listen to. Even average consumers are simply more discerning when it comes to music. Almost everyone you can ask on the street has a favorite musician and can tell you a few things about different genres, how many people have a favorite painter and can tell you something about art? It's just a cultural thing.
Although maybe it would work, generating with a very low sample rate and upscaling that..
Launching soon.