Deep Learning Could Bring the Concert Experience Home | Hacker News Reader

Deep Learning Could Bring the Concert Experience Home

spectrum.ieee.org

23 pointspseudolus20 comments

> And yet even now, after 150 years of development, the sound we hear from even a high-end audio system falls far short of what we hear when we are physically present at a live music performance. At such an event, we are in a natural sound field and can readily perceive that the sounds of different instruments come from different locations, even when the sound field is criss-crossed with mixed sound from multiple instruments. There’s a reason why people pay considerable sums to hear live music: It is more enjoyable, exciting, and can generate a bigger emotional impact.

This is unintentionally hilarious, and true basically only if you're listening to jazz with 50 other people or less, or a small minority of seats at a classical music concert.

99% of the time when you're at a live music performance, you're listening to sound coming from big speakers all mixed together with zero spatial differentiation, and the sound quality is far muddier than on even the cheapest headphones.

People go to concerts for the in-person, real-life, emotional experience. Not the audio quality or separation -- sheesh!

Maybe the article has some interesting content, but it's hard to take it seriously when it has such a ludicrous intro as that.

The point of a concert is barely the music. It's the experience of GOING to a concert. A concert at home sounds boring as shit doesn't matter how you tweak the sound. You don't get to dance or mosh or crowd surf or jump up and down with hundreds of other people at home.

I just watched the Brandi Carlisle live IMAX show last night and it changed some of my opinions on this. Incredible sound quality better than any concert I’ve been to, and a live experience surrounded by other people.

Are you kidding me? Most people I know complain how live music sounds worse than the recordings they're used to listening to, and this has been true for decades! I don't know anyone, and I'm a musician, who goes to live music because they think the music sounds better. No one.

What people do enjoy is the entertainment and seeing improvisational playing or flourishes added to their favorite songs. Plus much has been written about people enjoying the community aspect of live music: people enjoy being together sharing an experience with an artist they all enjoy. They don't go because the music sounds better and no matter what you do you can't bring the experience of being with others to you sitting alone in your living room.

It depends on what music you are talking about. One of the reasons that people attend nightclubs that play electronic music is to feel the low-end (literally). Most people will never have the space or means to equip their home with that kind of equipment and play at that volume. Certain genres of electronic music are specifically produced and mixed for big sound systems.

However, the article tries to sell the fact that live music is more 'accurate' or 'precise':

> At such an event, we are in a natural sound field and can readily perceive that the sounds of different instruments come from different locations, even when the sound field is criss-crossed with mixed sound from multiple instruments. There’s a reason why people pay considerable sums to hear live music: It is more enjoyable, exciting, and can generate a bigger emotional impact.

Which is an odd claim and one that I will disagree with.

> At such an event, we are in a natural sound field and can readily perceive that the sounds of different instruments come from different locations

Huh? Maybe for orchestral music or small house shows without a centralized PA. Any other concert is going to have the musicians mic'd or going line-in to the mixing board and then played in stereo through massive speaker stacks. At a concert with more than 1,000 people you're not hearing any of the onstage sound.

This will be great for chamber music or orchestra but I don't see how is this going to work for EDM because it often has very little to do with "organic" sound that can be split neatly into separate channels and "put" on a virtual sound stage

...yet. Electronic musicians generally haven't had tools that let them place instruments in 3D space, but give them the tools and they might well find uses for them.

Maybe they won't use it for dance music specifically but electronic music isn't just for clubs.

The premise of the article is that you can take any existing stereo music and convert it to 3D "magic" space using AI because it is not economically possible to do it manually, no body is going to re-mix millions of tracks for 3D.

My point is that no "magic" AI is gonna work on electronic music because it simply is not natural sound you can target algorithmically and produce something that makes "sense"

I agree with you that there wont (or shouldnt) be a one size fits all AI solution for millions of different pieces of music. The same reason why mastering audio is considered an art form as much as a science applies here-- it involves deep understanding of intent which goes far beyond simply analyzing waveforms

Theres also a lesson to be learned about using sorts of style transfer to add dimensionality to past artwork which is that maybe the artist would have deliberately omitted these new dimensions even if they were made available at the time, so the new dimensiona effectively change the original intent, and shouls may be considered an entirely new artwork with somewhat different authorship and different message altogether

What is so special about "not natural sound" that precludes algorithms to work on it?

The producer would have to agree that the stereo to "magic AI" upmix is what he wanted? Get it? A human would have to agree that the upmix makes or does not make sense to him or her because ultimately it is his or her work, not AI's work. This is easier with chamber music or orchestra because it is just natural instruments at some defined position but electronic music is just a big mess positionally with no rules that AI can tackle sanely...

The analogy here would be if we started converting 2d paintings to 3d scenes and then we would claim that it is what the artists imagined. It is not!

The one thing that I appreciate over a real concert versus a recording, is attention. At a concert, I'm forced to close my phone, sit up, and pay attention. There is nothing else there to do, so I do.

At home, I don't last 5 minutes listening to a recording. I'll pick up a book, my kids will walk in, and before you know it we'll have Netflix on.

Come to think about it. There are surprisingly few concerts on Netflix. Seems like an easy niche to fill.

>At a concert, I'm forced to close my phone, sit up, and pay attention

I'd love to know what concerts you're attending! I see phones everywhere, to the point where I am moving my head / shifting spots on floor (for example) to make sure I can watch the concert with my eyes instead of recording a video I'll soon forget.

I doubt if the aural sensor makes all the "realism" of being in a concert.

Need to think about it from musicology perspective.

I tried the app, but found it gimmicky. I preferred the original mix, and really didn't care if I could emphasize one instrument or another -- the recording engineer already did a better job than I could. Also, it seemed to add some noticeable distortion to the music.

i dont quite understand how upmixed tracks are going to sound better than an original file through a generously amplified minimal hifi system. Upmixing gives artifacts - unless this algorithm magically separates audio without them - this will be worse. i get incredibly holographic sound with full range speakers (no phase distortions from having tweeters/mids). All i have to do is press play - i dont configure anything - i've never once wanted to turn up and down elements in a mix while listening to them.

> Upmixing gives artifacts - unless this algorithm magically separates audio without them

We've all seen deep learning accomplish some pretty magical things over the past couple years, so I wouldn't rule that out.

Misses the whole social aspect of the concert experience

>We believe that soundstage is the future of music recording and reproduction.

Sorry but your market is dying and the money is drying up and the trend is moving the other direction for 95% of recorded content that people engage with on a daily basis. This platform is great for cinema or high end VR projects. I’d rather hear a recording made in Jack White’s one-take-cut-a-record booth at Third Man Records than whatever style this might be used for in a practical music recording sense.

Also, we have enough proprietary and patented audio things to last a couple more lifetimes. 320kbps and a pair of Sennheiser headphones is better than most listeners will ever need. Very few people listen to music in ideal or even decent environments. That’s why we made music portable, so we take take out love of it places where it’s not accessible otherwise (unless you’re like me and bring your own guitar to the park).

In short, release it free for all to use or prepare to be called Sony-Lite for doing proprietary user hostile things.

Reply on news.ycombinator.com