Show HN: YouTube Musical Spectrum – visualizer for YouTube with musical notes | Hacker News Reader

Show HN: YouTube Musical Spectrum – visualizer for YouTube with musical notes

github.com

72 pointsmfcc6432 comments

Cool, man! I've wanted a standalone player to do this ever since WinAmp used to support this 20 years ago, but seemingly the capability was lost to time when everyone decided they'd rather watch recorded video. The composer Stephen Malinowski's YouTube channel is one of my favorites because all of the videos are visualizations of music he's generated with custom midi-reading software he made, i.e.: https://www.youtube.com/watch?v=ffYKCNY6kUk. He first wrote this software in 1985 a decade after getting the idea from listening to Bach on LSD.

It's just too bad this is a browser extension because I really don't want more of those. I get that the economics probably don't make sense to do any more, given the effort to make a standalone desktop app would be a lot and a website people could upload tracks to in order to get a visualization would force you to pay hosting costs.

Actually, you can use mpv player and install visualizer.lua on mpv-scripts repository. The engine of YouTube Musical spectrum (showcqt-js) is a javascript port of ffmpeg showcqt filter.

For any Arch based users, I made an AUR entry for this;

https://aur.archlinux.org/packages/mpv-visualizer/

That Stephen Malinowski link was great. I don’t know that song by name, but I could unmistakably tell what song it was just by looking at the visualization (even though I left the video muted)!

Also a fan of Malinowski's animated music videos. My immediate thought was that it would be cool to see this extension sort of mimic his piano roll animations by having it appear vertically on the left. Less pixel space though, so I don't know...

Would also be cool if the dots on the pitch line that represent accidentals were perhaps a slightly darker shade so it'd be easier to see the typical piano pattern in them. (Like ⦾⦿⦾⦿⦾⦾⦿⦾⦿⦾⦿⦾ kinda thing.)

Anyway, to original poster: very cool extension, thanks for sharing! :)

For a website, probably you want to try this example at https://mfcc64.github.io/html5-showcqtbar/

When I read notes, I express through interpretation as a spectrum of emotion. So this spectral visualization is pretty much how I visually interpret musical notes, especially when playing by ear.

Can you detect notes with missing fundamental?

I left this comment with a video screencapture of the extension in action > https://github.com/mfcc64/youtube-musical-spectrum/issues/2

> As someone who has been learning how to sing for the past year, I have tried many spectrum tools and have come across one that is really useful for singing with my own voice but isn't that great for multi-frequency visualization and therefore isn't that good for learning other songs but THIS extension really fills a void for learning other songs!!

I like how I can tell the voice from the instruments in this song by the vibrato visualization: https://www.youtube.com/watch?v=450p7goxZqg

Awesome to see projects like this! I have music-to-visual synesthesia when smoking pot and making something that shows how music looks to me is a project that I’d like get to sometime also. My problem is that I don’t really care to get high often enough to get it done. It would look quite different than yours or Stephen Malinowski’s visualizations.

Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension.

Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc.

>Why does the extension on Firefox block AV1 by default? It seems unrelated to the purpose of the extension.

Because AV1's decoding is slow and on some low-end hardware it makes the animation stutter.

> Also have you considered coloring the spectrum according to the musical note? So every C is red, every E is green, every G is cyan, etc.

The color is used for left-right channel separation.

cnlohr's colorchord does something similar to what you mentioned.

https://github.com/cnlohr/colorchord

This is pretty cool.

Does anyone know any stand alone feature rich audio visualizer for Windows? I am surprised that Spotify never included any visualizer for the desktop client. I miss the old Winamp days.

There is a standalone visualizer that is my go to, projectm. It's a spiritual successor to the visualizer from winamp days and is compatible with the old milkdrop presets.

I haven't been able to find a proper way to set up mesh size on windows but if you're running it on linux or android, those options are available too.

https://github.com/projectM-visualizer/projectm

Just tried it. Like you said, it mimics Winamp Milkdrop but it has not aged well.

There are apps like Plane9 (very similar to Milkdrop from Winamp) and the commercial Luminant Music. I'd say FL Studio's integrated plugin ZGameEditor Visualizer is probably the top alternative considering the sheer number of presets and templates as well as configurability. They're all listed at AlternativeTo.

https://alternativeto.net/software/projectm/

This is a very unique, fascinating; useful - and - above all - interesting utility. Thanks for doing the work on this. :).

Thank's :).

I knew there was a payoff from browsing HN today!!! This was totally worth it!!!

Thank's for the support.

I'd like this visualization on Audacity.

I'll be glad if someone ports it to Audacity.

Which transform are you using?

The visualization engine is showcqt-js (https://github.com/mfcc64/showcqt-js). It is a javascript port of ffmpeg showcqt (Show Constant Q Transform). Probably, more correct name is Variable Q Transform.

I've implemented the Constant Q transform before, and it does work well for western music. Are you using an autocorrelation to boost the fundamental? It can work well, particularly when there's just one note being played

No, I don't use autocorrelation. Of course, it isn't a pitch detection software.

It's a cool project. I'm just curious, because I haven't bumped into anyone else who's dabbled with the CQT before.

So in what way is it variable Q? I thought the innovation of CQT over a regular FFT is that the bins represent each note step with the same 'Q', making notes more distinguishable? (whereas a FFT has more 'Q' in the high frequencies)

I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.

linear STFT's window length in time domain = k (constant)

CQT's window length in time domain = k/f

showcqt-js window length in time domain = a * b / (a / c + b * f / (1 - c)) + a * b / (b * f / c + a / (1 - c))

whera a = 384, b = 0.33, c = 0.17

I also apply asymmetric window to reduce latency before doing VQT.

> I think having an autocorrelation option on the visualization could be cool, as it could reduce the spikes from the overtones (and also show a missing fundamental). But I think you'd need a different convolution for each instrument.

Research needed. Doing autocorrelation on simple single instrument monophonic audio is probably easy. But doing it on complex multi instrument audio isn't easy. Probably, it needs some sort of machine learning.

> showcqt-js window length in time domain = a * b / (a / c + b * f / (1 - c)) + a * b / (b * f / c + a / (1 - c))

> whera a = 384, b = 0.33, c = 0.17

So this is some sort of compromise to increase speed? Have you thought about implementing in WASM?

> Probably, it needs some sort of machine learning.

Agreed, but for a visualisation, it could just be a parameter just for the user to mess with.

> So this is some sort of compromise to increase speed?

Partially. The main purpose is to increase time domain accuracy on low frequency. If you want to do experiment on window length (in time domain), use ffmpeg showcqt filter (https://ffmpeg.org/ffmpeg-filters.html#showcqt). showcqt-js window length is hardcoded to tc=0.33 attack=0.033 and tlength='st(0,0.17); 384tc / (384 / ld(0) + tcf /(1-ld(0))) + 384tc / (tcf / ld(0) + 384 /(1-ld(0)))' (https://github.com/mfcc64/mpv-scripts/blob/a0cd87eeb974a4602...).

> Have you thought about implementing in WASM?

It is implemented in WASM.

> Agreed, but for a visualisation, it could just be a parameter just for the user to mess with.

If you want to do experiment on it, showcqt.js exposes intermediate color data cqt.color[]. It can be modified arbitrarily, including autocorrelation. If some day I do experiment on it and find that the result is satisfying, maybe I'll include it in YouTube Musical Spectrum.

> The main purpose is to increase time domain accuracy on low frequency

I thought so. The main issue with regular CQT is latency on low frequencies due to the excessive window required. It makes it unsuitable for real- time applications, as you know. Thanks for the insights

Reply on news.ycombinator.com