youka

good luck! here's the relevant code https://github.com/youkaclub/youka-api/blob/master/youka/ali...

My little secret hack for better tts results is to make the singing sounds like speaking. Currently I’m using Sox pitch filter, do you have another idea how to achieve that?

I don’t install it system wide, just download a single binary into youka directory.

I’ve tested every single ASR alignment solution that mentioned here https://github.com/pettarin/forced-alignment-tools, but they all performed poorly compared to Aeneas, even with good language models (English)

I preprocess the vocals using Sox, so the female singing become more like male speaking

Thanks for your reply, I’ll add copyright notice soon. I didn’t really tried to keep it private, otherwise I could just ignore the question. The reason that I didn’t cite it in the first place, it’s because I’m still testing few alternatives.

Based on your experience, which alignment method/system is the state of the art? (I’m looking for accurate word/syllable level alignment for Youka)

The question is, how can you do it automatically..

Just reopen it and you will be fine (I don’t have free 99$/year for apple code signature)

You are close enough, so I have to respect my word. I’m not a genius, just a lego builder, I’ve tried a lot of methods, from DL to ML but aeneas project (with some optimizations) gave me the best results. Amazing project and even better personality. Take a look at https://github.com/readbeyond/aeneas Together with espeak-ng, you can get good results for line level alignment for 108 languages.

not even close

So let’s play, if you can guess the align method I’ll open source it :)

I’ve considered few options to install ffmpeg, and choose that way. I’m open to other suggestions

thanks but the video is unavailable

Youka aligns lyrics automatically, you have left nothing to do

Looks like a server-side bug (can't really handle more that a single split process concurrently), I'll add queue in the next version.

exactly

I'll add some explanation soon. Here's the main process:

Search your query in YouTube using https://github.com/youkaclub/youka-youtube

Search lyrics using https://github.com/youkaclub/youka-lyrics

Split the vocals from instruments using https://github.com/deezer/spleeter

Align text to voice (the hardest part) using some private api

You right! I'll add illustration gif soon.

hi Reubend, happy to hear that you like it! the overlap looks like a bug in chrome for mac, it never happened to me in safari or mobile browser..

hi godot, thanks for your detailed feedback! i really think about product&fun before money. shorter processing time requires costly GPU.. as i see it, it's a really useful tool for the long tail, because for the popular songs you can use other great services thats produce it professionally (like sing king youtube channel), but the total count is only 30,000 songs from total of 60 millions songs!

thanks mrieck! currently is just for fun, if i got sued i will take it down and open source it. SuperAnimo looks very nice (idea and implementation) but yeah, product-market fit is hard to achieve.

hi ghego1, thanks for your feedback. lyrics to audio alignment is unsolved problem yet, so i use the best tools in existence, which give very good result for 50% of the songs in almost any language. you can learn more about it here https://www.music-ir.org/mirex/wiki/2019:Automatic_Lyrics-to...

hi jan6, youka here. i'm still in alpha stage, so bugs may happen. i'm search lyrics mainly in public sources. tell me which song you want, i'll fix it for you.