This kind of app appears to be prohibited in the YouTube terms of service https://developers.google.com/youtube/terms :
II. Prohibitions
Your API Client will not, and You will not encourage or create functionality for Your users or other third parties to:
8. separate, isolate, or modify the audio or video components of any YouTube audiovisual content made available through the YouTube API;
9. promote separately the audio or video components of any YouTube audiovisual content made available through the YouTube API;
So I can't imagine it lasting long.