Unless you need to convert the audio format, I don't think you need ffmpeg at all - youtube-dl can just download the audio stream and save it to a file. Just list the available formats from a video and choose an audio-only format.