Most of the good resources where already mentioned, but these are some I missed:
- Musimathics Vol I & II: This will teach you the basics of Music and DSP in a very approachable way. I wish I had these when I started writing audio software.
- The Art of VA Filter Design: https://www.native-instruments.com/fileadmin/ni_media/downlo...
If you are programming in C, these libraries are very useful:
- PortAudio: Cross platform audio playback
- libsndfile: Easy and cross platform sound file io
- ladspa: Really simple plugin API, useful if you want to make or use audio plugins without all the boilerplate of vst or lv2