I have a BeagleBone Black but it still hasn't made it out of its box. I'm hoping to use it to write simple synths/audio effects and connect lots of buttons/pots/switches/sensors and such to the GPIO pins.
In the meantime I've been using the Teensy to build I/O devices and then send MIDI/OSC to and route audio through my laptop.
The 3.8 Kernel was the only one that supported Device Tree Overlays. I found that to be easiest to work with, although there are a few bugs. Last time I tried, removing overlays did not work reliably - a reboot was usually better.
I am currently on Ubuntu 14.04, which unfortunately does not support device tree overlays. I installed it a few weeks ago and kept a few notes on the process here: