How difficult would it be to wrap your code in VSTi plug-in and produce a DLL? This would make it easy to hook up to midi controller and other devices using any realtime VST-capable host.
I would assume trivial. I'm a linux person, so I've just wrapped it in jack stuff - but this is what these plugin APIs are designed for.
I prefer LV2 plugins as they integrate nicely into the rest of system. As I dabble in sound/music only rarely, Jack's all-or-nothing approach doesn't suit me at all. Also, it requires 5+ windows just to set everything up.