I would think you can deal with platform-specific issues by saying "we support Ubuntu [or whatever you're willing to do]; anything else you're on your own" up front. But yeah, audio is a general weak spot.
Testing and packaging for, say, the top 5 distros would get the effort-per-user to an even crazier extreme -- we'd literally be at the point that supporting a user on Linux takes 200 times more effort than supporting a user on Windows. You either need a very large number of users, or a very large value per use for that to make sense.
For non-OSS projects, most of the time the smart decision is just not to support Linux. I've worked at other pro-audio companies, and they had Linux versions internally that they didn't release because of the support costs.