Try putting audio through an ethernet transformer and see how much comes out the other side... I bet it won't be much, and you'll get extreme distortion unless your signal levels are very low.
On the low end, you have core saturation. On the high end, you have parasitic inductance and capacitance, as well as loss from hysteresis and eddy currents. Solving these problems basically means making the transformer larger and more expensive.
I did a quick search on Mouser, and found wideband (20 Hz - 20 kHz) audio transformers at around $6 in small quantities. Let's say that you get it for $3 in bulk. At that point, you're adding something like $20 or $30 to the retail cost of whatever it is you're selling.
If you're okay with the telephone-quality of the simple modem transformers, then you might as well use the optoisolator--neither option sounds good, but the optoisolator is cheaper. You know, as long as you have a power supply on both sides--which is not a given.