That seems a problem with RTP more than with wireless connections.
This sort of substitution can work, but it requires very stable, low-noise links (that is, with little to no burst noise) and the cooperation of devices and network equipment.
TCP’s reliability comes at the cost of increased, inconsistent, and unpredictable latency; which is fine for downloading a file, browsing the web, or streaming video with a 30-second buffer; but unacceptable for a real-time call.