Hmm… Wonder whether it would improve things to create a virtual environment with point sources actually that distance apart, and model the reverb / echoing you'd get from that, so your brain "intuitively" understands the delay.