If I understand the paper correctly, the architecture is very odd. As far as I can tell, it's equivalent to a deep CNN, with the facts being fed into every couple layers. Why did they break it up into parallel invocations of DNNs, and several layers of DNNs? Very strange.

I imagine a simplified architecture of RNN->Highway CNN->Answerer would achieve better results while being more homogeneous.