The reason facing east-west (or was it north-south, now I'm unsure) made such a difference in framerate was the color and height maps were ray marched in straight lines up from the bottom of the screen to the horizon. This meant you were zipping through the color map in straight lines, wrapping around to the other side if the ray went far enough.
When those straight lines lined up with the color and height map (north-south), life was good (and when a ray marched up a sheer canyon wall, life was VERY good.) But, when those straight lines went perpendicular (east-west) to the color and height map, you were blowing through the L2 cache constantly and going to main memory very often. I imagine on modern hardware these cache misses wouldn't amount to much measurable time, but on a 386dx with 8megs of RAM, the impact was very clear.
Novalogic was the only programming job I ever had where I got my own office with a door. ;) When I was with them, they had a policy of one game developer per game which I never saw again. Maximum cowboy coder energy, good times.