You can't see around cars to be able to pass. Okay, so you often don't pass.
But much more importantly, car headlights aren't symmetrical. The one pointing towards the oncoming lane has a "cut" beam to avoid blinding other drivers while the other has a taller beam to improve visibility of anything entering the road. When you drive LHD in a RHD country, or vice versa, you are blinding other drivers while having low visibility of pedestrians and animals.
Technology Connections did a great video about this. His solution was to stop driving his RHD car in low light. https://www.youtube.com/watch?v=c2J91UG6Fn8