A compass would probably be of marginal utility if you're flying a "great circle" route across an ocean. For instance, if you were to fly non-stop from Chicago to London, your initial route will have you heading north east, but at the end, you'll be heading south east.
That's why ships would follow a loxodrome instead of a great circle route. They'd still arrive in port, though.
While this is true, an aircraft has very critical fuel requirements - taking a longer and less efficient route combined with a faulty navigation system is a factor best avoided.
I was about to correct you sand say "no, they'd follow a rhumb line", but then I looked it up and realized they're two terms for the same thing. TIL!
All airplanes (or almost all? I'm sure someone will know of an exception) have compasses. But without data on wind direction and speed, a compass isn't much good over any appreciable distance. Once off the ground, an airplane is like a boat crossing a swift river. The boat won't land where it's pointed; it'll move with the current as it crosses and end up landing downstream of where it's pointed. So it is with airplanes. They don't fly where they're pointed; they move with the wind as they fly.
There'd probably be too much metal / electrical interference for a regular compass to work.
Airplane compasses are indeed affected by the electrical system, but not so much as to make them unusable. A maintenance process known colloquially as "swinging the compass" involves turning on the electronics, positioning the aircraft over a compass rose painted on the ground, and noting the error at various bearings.
As for the metal, most metal airframes are aluminum.
Inverting compass readings into position is highly problematic because of interference, and noise.