It's not quite measurable; the magnetic potential is gauge invariant, which means, among a great deal many other things, that it has no well-defined measurable value. However, it is certainly physical; things like the Aharonov–Bohm effect prove that.