I'd say sensitive at the very least. Like social security numbers they shouldn't be, but when places use them for identification without checking authenticity and authorization...
They're a form of Government-issued photo ID, so not "secret" but definitely "sensitive".
At least in Australia, a passport can be used as your primary ID for a lot of stuff such as renting houses, buying mobile phones, connecting services to your home, booking flights, renting cars, etc etc etc.
Yes. The bottom of the post covers some of the things you can do with the number.
Yet good luck traveling without actually surrendering them to all kinds of places you'd rather not. Like hotel clerks basically everywhere.