They're a form of Government-issued photo ID, so not "secret" but definitely "sensitive".
At least in Australia, a passport can be used as your primary ID for a lot of stuff such as renting houses, buying mobile phones, connecting services to your home, booking flights, renting cars, etc etc etc.