Yes your files stay on your server, unless you share them with a friend on another server. It basically just uses libp2p as an internode communication protocol, and you need auth to retrieve ciphertext blocks.
At the moment if your client accesses a server with a mirror, and your primary is offline, then you can read, but not write. Writes are proxied to your primary, and thus need your primary to be accessible.