Pretty sure that's because Electrum doesn't download the entire blockchain but relies on servers for most of it and only downloads a certain amount of recent history and trusts servers for older history. And it doesn't have to be that server, which is why app lets you change the URL, any server serving blockchain history you trust is fine. It's using your passphrase as the one and only seed for key generation, so it can reliably generate same private key every time. Someone correct me if I'm mistaken on any details.

Electrum is a thin client that has to ask a full node for the chain so that it can pluck out the relevant transactions to rebuild your local wallet state.

https://en.bitcoin.it/wiki/Thin_Client_Security

It knows which transactions are relevant because of the keys that can be deterministically generated from your seed, but the seed itself isn't sent to a server.