That's the point. You hash the number, and keep the hash. Send the first few digits of the hash to the server, it sends you a bunch of full hashes back. If your hash is in that list, then you know the server knows that person.
The hashing is so people can't easily get a list of all the phone numbers, which is easy to work around, but, then again, they could just hammer the endpoint querying for all the various numbers anyway.