You don't need to have their number, if someone else has both of your numbers and has uploaded their contacts list then you're linked in that third party's list. Equally likely it's linked via the wifi SSID or GPS?

If you both use the same wifi, then it could see you as the same IP address and recommend you to each other.