Windsribe and iVPN.
388 karma · joined October 27, 2022
Things I am responsible for at IPinfo:
- https://is.gd/devrel - https://ipinfo.io/community - https://ipinfo.io/probe-network - https://ipinfo.io/developers - https://ipinfo.io/lite
I am available on LinkedIn more than on my inbox.
https://community.ipinfo.io/t/can-we-detect-ai-agents-we-can...
Most AI crawlers self-identify with a UA. However, Grok uses resproxies and sends a high volume of simultaneous requests. Even though we can detect resproxies, it is not possible to map these resproxy IPs to grok.
I still could not figure out why I saw legitimate Googlebot IPs when I requested Perplexity to review the website. I verified those Googlebot IPs using both using UA and the listed IP address ranges published by Google.
---
Edit to my parent company
---
We need SSH through a public IP address and use Ubuntu as an operating system. Since we need to determine where these servers are located, we need to collaborate with universities, various organizations, IXP, DCs, telecom/ISP consulting companies with a mature understanding of network engineering.
Managing 70k probes is not going to be super hard.
Managing 1,400 servers is just a normal business operation, not a technical challenge. Each probe has a standard OS-level configuration. Automation and configuration are deployed from a central system. Each probe is actively monitored and troubleshot. Data is dumped to a data warehouse. We make incremental improvements to our network. When servers go down, we talk to vendors.
We do a lot of novel engineering things from the infrastructure, data, and research team. Having a very identical set of servers really allows us to focus on product and performance engineering, not troubleshooting engineering. With application-based probing, I assume it will complicate things quite a bit, as there are different operating systems, different devices, etc.
For us, lately the challenge is not technical. It has been exclusively procurement. This quarter (https://ipinfo.io/blog/probenet-q1-2026-expansion), we exclusively focused on regional diversity which involved outreach to national ISPs or telecoms. Securing servers from telecoms is an extremely bureaucratic and expensive process. So, we are hoping to partner up with eyeball networks and the larger NOG community.
To support lots of ISPs, universities, and different organizations we have been asking them if they have an old laptop lying around that they can host our software on. Goal is to reach 70,000 probes within the next couple of years.
It is a simple probe software and we share some data or we can pay 20-30 bucks a month for it. We have a couple of NUCs in remote regions but no laptops yet. Basically, we are even happy if an ISP (or any one) hosts our software from a laptop dangling by a charging cable from a socket in some random corner.
We can send over a RPI or NUC, but with remote hands, and setup and all that it can get quite expensive. So, we always first ask if they have an old laptop lying around and can install our software there.
For us, at least, we are not interested in the hardware aspect. We are interested in the network. The old laptop approach only acts as a last resort. We will be more than happy to go with the predictability of a traditional VPS hosted in a traditional data center. Colocation, no matter what form it takes, involves a lot of moving parts.
Email me: abdullah@ipinfo.io
I think there is an issue with setting up our API.
We made that report independently and, according to our analysis, we only identified three VPNs: Windscribe, Mullvad, and iVPN to not have virtual VPN server locations.
> Just to clarify: You are suggesting that we don't pro-actively enrich every IP address, store IPs, and only enrich them when troubleshooting something?
I think you should experiment with this yourself a little. The Lite API is completely free. So you can do ingestion enrichment and post-enrichment enrichment. See what works best for you.
That is a very good question. Now, geofeed does not have a verification system. Active measurement is something we use to verify ASN or ISP itself.
Even active measurement has its own limitations. Now in those case where we see active measurements not producing reliable data, we do reach out to ISPs and ASNs to purchase a server in their facility. Geofeed as a system is voluntary and most major ISPs actually do not maintain or even publish that. For example, today I found out a major UK-based telecom geolocated 500k IP addresses in a town with 200k people. ISPs are not inherently incentivized to maintain the accuracy of their self-reported, voluntarily published location data. So, we do proactive outreach to purchase a server from them so we can provide consistent accurate data for their IP addresses.
On the matter of advertised locations not matching actual location, I highly recommend reading this: https://ipinfo.io/blog/vpn-location-mismatch-report
For residential ISPs, we do a lot of outreach and open communication to build a good partnership with them. The goal is that we pay for the privilege to report accurate data for them.
> How often does your active measurement data disagree with geofeed data?
Very frequently.
Here is the summary peer reviewed research paper on this matter: https://community.ipinfo.io/t/ip-geolocation-and-geofeeds-wh...
Active Measurement (1,330 probes, 27.7M RTTs):
- Country-level: 92.0% accurate → 8% wrong country
- City-level: 79.6% accurate → 20.4% wrong city
- Country-level: 84.5% accurate
- City-level: 29.9% accurate → 70% wrong city
Primarily through active measurement, we are also running a lot of research around more reliable mobile geolocation data.
Because our data is updated daily, I think due to the refresh rate we have an accuracy advantage.
> If I am troubleshooting a support case that is days/weeks/months old, wouldn't this mean that enriching this information at a later date may give me different data than what it was associated with at the time the requests were made? My understanding was that IPs get re-assigned.
You will be surprised to know that historical IP location does not have much demand.
If you are evaluating a support case after some time, you should work with your current data. If the customer raises a question, you address this in real time with their current IP address.
Usually, I do not recommend storing historic IP geolocation information. In most operations, the enrichment happens in real time within the day. Unless you want to do periodic reporting of some sort.
Internally, we of course have the data, but because our IP geolocation is so accurate, it currently sits at around 700 MB. If you add a historical layer to that data, it will be a terabyte of data. There is not much consumer need for it.
> How frequently do IP-to-location mappings change in practice?
https://ipinfo.io/blog/how-many-ips-change-geolocation-over-...
On the city level is 1.3% each day and 16% each month.
> Do you offer historical IP data snapshots?
I highly recommend that you work with current day's data.
In cases where we provide historical data, it is usually for academic research.
---
Let me know if you have any more questions.
However, we do not offer city level data for free. > How would one even go about verifying it?
We believe we are the most accurate IP data provider out there, but you should come to that conclusion yourself.
I can tell you why our data is super accurate compared to the rest of the industry. The industry as a whole uses self-reported information that is offered by ASN and ISPs. It is called "geofeed". The issue with geofeed is that IP geolocation providers do not tend to verify the accuracy. Many providers just aggregate these public records and repeat what the ISPs and ASNs want them to tell them. This is a quite bad practice.
So we built a network of distributed servers (currently 1360 servers across 160 countries) that run ping, traceroute and other internet measurements and try to infer the location of IP geolocations. This means when you come to asking how do I know you are accurate, we can share our active measurement data and tell you that this is the evidence.
Now, comes the qustions of how you identify accuracy yourself.
First, if you have access to a large pool of known locations of IP addresses, you can run comparisons across different vendors. You need a GPS-backed device to locate IP addresses.
If you do not have a large pool of well-known location IPs, you can take a sample of IP addresses and check them yourself across multiple vendors. You can then use a tool like ping.sx or our own tool ipinfo.io/probenet/live to see evidence of where these IP addresses are located based on latency.
Do not bet on consensuses among IP geolocation providers; run your own tests.
Our data was evaluated by peer-reviewed academic research. You can take a look at that as well, if you want.
> I am not really using this data for anything other than have enough data to troubleshoot customer support/fraud.
Now, I will be honest...you should not pay anything to us. The way you have describing your issue, it seems like the free services we already offer that should satisfy your need.
Do you really need large scale IP address enrichment of all the IP addresses that visit your website? If yes, then for the first layer use our free data that provides ASN and country information.
Then, when you need troubleshooting with your customers, you can look up those individual IP addresses for free on our website, where we provide all our data for free access.
---
Let me know if you need any help, always happy to answer questions.
Our country-level data (which is free) is 10-15 times larger than the free/paid country-level data out there. We constantly hear that the size of the database is an issue. The size is a consequence of accuracy in the first place. So, it is a balancing act.
I am curious about your experience with us so far.
From what I know, whenever a router is backdoored or a resproxy SDK gains access to a device to use their bandwidth, the access to that pool of devices is often shared among multiple resproxy vendors. Many resproxy vendors do not have their own SDKs for their services.
Also, as far as I know, not many resproxy operators manage their sim farms or hardware pools. It is mostly based on compromised devices or SDK access.
To help the system, we are reaching out to IXPs, major telecoms and peering agencies to advise them on how to peer and make critical internet routing decisions. We want to tell them on how to engage in data-focused peering, how their IXP is perceived from a broader internet data perspective, and how their packets from the IXP travel across the internet. We hope this colloboration will bring much needed efficiency in internet routing.
---
Our research scientist, Calvin, will be giving a talk at NANOG96 on Monday that delves into active measurement-based IP geolocation.
Latency variability is a huge issue. We run both traceroute and ping data, and we observe that there are entire countries that peer with IXP thousands of miles away in a different continent.
We bought a server from the oldest telecom company in the country and recently activated it. Currently, there is a 20 ms latency when traffic is directed towards the second oldest telecom. The packets have to travel outside the country before coming back in. This is a common phenomenon that occurs frequently. So, we usually have multiple servers in major cities since various ASNs have different peering policies.
For us we can map those behaviors and have algorithms and other data sources, make measurement-based geolocation perform well.
We are hoping to support IXPs, internet governance agencies, and major telcoms in identifying these issues and resolving them.
https://community.ipinfo.io/t/wrong-geolocation-based-on-ip-...
Our free database is licensed under "CC-BA-SA" (freely distributable but requires attribution) because of accountability. If you use our data as an enterprise or a free open-source project, if there is any issue, you can come to us and talk with us.
It is not even end-users. We maintain open communication policies in general. Even if a streaming service does not use our data, if they come to us, we try our best to help them based on our industry knowledge.
We have tons of historical traceroute data patterns, and generic traceroute behaviors are likely modeled out internally. So, if you can spoof the traceroute to your IP address, our traceroute-based location hint scoring weight for that IP address will decrease, and we will rely on the other location hints.
You have to be extremely deliberate to misguide us. But I would love to see this in action, though.
The problem is that everyone knows we are the most accurate data provider and our growth is exponential. To my knowledge, most cybersecurity teams use our data to some degree. We cannot risk having any secrets out there that could disrupt the accuracy of the system. We are aware of several cases where accuracy may be affected, with the most notable being adversarial geofeed submissions.
If the issue is an adversarial geofeed submission, it is a well-known problem. When active measurement fails, we have to fallback to some location hint. There are layers of location hints we have to fall through to ultimately landing on echoing geofeed location hint.
But aside from that... I'm not sure what could possibly impact us. A substantial systemic malicious change in data accuracy seems highly unlikely and quite impossible.
Internally, we have an anycast database. I believe we can also provide all the location hints we see for each anycast IP. It is generally niche data though.
We with multi-billion-dollar corporations, and for every product integration we maintain an active, visible presence in their user communities.
For example: https://community.cloudflare.com/search?q=ipinfo%20order%3Al...
Customer support teams are encouraged to build support pipelines that either route data-related questions directly to us or send users directly. We remove friction rather than hiding behind layers of enterprise support.
We make a deliberate "account manager for everyone" effort when introducing ourselves to a partner's user community. We engage with influential community members and MVP users and encourage them to contact us directly when issues arise. We also connect with the engineers who work hands-on with our data and make it clear that they have a direct line to our engineering team.
We actively and aggressively monitor social media for reports of issues related to our data within partner platforms and engage with users directly when something comes up.
To be honest, this is not difficult. Once or twice a month, we may need to present evidence to a user to explain our data decision.
This is not a paid add-on or a special clause in an enterprise contract. Our customers do not pay extra for this level of engagement.
Developers hold us in high regard. Maintaining that trust requires ongoing investment of time and resources. We fundamentally believe developers trust us because of the quality of the product and the lengths we go to provide clear, honest explanations when questions arise.
That is why we have the IP to country level data available for free. As you have recognized the fact that country level data is good for security, we are willing to take a massive hit on potential revenue to allow everyone to use our country level data for free, even for commercial purposes. We literally built separate dedicated infrastructure that provides unlimited queries for our IP to Country data. We want to ensure that everyone has access to reliable data.
For us, based on active measurements, what we do is distribute IP addresses to more densely populated areas. The issue is that we are good at zip code level accuracy, but it is impossible for us to get street addresses correct for residential internet connections. Even if we get geographic coordinates fairly close to you, it is largely coincidental. Our accuracy radius goes as low as 5 KM.
However, consider hotels, conference centers, airports, train stations, etc., where large numbers of people gather and where there are a few public WiFi hotspots that usually remain in the same location. We can identify the exact building from those WiFi hotspot IP addresses.
We have approximately 1,200 servers in operation. Simply by knowing which data centers house our servers, we can reliably identify neighboring hosting IP addresses to the exact data center.
Everyone in our engineering and leadership is very close with various CDN companies. We do echo this idea to them. It is not IP geolocation; we actually have a ton of routing data they can use.
We have seen this in practice. For example, when we deployed servers in Gambia, even traffic between local networks often left the country and came back due to limited peering and little use of the national IXP. Stil, the overall routing patterns were still learnable once you look at enough paths.
For VPNs, we are measuring the location of the endpoint IP itself, not user traffic inside a tunnel. If routing only changes after a tunnel is established, that is a service level behavior, not the network location of the IP.
Anycast and tunneling are things we explicitly detect. They tend to create clear patterns like latency clustering or unstable paths, and when we see those and flag them as anycast IPs by defaulting to their geofeed location.
See the classic: https://ipinfo.io/1.1.1.1
But I am not sure if we use them extensively. I think, as we own and operate the ProbeNet, much of the data collection efforts can be done through that in a scalable manner.
I talked to someone who bought a /24 from South America to be used in the United States for office use. I asked him to tell everyone to get on WiFi and keep Google Maps running. Apparently, that solved the issue.
Interesting. I would love to know how this is possible. Like with Geofeed or something else?
I have seen a Europe-based cloud hosting provider's IP ranges located in countries where Google does not provide service. This is because these IP ranges are used as exit nodes by VPN users in that country.
Device-based IP geolocation is strange. We prefer IP geolocation based on the last node's IP geolocation. We hope to collaborate with Google, Azure, and other big tech on this if they reach out to us.
We are trying to work with ISPs everywhere, so if port level geolocation of the IP address is common, we surely need to account for that. I will flag this to the data team. To get the ball rolling, I would love to talk to an ISP operator who operates like this. If you know someone please kindly introduce me to them.