reincoder

Thank you. I hope you don't think I am pushing back or being sarcastic in any way because I personally like your work and really appreciate your comment. This is a typical conversation in a very "HN" way. Sometimes, it's hard to determine tone in written communication.

I appreciate that you make a wide variety of data accessible, and that has a positive impact. Our goal is to be the most accurate IP data provider, with geolocation being our foremost priority. It requires a "significant" amount of effort to simply be better than the rest of the industry. If customers want that bump in quality they will pay for our services. The ProbeNet has about 700 servers now, and nobody is investing in that level of infrastructure for IP data accuracy.

The whole game for us is accuracy. Consider our free IP to Country database. It is so accurate that it goes down to individual IP address (`/32`) levels for country-level locations. This high level of accuracy, however, results in a bigger file size. Some developers compromise on accuracy over file size. While rounding up the ranges to `/24` and updating them twice a week would reduce cost, it does not align with our philosophy.

> ipapi.is for sure detects some hosting provider's that you don't

ipinfo 91.102.88.0 -j | jq '.asn' { "asn": "AS47292", "name": "Sentia Denmark A/S", "domain": "sentia.com", "route": "91.102.88.0/21", "type": "hosting" }

ipinfo 185.45.13.144 -j | jq '.asn' { "asn": "AS9009", "name": "M247 Europe SRL", "domain": "m247.com", "route": "185.45.12.0/23", "type": "hosting" }

ipinfo 185.50.248.0 -j | jq .company { "name": "ATOMOHOST LLC", "domain": "atomohost.com", "type": "business" }

For the last one, it is not associated with an ASN, so the company field can be used as a proxy. As that company's IP data is a bit vague labeling it as hosting or business is difficult.

We have four types of "types": business, education, hosting, and ISP. This applies to both ASN and companies/organizations. For us, it is not simply a binary classification of hosting and non-hosting. Instead, we categorize IP ranges based on a statistical model with assigned weights. These ranges are then aggregated to determine AS types. When our customers say why is that this or that, you can show them the underlying reasoning.

But still, I grok and understand you. I really appreciate your feedback.

Hey, I am happy to see you here! What do you think of the event? I think you might find it interesting how these IPs were submitted. I would love to hear your thoughts.

In our previous interaction, I mentioned that I learned about CGNAT-based web scraping from your article on your personal blog. In this event, people took it to a whole other level. People were churning carrier IPs by the hundreds to win a pair of socks.

> our hosting detection is more advanced

When it comes to hosting detection, we haven't published much of our methodology publicly yet. But I appreciate your article on it: https://ipapi.is/blog/detecting-hosting-providers.html

I am not sure if I would say that your hosting detection is more advanced than ours because we have already productized every aspect discussed in your steps.

Step 1: "Download a List of Top 1 Million Domain Names" → 10 million domain names available for free download here: https://host.io/rankings

Step 2: Lookup the IP address for every Domain Name → https://ipinfo.io/products/hosted-domains-database

Step 4, 5: Obtain WHOIS Records for every IP Address → https://ipinfo.io/products/whois-database

Step 6: Crawl the Website and Classify the Website Text → https://host.io

This is just scratching the surface; a lot is happening internally. We have a team of talented engineers and a decade of experience. I am not trying to be disrespectful; I just wanted to share the information, that's all.

Our pricing approach is that we have an uncompromising approach to IP data, and organizations pay for the quality. However, for developers, small businesses, and students, we provide a generous yet highly accurate database and API for geolocation, which is free to access. We offer 50k reqs/month, tokenless API access, and a free IP to Country ASN database.

I work for IPinfo. I often see IP geolocation data being used for compliance purposes, so I will share some insights.

This question relates to algorithms and the definition Facebook has set up for "EU User". What parameters have Facebook's legal team considered adequate to label an account belonging to an EU user? It is difficult to say.

They will probably look into multiple factors like phone number location and GPS-backed data to identify the true location of an EU user. Now, the interesting part is that if you use a VPN quite frequently, you will notice that IP geolocation data is treated as a universal truth on cookieless interactions. Facebook will likely have a significant amount of data beyond IP geolocation data to infer the location information of their user. But they are likely not going to do that as they launch this new feature. So, it all comes down to how seriously Facebook wants or is required to identify an EU user.

I work for IPinfo and I am glad to see privacy detection working. Internally, one thing we sometimes discuss is whether we should use "scores" when it comes to IPs.

Personally, I think there is a market for indicating whether an IP is good or bad and giving it a reputation score. However, as a company, we prefer not to do that and there is a good reason for it. There are two core caveats:

1. Ambiguity: Scores can often be ambiguous when determining whether an IP is an "anonymous IP". Having a boolean response for VPN, proxy, or Tor is a simpler solution and we prefer simplicity.

2. Accuracy: We prioritize accuracy and strive for complete coverage. Some "ip reputation" providers essentially just repackage threat feeds. We want to avoid getting involved for now.

Our policy is to be reliable and allow our customers to sell IP reputation and cybersecurity solutions using our data. If they wish to create reputation scores using our data, they are more than welcome to do so.

What a coincidence! I am the DevRel of IPinfo, and I was just about to reach out to you. My teammate (Shoutout to Max) has already mentioned your project on our team Slack, and I was thinking about a polite way to pitch our free IPinfo Country ASN dataset to your project. I will open a Github Issue right now.

At least for us at IPinfo, this is not the case:

- Tokenless IP geolocation access: Comes with full accuracy and daily updates. Supports HTTPS. 1k requests/day.

- Free tier IP geolocation access: Still comes with full accuracy, daily updates, and supports HTTPS. No credit card required. 50k requests/month.

- Free IP to Country ASN database: AGAIN, full accuracy and daily updates. Because it is a database, it means unlimited access. You can download the database and do whatever you want with it. Unlimited requests per second.

Anyone who wants truly unlimited access to IP geolocation should set up an IP database on their backend. There is a special and easy-to-use database format called .mmdb specially designed for that.

I work for IPinfo and let me go off on a tangent here. We use our own infrastructure system to probe IP addresses and determine geolocation. This system is based on technology similar to GPS satellites (ping and check RTT from multiple servers).

Other providers mostly rely on WHOIS data and other public databases. These sources are not very accurate and often contain stale data because the IP range geolocation reporting mechanism for ISPs is mainly voluntary.

Sometimes, even though an IP address is pingable and we have evidence of the round trip time, the majority of providers will indicate one location while we indicate another. So, who is correct in this situation? Is it the majority or us?

If someone needs to verify the location of an IP, I usually recommend visiting that site first. If there are any discrepancies, I recommend pinging the IP address and checking the WHOIS records.

I work for an IP geolocation company and precision isn't the main goal of IP geolocation. Accuracy is the main goal. IP geolocation needs to be precise enough on a city or zip code level at best. If precision is the goal, IP geolocation is not a drop-in replacement for GPS geolocation.

Really appreciate it! Thank you very much. I will write a PR ASAP.

The CLI has a bunch of features. I highly recommend checking out the `help` output for the CLI. Personally, I use the `grepip`, `bulk`, and `summarize` commands on a daily basis.

Thank you very much. I really appreciate you opening a ticket. We will look into it and keep you updated.

The pleasure is all ours. Your tool is really awesome. I have bookmarked it. It addresses a very practical questions users have about distance calculation. Now I have the right tool to recommend to users.

Would it be possible to allow access token usage in the tool, please?

That is really surprising. For geolocation correction, we have a self reporting system: ipinfo.io/corrections

I forgot to add the link in my original comment. But as you mentioned we are showing VPN as true, that indicates a very interesting situation. Can you ping our support, please?

https://ipinfo.io/support

If you can also mention that, you came from HN, that will be fantastic. We will open an engineering ticket first thing Monday and look closely into this. The situation is unusual, and I am sorry about the inconvenience.

Can you check IPinfo.io, please? If we are showing any inaccuracy in your IP geolocation data for you, please reach out to us. We will fix that ASAP.

Disclaimer I work for IPinfo.

If you can download the free IPinfo IP to ASN database CSV file. You can run the following command to get the IP ranges of an ASN.

```

grep AS19551 asn.csv | ipinfo range2cidr | cut -f1 -d ','

```

You will need to install the IPinfo CLI to convert the IP range to their CIDR equivalent. I believe the range2cidr command is also available as a standalone binary as well.

After that, you might want to aggregate the IP ranges to bigger ranges. I have used this in the past for this:

https://github.com/job/aggregate6

I work for IPinfo. We have higher level IP category information on our website for free.

We categorize ASN and companies/organizations based on 4 categories: ISP, Education, Hosting and Business. This ASN level categorization are done mainly from WHOIS and other public internet records.

We don't sub-categorize by schools, university, public research institutions, k-12 etc. The reason is accuracy. Even though I can understand the possible methods for doing this, the issue is that it can not be done reliably at scale.

As a data provider, from our end we hope to provide the highest possible accuracy and vouch for the service we provide. For this level of classification, we will generally request users to say what data they need from us, and we try to help them come to a solution that they have to build on their own. They can do whatever classification they want to do based on their personal level of tolerance for accuracy.

I work for IPinfo.

>the ASN is a string of asn and name, no org or country

Yeah detailed ASN information on the API level is available on paid tiers.

However, we have a free IP to ASN Country database that is updated daily and provides full accuracy. That database provides ASN, AS organization, and AS domain information.

I think the reason we don't have more ASN details on the API even though we provide more information for free in the database is because of the target audience and usability of the IP metadata.

Our API allows for tokenless API access for up to 1,000 request/day. There is no compromise with data accuracy. The goal is to be generally useful and extremely fast. Detailed ASN information is still freely accessible from the website, though. So, we have to balance out free API, free website information and paid API/database.

I have been reaching out to folks to adopt the free IP to Country ASN database, as it is free to use and provides full accuracy. If anyone wants to build a public API on top of the IPinfo free database that return more ASN information available there, they are more than welcome to that!

Hi, I am the DevRel of IPinfo. Every time I mention this blog anywhere, I will preface it by saying, it is bit on light on technical details. Hence the usage of various examples like sonar, bat and stuff like that.

We are going to write a followup to this blog which is going to be more technically heavy. The core purpose of this blog was an intro to a series of blog's on IPinfo's IP geolocation methodology.

You can throw us a checklist if you want for the followup article. We are trying to make internet data as a concept as approachable we can, any feedback or any topic you want us to cover will be greatly appreciated.

I work for IPinfo.io. For geolocating satellite IP addresses, we use a combination of geofeed, rDNS hints etc. Starlink's geofeed data is easily accessible [0] which lists out geolocation information of their IP ranges. We use rDNS hints as well where you can find some geolocation hints from rDNS records.

[0] https://geoip.starlinkisp.net/feed.csv

Looking at the IP metadata of the IP address [0], nothing stands out. The ASN belongs to large teleco. What you can do is just block the IP address and keep a note of IP address.

[0] https://ipinfo.io/2401:4900:1f38:7402:5569:2e45:3bb:9c0d

I work for an IP geolocations service, even I hate this thing.

They call it "web experience personalization" in the industry, and it is annoying. I have never recommended anyone to do that. The best way to do website personalization through IP geolocation:

- Taxes and stuff (if applicable)

- Delivery costs (if applicable)

- Putting the user's country first in those country selection drop-down menu

And that's about it from the top of my head. In my experience, these translations never work and only create distractions. Regardless of the positive intention the website has, using Google Translate to create a native language version of the website is just not a good idea.

Fantastic stuff. I work for IPinfo.io and I actually came across your site about a month ago. I was planning to contact to you about mentioning the free IP databases [0].

However, when I saw that a few API didn't return any response, I thought maybe the site was not maintained.

[0] https://ipinfo.io/products/free-ip-database

----

Tangent

I find the geographic coordinate values returning up to 15 decimal places is absurd for an IP geolocation response. IP geolocation is never that precise and, this level of "precision" is not warranted and frankly distracting. Like at best it should be 4 decimal places.

relevant xkcd: https://xkcd.com/2170/

Our VPN recognition is behavior-based. So, there is probably a chance that the IP address you are using is showing some of those behavior patterns.

A behavior pattern could be that your IP address is being shuffled around random locations that go beyond the normal location shuffling of an ISP connection.

Also, if your IP range is listed in some public datasets that belong to a VPN service, we could recognize your IP as a VPN.

Please reach out to our support and let us know about this. Thanks

If you can give me some information about the IP address or the IP range, I can take a closer look.

I guess it is because of IPv4 trading or IP address shuffling.

As far as I know, Hertzer, like many hosting companies, is buying IPv4 addresses around the world. Here is an article on the IPv4 trades:

https://tech.marksblogg.com/ipinfo-free-ip-address-location-...

When a company buys an IP address block or relocates an IP block from one of its data centers to another, the location of those IP addresses changes.

If your IP address is static, but we have made an error in geolocation, I would love to take a closer look. You can email our support (support@ipinfo.io) and send a link to the comment. We can discuss it further from there.

I am happy to assist any business, organization or person to use our data. I enjoy talking with people and helping them with the technical and philosophical aspects of IP metadata.

If you progressively remove the layers of metadata associated with internet-connected devices one by one from GPS data (user-permitted action) → IP geolocation (IPinfo.io)→ IP address (ISP), and come to a system that is completely anonymous, you have the dark web.

The dark web represents an ecosystem that is completely devoid of ad tech because there is no identity, and the users are purely homogenous. The system itself is not widely adopted because internet participants need to feel secure from cyberattacks and harmful words. Traditional internet services do not exist there because they are getting bombarded with anonymous attacks, and they cannot provide any value to the customer because they don't even know who/what the user is.

IP metadata helps with running a business and serves as a way to protect yourself. There are organizations and users who actively utilize our services to check email headers, find the IP address, and lookup the IP metadata of those IP addresses. The data is utilized for recognizing spam and phishing emails. Conversely, our service is also used in adtech-driven cold emails and newsletter services.

IP metadata is required for a functioning and widely adopted internet.

I appreciate it, sir! If you have any questions or feedback, please let us know.

The challenge of being a data provider is that you can use our data in a million ways, and we don't have coverage of all. So, when you come up with questions or ideas, we can help you better.

As you mentioned, audit logs. I highly recommend you look into the ASN field.

The ASN identifies an organization that owns a block of IP addresses. In my experience, I have found that the combination of ASN+Country is the most valuable information you can use in spam and fraud detection. You can fake the IP geolocation information with a VPN. However, it is not as easy to fake the ASN information of the IP address. So, when you use a combination of country + ASN, you can have a robust cybersecurity system.

Thank you for the feature request.

We usually just send users the documentation of ingesting the data in CSV or NDJSON format (Newline Delimited JSON). We don't actually get many requests for data downloads in Parquet format. I think we have a few customers where we deliver the data in parquet format directly to their cloud storage bucket.

But keep an eye out for our emails if we announce the parquet data downloads. I will talk with the folks about this.

BUT, there are some good news.

At least for the free database, we deliver the data directly to data warehouse platforms. Not even storage buckets. And we supply a good amount of documentation.

We have the free database in Snowflake, GCP, Kaggle, and Splitgraph, and we are working on a few more deals. For the free database, atleast, we are working on better things than parquet. Like literally one-click solution to bring the IP data to your data warehouse.

Kaggle: https://www.kaggle.com/code/ipinfo/ipinfo-ip-to-country-asn-...

Snowflake: https://app.snowflake.com/marketplace/listing/GZSTZSHKQ4QY/i...

If you want to use our free IP database on Google Cloud or BigQuery, please send us an email (support@ipinfo.io) and mention that the DevRel sent you from HN. I can easily set you up with the free IP database in GCP/BQ.

That is part of the reason why I am here in HN. I am pretty down to earth and willing to talk with anyone about IP metadata.

Our business approach is that we will come to you before you come to us. Whenever people complain about something related to IP data, I will just reach out to them and verify that we provide good data. Our support team is extremely responsive. We also have a community where we walk through users about IP metadata. I am super active on HN, Reddit and Twitter.

The challenge for me is usually that we will tell people, "We are providing good geolocation data for you, but the service that provides geolocation data to the service you are complaining about is not using our data. Our hands are tied." I have reached out to bigger corporations and streaming services to say, "You are providing bad data to your customers. I am not even trying to sell you something. Even using our free database could lead you to better results." But I have not heard from them yet. It is frustrating to me as it is for the person with bad IP data.

IP geolocation is never going to be absolutely accurate or precise. We understand that completely. Aside from continuously investing in improving our data from a product standpoint, we also take a very human approach of trying to fix every geolocation complaint one by one.

We don't have any free data for that. We have historical data that we sell as part of our custom enterprise deals. Historical data requests are rare, though.

A time series IP database requires a substantial amount of storage and computational cost to query, as I imagine. The city level geolocation data we have is ~1.5 gb in size. IP range data is complicated to query efficiently as you need to understand data platform settings and good amount of computer network math and computer science stuff. Adding a layer of time series complexities on top of that, makes this process quite difficult.

To give you some context of how IP metadata lookups work, you can check out this article

https://ipinfo.io/blog/ip-address-data-in-snowflake/

Even if you keep all your database in a binary format, the computational cost is still non-negligible.

That is a good point. I will try to add some additional points to your statement.

In the context of a butcher and a vegan we are not either of them, we would be someone who sell salt. Salt as an ingredient is used by vegans, vegetarians, and meat eaters.

Salt is useful, and it is a complimentary item that makes the food taste better. But it is not a main feature of any dish. Salt plays an important and somewhat universally required item in any dish, but this is not the main feature of the dish you eat.

We are a data company. This IP metadata we sell is important to run a service that is connected to the internet. We are focused on serving cybersecurity as we are focused on other industries which includes adtech and sales intelligence.

IP metadata is used in threat intelligence, building firewalls, and attack surface management. On the other hand, when you are running a business at scale, you need to invest into understanding your target demographics. That is where largely adtech companies use our data. They need data and context to understand who their users are.

IP metadata helps to prevent cyberattacks to happen at a very early stage through firewall blocking. IP metadata, used through threat intelligence, can further strengthen firewalls to prevent future attacks. And IP metadata, can also be used in website personalization, adtech and sales intelligence etc.

We believe that every internet connected services for profit and non-profit alike, should have access to some security resilience, that is why we offer so many good quality free data.