Hacker News Reader

Comment by codegeek | Hacker News Reader

Is there an easy way to block any requests originating from VPS etc instead of residential/commercial IP from legitimate users ? I know cloudflare does a few things but I really want to figure out a way to block any request say at nginx or caddy (reverse proxy) from reaching origin servers if they are not from an IP that is not a VPS etc.

View on HN

I run an open source honeypot that collects these botnet scans and produces blocklists.

Blocklist download and configuration: https://knock-knock.net/blocklist

Honeypot dashboard, where you can see attempted attacks in realtime: http://knock-knock.net

API: http://knock-knock.net/api

This looks cool, where can I find the source?

It's on github with an MIT license: https://github.com/djkurlander/knock-knock. Have fun!

Thank you!!

Thank you for sharing. I will take a look.

Cool site. I was curious and dropped your 100k list into a reverse DNS lookup site [1]. They may still have some of the records cached. I recognized quite a few of the scanner nodes and some other usual suspects.

[1] - https://adver.tools/reverse-dns-lookup/

Yeah, that’s pretty interesting! You can also see a live view of the ASN/ISP leaderboard by going to https://knock-knock.net and choosing ISP from the carousel. That’s ordered by bot transaction count rather than IP count though.

It never ceases to amaze me that these ISPs don’t bother to shut down the botnets. They could do so very easily. For example, they could identify the IP address of every bot that hit this honeypot with their ASN with one API call: https://api.knock-knock.net/check-asn?asn=<asn number>. (See https://knock-knock.net/api). They just don’t care!

The ISP's do not have a financial incentive to shut them down. To them that's a paying customer. The feds will go after the big botnets if they are touching financial networks or siphoning enough money from people because there is usually a few big bank accounts and virtual currency exchange accounts they can seize once big enough to look good in the media. That's why it's on us and a few big CDN's to block some of them.

What if I have a slimy TV box or nasty on my phone, living on my network?

You take IP down, you kill the cancer but you also end up killing the patient.

Not the person you are asking but site operators can not tell intent. It could be something nasty on the network or a botter feigning ignorance.

I'd say its probably an acceptable casualty in the battleground that is the internet especially for little one-off sites hosting blogs, forums, chat servers, etc... For a bigger site I would expect that person may have to open a ticket with the platform such as Amazon accepting that some CDN's and firewalls may be harder to get the block removed. This is why we can't have nice things.

thats the hard part, right - my 76 year old dad is on his banking app while his samsung TV is allowing a bot to try and take over other accounts at the same bank on the same IP.

IP Blacklists, no matter how good can't stop this. You have to start using stats or deep-diving telemetry.

https://darknetdiaries.com/episode/172/

thats the hard part, right - my 76 year old dad is on his banking app while his samsung TV is allowing a bot to try and take over other accounts at the same bank on the same IP.

So appeal to emotion doesn't fly with me. If grandpa is 76 in the year of our lord 2026 that means he was 50 when the internet was getting popular and 59 when cell phones became very popular on the internet. He's not much older than I. He knows what's up.

God help the makers of that television if he finds out it has been spying on him and dorking around with his traffic. If they are lucky he will just take a baseball bat to it. If they are unlucky he will fly to their headquarters and end up on a viral bodycam video likely with a lot of supporters that will bail him out of jail.

IP Blacklists, no matter how good can't stop this. You have to start using stats or deep-diving telemetry.

I use a myriad of methods including IP blacklists. That's my choice and every site operators choice. I do not have to use deep-diving telemetry but you are free to do so.

It doesn't dork around with his traffic, it makes its own traffic. In exchange for a discount.

Nothing wrong with contacting the customer, and taking down the IP briefly until the customer can deal with their malware hosting toaster.

However, I can see the argument for giving the customer 24-48 hours to resolve the problem.

Better yet, it's the only CGNAT address for a little town in Brazil. You just banned an entire town from your site.

There are several methods. [1] The most aggressive method-02 and method-03 on my document will block VPS and some data-centers but that also means it will block some legit users that are on a VPN. Most VPNs transit a data-center. If experimenting with these methods use a test server that you do not care about and set up a dummy site and ask people in your circle of friends to test it. I have to step away for a bit but if you have questions I will try to answer.

[1] - https://nochan.net/b/Internet-Crap/20260606-How-To-Block-Som...

Your site does not currently seem to be reachable / responsive when I try to reach it from a US Comcast IPv4 address - you are not advertising IPv6.

Edit / Update: It was Apple's Private browsing mode that causes it not to respond. I can now see it when this is disabled.

I've noticed they strip away a header [1] in private browsing mode but I don't know why they do it since it does not disclose anything about the person. I think that may be the same thing that causes some people grief on Cloudflare as well.

[1] - https://caniuse.com/?search=sec-fetch

Per your link...

> block http 1.1, real users only use 2.0

Chrome on android and Firefox on linux both appear to use 1.1 still...

By default they use 2.0 [1] unless someone or an addon disables it or unless the person is on a really old version. OperaMini however will use 1.1. No idea if anyone here uses OperaMini.

There are some reader apps that act as a proxy that only support http/1.1. Be careful, some of those are not just readers and do not trust what they claim to be the source code. Some of them are created by cute and fuzzy bunnies.

There are a number of botters on HN, some that control residential and phone browser-hijacked systems. One was sending me playful messages the other day. I enjoyed the bot block-jousting with them.

[1] - https://caniuse.com/http2

Apparently nginx didn't enable it on my new server... why it's not enabled by default, I don't know... I retract my idiocy :P

No idiocy, sometimes defaults change or don't get updated by a deployment script. Different distributions may have a slightly different default configuration file depending on how involved the artifact maintainer is with the project. Or put another way, I've done far sillier things.

> /commercial IP from legitimate users

No, because legitimate users do not just use residential and "commercial" IPs. Like me, right now

You are the 0.001%

Much more than 0.001% of people care about their privacy or (the larger portion) do not have unfiltered access to the internet.

I second this. When I have tested blocking VPS/data-centers to my silly blog there were about a dozen people on HN [1] that could not view my site out of the roughly ~17,000 (not counting bots) that could. It's not a big number but those are real people and they count.

I am going to move full blocking to a test node that people can play with but I have to finish working with Claude to revise someones repo is is no longer maintained because one does not simply put an anonymous chan board on the great wide open internets without some critical thinking.

[1] - https://news.ycombinator.com/item?id=49060945

What did you use for detection?

The complaints in the thread. I am aware the lurkers would not have said anything.

There were a dozen people who reported not being able to access your site.

The complete hysteria people go to over the near-non-issue of bots is, well, completely hysterical. Don't be that guy.

Yes but it’s not cheap. Maxmind and ipinfo etc sell a tier that tells you this information, then you can 403 based on it. But the price is nuts like $40,000 a year.

You don't need to spend anywhere near $40k a year to get that info... You don't even need to spend $1

What's your strategy?

Focsec.com IP database for offline use (datacenters, VPNs, proxies, bots) runs around $1k/month for internal-only use.

I work for IPinfo. We offer IPinfo Lite for free. With a little bit of time in identifying the ASNs, you can implement a decent way to block a good number of bots fairly easily using the free data alone.

Good to know, thank you. Would you do this by fully blocking particular ASNs? Or something more granular?

You can block entire ASNs. If you are frustrated with bots, blocking Tencent's entire IP address space would have very few downsides.

If you have fail2ban or NGINX logs, you can use our CLI to summarize those IPs and identify the ASNs you want to block. But before you block entire ASNs, make sure they are not classified as "ISP" type. For that, visit our website's ASN page first.

I have quite a few community posts around this approach. https://community.ipinfo.io/

If you have raw logs, you can send them to me as well, and I can review them and provide some guidance.

Appreciate it, I'll check out your posts.

Open source works just fine for this - take a look at geoacumen-country

https://github.com/geoacumen/geoacumen-country

Isn’t that only countries though? They were talking about detecting VPS

See my comment in the parent - there are cheaper options, don't use maxmind!

No, because the bad guys use residential proxies if you block DCs. They just prefer not to because it's more expensive. Meanwhile that silly HN project that scrapes and reformats your site is now dead before it began. Who are you actually defending and from what?

Reply on news.ycombinator.com