HNHacker News
TopNewBestAskShowJobs

santah

498 karma · joined June 10, 2019

submissionscomments
santah··on Ask HN: Someone is proxy-mirroring my website, can I do anything?
Same thing happened to me and my service (https://next-episode.net) almost 2 years ago.

I wrote a HN post about it as well: https://news.ycombinator.com/item?id=26105890, but to spare you all the irrelevant details and digging in the comments for updates - here is what worked for me - you can block all their IPs, even though they may have A LOT and can change them on each call:

1) I prepared a fake URL that no legitimate user will ever visit (like website_proxying_mine.com/search?search=proxy_mirroring_hacker_tag)

2) I loaded that URL like 30 thousand times

3) from my logs, I extracted all IPs that searched for "proxy_mirroring_hacker_tag" (which, from memory, was something like 4 or 5k unique IPs)

4) I blocked all of them

After doing the above, the offending domains were showing errors for 2-3 days and then they switched to something else and left me alone.

I still go back and check them every few months or so ...

P.S. My advice is to remove their URL from your post here. This will not help with search engines picking up their domain and ranking it with your content ...

santah··on I'll redesign any 3 websites in less than 3 hours: (Challenge?) HN
https://next-episode.net

I always thought it could look a lot better, modern.

santah··on Ask HN: How do you log TV shows?
I'd recommend next-episode.net as I use that for more than 15 years now, but then again - it's my service :)

It also has iOS and Android apps ...

Check it out and let me know !

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Update 3: After about a day - they expanded their pool of IPs to more than 2k new ones.

I blocked what I could and then I blocked the whole country (Russia) behind a CloudFlare javasript challenge (so that any legit traffic that wanted to pass it - could).

Everything stayed blocked for about a day and now it seems they gave up:

- all domains that pulled dynamic content from my site now show some other content and do not try to scrape Next Episode

- all static domains that delivered the cached version of the Next Episode homepage now deliver some other website(s)

- CloudFlare shows no further traffic on the firewall rules I've set (so the bots from those IPs are, for now, gone)

As this post is relatively old now (more than 3 days) I doubt many people will go back and check for new developments, but I wanted to give this the proper closure and let you know of the apparent happy end.

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Thanks for the suggestions.

I've added filing a DMCA removal request to Google to the list of things to do if this continues ...

The rest of what you mentioned we've discussed in prior comments and are indeed helpful in mitigating this.

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Btw, forgot to comment on your remark about the forum.

The thing is, I have "nofollow"-ed all links to the forum from the main site. Honestly, at this point - I don't even remember why I did it, it must've been close to 10 years ago.

Now this makes me think if I should remove those nofollows ...

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Including all IPs I blocked today, they're spread between 5 different ASNs. I may resort to blocking them eventually, but for now - individually blocking the IPs (even in the thousands as it is) - seems to be working well enough.

As for user agent - they're using a very common, real browser user agent that's impossible to distinct from legit users.

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Ey thanks for taking the time to write this, very helpful during this stressful time :)
santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
It's basically what I've done, though have not automated it yet.

It even prepares the expression snippet for me to paste directly into a CloudFlare firewall rule.

That's how I got to quickly identify and ban almost 2000 different IPs.

If they continue to expand the IP pool I may need to automate it though.

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
I'll check out what SurferSEO is suggesting, thanks for the recommendation, though generally - I try to avoid tailoring my content for bots and optimal SEO.

Good catch with the spammy forum profiles - I moderate the forum quite heavily, so there is no spam there, but I never thought to monitor inactive profiles. I'll clean them up and will see what I can do to try and prevent those from popping up in future.

Thanks

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Thanks for the kind words :) It is indeed both terrifying and super exciting and interesting to figure out what is going on (and how to stop it).
santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Update 2: After banning close to 2k individual IPs, it looks like I got it under control, for now.

I wonder how banning so many IPs affects CloudFlare performance and if I should optimize it to block whole IP ranges instead ...

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
They rewrite all URLs pointing to Next Episode to their domain ...
santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
This is a good question I don't have an answer to.

I'll try and find out and also I'll have to learn exactly what "slowloris" is. It may be helpful indeed!

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
A honeypot is exactly how I caught the IPs the first time around.

Problem is - right now I'm over 250 (new) IPs and they keep piling up (their domains now rarely use an IP more than once).

I may have to block entire ranges of IPs or whole ASNs.

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Update: After a week of doing nothing - they finally noticed their thing is blocked and sprang into action.

Apparently, they expanded their pool of available IPs they pull data from and now they seem to be endless (so some of the scraping domains actually work now).

I'm investigating what I can do about it. I'd appreciate any advice!

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Not sure what you mean.

The AD code there is mine and the only thing they added is some JavaScript code for a Russian web page counter ...

Can you please remove the link to the offending site as not to give them any link juice?

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
But what are they ranking higher?

Only the domains that display the stolen content, with no ads or anything.

What is in it for them?

It doesn't make much sense to do it other than if you're trying to affect someone else's ranking by causing them to be penalized for duplicate content. No?

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Until recently, Google had a form you could report scraped pages that outranked your original content.

https://docs.google.com/forms/d/1Pw1KVOVRyr4a7ezj_6SHghnX1Y6...

The fact they removed it tells me they're confident enough in their algorithms nowdays, but that doesn't mean they're perfect.

I can see my own content being outranked by the scraping domains in this very case.

Just google "Streaming Services Availability added to Episodes and Movies" and you'll see.

If they can outrank me, I don't see how this will not affect me negatively by my content being considered duplicate ...

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
I think I can do this easily yes.

I can either fill their content with pornographic stuff or simply redirect it to adult websites.

I think if that happens - Google will de-rank and de-list them super quickly.

I chose not to, because I'm not sure if it's not going to affect real people ...

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
I believe so, yes.

Do you see anything wrong?

santah··on Tell HN: A case of negative SEO I caught on my service and how I dealt with it
Hey there and thanks for looking into it.

I'm aware of the algo update that happened in December (and that it correlated very closely with my drop in rankings).

However, in my experience - even though jumps or drops in ranking are almost always triggered by such updates - there is a good reason for it to happen.

But you're right, what is happening here may've also had no effect whatsoever.

In any case - I found it because I dug up to try and what was going on and I can't explain what I found in any other way.

Even if it didn't affect ranking - does it look like negative SEO to you or do you think something else is going on here?

santah··on Tell HN: Someone seems to be trying to make people more fearful through YouTube
I don't believe this is just a collection of videos.

To me - it rather looks like a purposeful bait and switch manipulation targeted at people who search for TV show trailers on YouTube with the intention to influence them (and as you said - "make them fearful of blacks").

As I mentioned - I've stumbled upon other channels doing the exact same thing following the same formula.

I don't think it's just racist people who happened to upload such videos.

Also, I kinda wonder - what other niches they may be doing this in? TV trailers may be only one of many such niches they target ...

santah··on Pidgin – A Universal Chat Client
Yep!
santah··on Ask HN: Successful one-person online businesses in 2021?
Thanks!
santah··on Ask HN: Successful one-person online businesses in 2021?
Yes - this business supports me and my family and I don't have or need another job. I'm lucky to have found something I'm passionate about and that I can be helpful to others with.

That said - I haven't had a day without working on it (for at least 30minutes to handle support email) in more than 15 years. Also - I plan all vacations around it (well, those are really vacations for my wife, I try and keep working as much as I can on them) by making sure we always have good internet connection and we're never somewhere where we won't have internet for more than a day. Thankfully, that's fairly easy to accomplish nowdays, but I can't, for example, do a multi day camping or hiking trips.

What I'm trying to say it it comes at a price and it's not all great even when it's successful.

And yes, of course I feel I'm competing with the other options - some with huge teams and VC money, but at the same time I regularly check them out and I'm confident my product brings enough value to stand its ground - it doesn't have tens of millions of active users monthly - and it doesn't have to. I'll be happy if I can keep the current scale and the level of satisfaction of my users forever.

Now, I wish I had some advice for someone with a new product in this space. The truth is I have no idea how viable or hard for someone to start something new may be.

If you want - let me know specifically what you're working on and I'll give you some feedback (either in here or through the contact form on Next Episode). (it's fine if you prefer to keep it under wraps too)

santah··on Ask HN: Successful one-person online businesses in 2021?
https://next-episode.net is a service I run for more than 15 years now.

It's a TV / Movies tracking website/community that helps people organize what they watch and find new shows to watch.

I started it for myself (because I couldn't find such tool back in the day) while I was working as a front end developer in an outsourcing company and it slowly grew and became my (only) day job.

There is a lot to do in terms of user support and constant maintenance and development (considering also the iOS and Android apps), but I've made the conscious decision to stay a sole developer and not try and hire people to help. For me - I feel this works great even though I realize I may be missing a lot by not trying to scale up by hiring.

Hope that makes sense.

I'm around if anyone has any questions.

← PreviousPage 2 of 2