MicahKV

Huh, well I guess there goes my theory about the incentive. What a bummer. I would have thought that at least with search engine scraping, they would stop expending the effort once the results dried up.

Well what I'm suggesting isn't about blocking the bots, it's about removing the incentive. So in this case, I think the more obvious it is the better. I would want them to realize as soon as possible that they are 100% wasting their time.

If anything, it might be best to return a page that explicitly states "Sorry, this search engine no longer supports SEO footprint search queries."

*edit for typo & wording

I'm not so sure about this. The spammers goal is to build up as big a list of link spam targets as possible. If one spammer chooses to only scrape minor engines and another only major engines, the one scraping the major engines will probably come out on top despite the higher cost. Whoever is abusing OP's search engine is likely doing it to supplement the data they are already scraping from the major engines.

For OP, I think simply not returning results at all is a more practical measure because it removes the reward completely. Captchas and bot detection keep the reward in play, while taking away the results entirely makes the entire pursuit futile.

That would probably help, but it's also a continuation of the cat and mouse game. There are plenty of captcha breaking services out there, it only cost about $1 to programmatically solve 1000 captchas.

So spammers have latched onto your search engine because they are getting useful results. They are able to systematically discover websites built on certain platforms that allow users to post content containing links, which they can target for link spam. It is very difficult to fight this on a technical level because there is an entire industry built around blackhat SEO, with all kinds of softwares and services dedicated to thwarting your defensive efforts. Even Google struggles to keep up with this.

However, they are also systematically feeding you their footprint lists. I imagine you could put together a footprint blacklist pretty quickly, and just stop returning results for any obvious spam queries like those containing "powered by wordpress".

It's not a very elegant solution I'll admit. It won't stop the bots from trying, and you may have to circle back periodically to add new footprints as they surface. But it's a potentially quick and easy way to stop rewarding their efforts, and the blackhat world is pretty used to burning out their resources so hopefully they will figure out it's a dead end and move on.

Been a while since I read up on this, but I think the problem is the energy cost of extracting CO2 from sea water makes it a wash at least in terms of CO2 reduction.

I don't see the bizarre part. Wikipedia says Barron was born in 1927 when we were still figuring out the basics of aviation, and just a few decades later he was making these plans in the midst of the space race. Considering the rate of progress he had witnessed in his time, it's not very surprising that he expected us to aggressively colonize space in his lifetime.

Maybe I have overly romanticized notions of what could have been, but I feel like humanity lost out big time when the space shuttle was chosen over a manned mission to Mars.

I like to think that advanced alien life may already be established in our solar system, and we just haven't detected it yet. Unlikely as that is, the solar system is a big place and we are still just scratching the surface in terms of exploring it.

Perhaps we have a highly advanced neighbor that values their privacy and has the technology to remain concealed from our "primitive" observation capabilities? Again, not likely but it's fun to think about.

The date becomes ambiguous, it just says "September 2020".

I contributed to this spike. I wanted to make sure I didn't mangle the spelling in my FB & reddit posts regarding President Trump's COVID-19 diagnosis. In case anyone is wondering, the gist of my posts was this: "For all the poetic justice or schadenfreude one may feel right now, this is really a very bad development."

When I switch to 30 days, the results appear to only be current to the 29th of last month.

> If you don't like it, you move somewhere else.

Yes, and if you don't like existing regulations you can just move to another country with different laws. No big deal, right?

I do operate my own site in addition to selling on multiple marketplaces, but even with my efforts to diversify I am dependent, in part, on the sales I get from Etsy. I would be in a lot of financial pain if I lost that revenue stream, so "just move somewhere else" is not really an option.

Sometimes regulations are necessary to keep big companies (Amazon, Ebay, Etsy) from abusing their customers (including small time sellers like myself). Regulations may not be the answer in this case, but it's at least worth discussing.

I do host my own website and manage all of those things, but it's not a question of the cost it's the fact that I can't opt out. It impairs my ability to predict & control my expenses, and it forces me to subsidize Etsy's ad campaigns that compete directly with my own website.

They are not a surprise, no. I have wondered before what marketplace sellers could do in the way or organizing/unionizing.

I'm one of those sellers who is being forced into Etsy's new ad program. It is frustrating and obnoxious, but I can't say it is surprising given the policy changes they've been rolling out the past few months.

It seems to me that online marketplaces like Etsy find their early success by bringing buyers and sellers together and just letting them do their thing, but as the marketplace becomes more established it's relationship with sellers inevitably turns adversarial.

I feel like we need some form of regulation to limit how marketplaces like Etsy can dictate how their sellers conduct business. Amazon has a program that allows "business" customers to place an order and pay the invoice 30 days later. Since Amazon only pays out every 2 weeks, this means I can be made to wait up to 6 weeks to be paid for an order. There is no way to opt out of this and I would be penalized if I refused to ship these orders.

The Apple in my classroom was also never used for the most part. And with no hard drive and no spare floppy disks, I couldn't save any of my work anyway which made it hard to learn past a very basic level. But it planted the seed, and allowed me to hit the ground running with QBasic :)

Wow, what a blast from the past. My introduction to programming was on an old Apple II with BASIC and LOGO. I was in the 6th grade at the time, and never did anything as complex as looping to draw spirals - I basically used LOGO like an etch-a-sketch.

I've been reading about VC and all it entails for years now, and it always leaves me with a sketchy feeling. Pouring so much time and effort into chasing capital, and then giving up so much ownership and being on the hook for big returns once it's finally attained... seems like a sure way to suck the soul out of any business.

Not saying VC doesn't have a place in the world, but I get the sense that a lot of founders pursue VC just because that's what startups are "supposed" to do.