8chanAnon

Third article in a series. Explores eval and self-invoking functions.

My next article will be on the topic of bypassing the Cloudflare bot protection. You can then compare with how Selenium handles this problem (if at all).

Every dark mode site in existence should fail that test.

>Some walled garden sites seem completely unscrapable

Any examples besides Linkedin? Tell me what sites you're trying to target and I'll have a look to see what can be done with them. It takes some pretty evil Javascript obfuscation to block me and only one site has been able to do that. I doubt that the sites you're hitting are anywhere near that evil, lol. I would appreciate it if you have a good example that I could use in a future article.

WASM runs in a sandbox. It can only talk to the outside world via Javascript so you can forget the idea that it might be a way to crack through browser security.

Maybe somebody will make a web browser with all of the security locks disabled. Sort of like the Russian commander in "Hunting for Red October" who disabled his missiles' security features in order to more effectively target the American sub but then got blown up by his own missile.

Is there a happy in-between? Maybe not. What looks perfect to one user might appear atrocious to another. What is a poor website operator to do???

I dislike black-on-white and don't understand gray-on-black which seems to be popular now due to gamma settings being cranked up to 11 or something. I try to use some color as an in-between but that may take some time to "perfect".

Red text on yellow - you mean the website? Would you like the text to be darker?

And Beautiful Soup should be BeautifulSoup. Who makes the rules?

I see. I don't have a wide-screen monitor (still using an old tube type until it finally expires but it's taking a few decades, lol). I've wondered whether people actually like reading websites on wide-screen. Some do and some don't. What would you suggest for a max width?

You could also try zooming in. My apps don't expand to full width because of the video box but you can zoom.

>bad formatting

If you can elaborate, I would very much appreciate it. I'm always interested in doing better.

Why use Puppeteer etc. when you don't have to? What is the argument for using these additional tools versus not using them?

Rather long so I'll read it later. Thanks for the tip. Got more or is that it?

I voted "light" because there is no "medium" option. Why is it a choice between gray-on-black or black-on-white? I would choose a "color" mode if there was such an option on the table. Maybe we can see a return to the yellow-on-blue mode which was popular with text editors a few decades ago?

I've been asking the same question myself though I'm not trying to get paying customers. I'm offering free stuff but I can't get many takers. One customer per 1000 views is pretty much the best you can expect. You need a million views to snag 1000 customers. I would suggest that you stick to your plan and go through the free routes first. This will help you to gauge the level of interest before you bankrupt yourself paying for marketing (which never lives up to expectations).

I do have an anecdote. I have a repository on the Internet Archive. One of my items had just a few hundred views but, all of a sudden, it accumulated over 3000 views. I posted a comment on the item asking what sparked the sudden interest. Someone replied that it was a user comment on Youtube. Makes me wonder if I'm not making a huge mistake by not registering on Youtube so I can post self-promotional comments everywhere.

Obligatory shameless plug is below.

https://8chananon.github.io

I've updated the app to remove the user agent string so that the link will not redirect to "m.youtube.com". I hope that's all that's needed to fix this.

Youtube just changed their code. The app is not currently working. Coincidence?

I might have to wait a bit to see what else gets changed.

Sorry about that.

Update: Fixed! Will keep an eye on it.

I have information that Youtube is redirecting from "www.youtube.com" to "m.youtube.com" when it detects a mobile user agent. The page data that is returned appears to be incompatible with the app's parsing code.

Will figure this out.

I have no idea what the potential compatibility issues might be with mobile. This has only been tested on PC. Any way to provide debugging info?

Hmm. The title was supposed to be "YouTube Player app in your browser".

Why was the word "your" replaced with "the"?

Who uses the Start Menu? I click on that maybe once in a blue moon to maybe uninstall something. This is just another reason to start doing the Linux thing once this Win10 machine needs to be replaced. Ironically, the reason why I didn't get into Linux is because whatever it was that I installed long ago looked the same as Windows and it had a Start Menu (by a different name). Yeah, I guess I could have reconfigured or replaced the desktop manager but it wasn't worth the bother at the time. The calculus, however, is changing rapidly.

The strong nuclear force is a figment of the physicists' imaginations. Likewise the weak nuclear force, dark matter and dark energy. The Standard Model does not align with reality nor reality with the mythical Big Bang, hence the need to dream up these unicorns. For some enlightenment, check out Miles Mathis:

http://milesmathis.com/

He may be wrong on some things and maybe he's really the CIA leaking secret info to the rest of us pleebs for whatever reason but a lot of what he says makes scary good sense. Regardless of whether you are convinced, you will come away with the impression that modern physicists are indeed a bunch of whackos who are just going through the motions in exchange for billions of dollars in grants. When they can't get their observations to match their predictions, they just fake it (the Higgs boson).

I sympathize. The task is easily accomplished with a properly coded regular expression but, damn, regex is haaaard. Also, there are too many gotchas in parsing random text. It's a no-win situation sometimes. Doing nothing might be the better idea.

We're doing this again???

This post should have more replies, most of which would be cynical because we've had it with virus scares. They've been doing it for all of our lives until they finally got what they wanted in 2020. Fool me once...

>Screw the games, I need to know what this thing does.

Sounds like me. My first fascination was with my Texas Instruments scientific calculator. Then I got an Ohio Scientific Superboard. I really really needed to know how these things do what they do. Now I know. Sort of. It has gotten mucho more complex since then.

I have a question for you. Why use Python at all? Or any of these puppeteer-type apps? If you're trying to scrape web sites then would it not make sense to do the scraping from inside of the web browser rather than from outside using these clunky and complicated methods? I mean, write the code in HTML/Javascript and load the page in any browser (Selenium and Puppeteer require Chrome). A local proxy server would, of course, still be vital even if only to bypass the CORS restrictions (though a good proxy needs to do much more than that). The bonus of running Javascript in the browser is the ability to use exec (important tool for cracking code obfuscation) and, of course, to parse JSON and HTML. Also, you can solve the Cloudflare bot challenge (or any bot challenge) by running the challenge in a new tab and stealing the cookies via the proxy server.

Just asking because I'm already doing this. I mentioned this months ago but I'm still preparing the tools for official release.

We have too many "particles" already. They keep coming up with these new-fangled ideas because they can't admit that the Standard Model is crap and needs to be scrapped. These so-called particles are all just manifestations of the same thing. Just need to figure out what the thing is. Also, the article is thick with "mad scientist speak". Is it not possible to explain it in English?

A simple search on "javascript import export" will yield much information on this topic. You can only import what has been explicitly exported. I don't know of any projects that are handled this way. Normally, you would just use the script tag in html. As for Node, I have only used "require" to import the standard modules but an npm package should import the same way.

>seem to leave out all instructions for how to actually get the files there, as if it's obvious.

I see this all the time and it's frustrating. I can't even begin to count the number of hours lost hunting down little snippets of information simply because some people think it's too "obvious" to mention. This is usually because people are pretenders. They don't actually know what they are doing because they are simply repeating what they saw on the Internet somewhere. You just have to laugh (to keep your sanity). My favourite example is people explaining how to make an SSL certificate. It's so simple, you just type this OpenSSL command. How do you get this "OpenSSL" they speak of? I've tried using OpenSSL and it's a bitch. It's bad enough that I've rolled my own code to make certs just so I don't have to use OpenSSL because I can't remember how I managed to use it the last time. Just have to laugh.

>And finally, if you answer this, can you spare a word about typescript?

Brother, you are asking a lot of questions that I can't answer but hopefully someone else will. I have to ask: why do want to use typescript? Did somebody tell you that it's better than javascript? For what, exactly?

P.S. Looking forward to further discussion on this.

Advice for the easily offended:

Sticks and stones may break my bones but words can never hurt me.

Some people never grow up and that's not an issue that can be solved with AI.

It's hard to go back to a life of being used and abused by the corporations. With the Covid policies, we found out just how little the corps actually care about their workers. Now they're begging to have them back. Good to see.

Interesting. However, it requires 5 megabytes worth of support code so the use case is somewhat limited.

Another article from the Wall Street Journal. How to get rid of the paywall? This extension works (there is a Chrome version too):

https://gitlab.com/magnolia1234/bypass-paywalls-firefox-clea...

In the case of WSJ, these two little things will bypass the paywall:

1) use "https://www.drudgereport.com/" as the Referer for "www.wsj.com"

2) block the cookies

I'm using a proxy server that allows me to do this so I don't need the extension.

Pick an Invidious instance:

https://redirect.invidious.io/

Problem solved.