Felk

I think most of the confusuion comes from "AT TIME ZONE" being the syntax for both the conversion from _and_ to timezone'd timestamps. Maybe it would have been more intuitive had the two operations gotten different wordings, e.g.:

timestamp to timestamptz: AS ZONED AT TIME ZONE ...

timestamptz to timestamp: AS LOCAL AT TIME ZONE ...

Thanks, looks very promising! I'll dig into this the next time I'll need to archive a web page.

I see that the author took a 'heuristical' approach for retrying tasks (having a predetermined amount of time a task is expected to take, and consider it failed if it wasn't updated in time) and uses SQS. If the solution is homemade anyway, I can only recommend leveraging your database's transactionality for this, which is a common pattern I have often seen recommend and also successfully used myself:

- At processing start, update the schedule entry to 'executing', then open a new transansaction and lock it, while skipping already locked tasks (`SELECT FOR UPDATE ... SKIP LOCKED`).

- At the end of processing, set it to 'COMPLETED' and commit. This also releases the lock.

This has the following nice characteristics:

- You can have parallel processors polling tasks directly from the database without another queueing mechanism like SQS, and have no risk of them picking the same task.

- If you find an unlocked task in 'executing', you know the processor died for sure. No heuristic needed

Downloading a DB dump and crawling locally is possible, but had two gnarly show stoppers for me using wget: the forum's posts often link to other posts, and those links are absolute. Getting wget to crawl those links through localhost is hardly easy (local reverse proxy with content rewriting?). Second, the forum and its server were really unmaintained. I didn't want to spend a lot of time replicating it locally and just archive it as-is while it is still barely running

The slowdown wasn't due to a lot of permutations, but mostly because a) wget just takes a considerable amount of time to process large HTML files with lots of links, and b) MyBB has a "threaded mode", where each post of a thread geht's a dedicated page with links to all other posts of that thread. The largest thread had around 16k posts, so that's 16k² URLs to parse.

In terms of possible permutations, MyBB is pretty tame thankfully. Only the forums are sortable, posts only have the regular and the aforementioned threaded mode to view them. Even the calender widget only goes from 1901-2030, otherwise wget might have crawled forever.

I originally considered excluding threaded mode using wget's `--reject-regex` and then just adding an nginx rule later to redirect any incoming such links to the normal view mode. Basically just saying "fuck it, you only get this version". That might be worth a try for your case

Funny seeing this here now, as I _just_ finished archiving an old MyBB PHP forum. Though I used `wget` and it took 2 weeks and 260GB of uncompressed disk space (12GB compressed with zstd), and the process was not interruptible and I had to start over each time my hard drive got full. Maybe I should have given HTTrack a shot to see how it compares.

If anyone wanna know the specifics on how I used wget, I wrote it down here: https://github.com/SpeedcubeDE/speedcube.de-forum-archive

Also, if anyone has experience archiving similar websites with HTTrack and maybe know how it compares to wget for my use case, I'd love to hear about it!

Thankfully some service providers noticed the awful convenience and are trying to proliferate themselves by offering a better service. For example mop.la let's you buy the ticket via credit/debit card, gives you a digital ticket and let's you pause or cancel the ticket a day before the next month. (I'm not affiliated with them)

You can use the Home key to skip cutscenes. Click on "show help" for more alternate controls

Can someone explain to me what the main differences to jq are, besides the syntax?

Wow, that looks extremely useful. I typically use docopt for all my CLI needs, but it looks like this could be really nice as well. I will have to try it the next time I'm building a CLI.

It is. As far as I'm aware issues like these are only problematic if you either manually run a workflow (it uses your credentials) or have a workflow with the "pull_request_target" trigger (uses a token with write access). The latter has a plethora of potential pitfalls and should be avoided if you can.

I believe you are confusing dark net with deep net

Well, yes. The question was whether you can sign _on GitHub_, so your private key has to be available to GitHub. You can always sign locally if you don't trust GitHub.

Yes. Here's an excerpt from their documentation on <https://docs.github.com/en/github/authenticating-to-github/m...>:

> GitHub will automatically use GPG to sign commits you make using the GitHub web interface

> Please delete user 01FEY4XQ988G9BYFH45JKQZF5R .. .

It's one thing to make a valid complain about there not being an automated account deletion feature yet, requiring you to send an email. And even then it's not that bad since it's literally a "mailto:contact@revolt.chat?Subject=Delete my account" link. It's another thing to make a temper tantrum out of it: https://imgur.com/U2DJwm7

Please don't be the guy that's technically right but an ass about it.

Note that this post was not made by any of the developers but by a third party, and they chose the title.

One thing some may consider "decentralized" is that you can self-host

I've heard that too. But I am using gmail and had no trouble registering.

There's nothing wrong with it per se, in fact Discord is very good software, but it being a for-profit company using closed-source code comes with the usual privacy and free-software concerns. For example here's a post on reddit about some privacy concerns with discord: https://old.reddit.com/r/privacy/comments/eiicah/trawling_th...

> half of our devs dont like sql much

I'd suggest you and your team shouldn't rule out SQL-like databases, given a lot of very competent NoSQL databases have SQL-like syntaxes (say Cassandra or ScyllaDB, what Discord went with). And regarding hiding it behind an ORM, if you want or need cream of the crop performance not only do you need to have chosen a database that fits your needs, but you will also need to occasionally work very close to the database to avoid abstraction inversion situations.

Is it really "disturbing" if it's maintained by mostly college students? If this was done by industry veterans I could understand your stance, but the way things are this feels uncalled for.

I can also recommend licecap (https://www.cockos.com/licecap/), which is quite minimalistic but gets the job done

I am personally abusing the block feature for this. I haven't tested if it is actually true, but I began blocking channels that I am not interested in and haven't noticed anything being recommended yet from any channel I remember blocking.

You may also be interested in taking a look at [quarkus](quarkus.io) in that case. It's a relatively new platform developed by RedHat that focuses on cloud usability, performance and developer joy, while using well-established APIs like JAX-RS, JPA or microprofile

Not official, but I have had good experiences with AvaloniaUI, which sticks very close to WPF and is part of the .NET foundation umbrella. It inherits quite a lot of what made WPF mature and robust

Note that this is from 2012

Speaking of memory leaks, I once had to debug a memory leak caused by console log entries. Objects that are logged to the console are prevented from being garbage collected, even if the console was never opened. That includes DOM nodes or I think also handles to WebGL textures.

For that particular use case it may be better to pass a constructor reference / new instance supplier to not rely on reflection.

While it does need to maintain backwards compatibility, Project Valhalla seems promising on improving on some of Java Generic's pain points. So it's not a dead end quite yet

As "better" isn't clearly defined, here are some things that some people might consider superior:

- primitives or any custom value types/structs can be used. No need for boxing or method duplication for primitives. - type information available at runtime, no need to pass 'Class<T>' parameters. - type constraint 'new' available, which means it must have a default constructor so you can do 'new T()'. No need to use reflection or pass a constructor parameter. - type constraint 'notnull' available - class-scoped compiler enforced covariance and contravariance, though I think you can get similar results with java's "? extends T" and "? super T" constraints