michaelbuckbee

(author here) I had looked at both Rqlite and the different commercial versions of this, but I didn't pursue them as they all seemed to require running an additional service on the host machines.

1. Very cool

2. If you were making it today would you just use SQLite?

(author) - It's genuinely delightful to know that you liked it.

This is something we seriously considered. The SQLite dbs are several hundred megabytes in size (millions of IP ranges) so while it would be technically doable to send around rules files as JSON or something more specifically suited there's still a number of wins that SQLite gives us:

- Really strong support across multiple platforms (we have clients for most of the major web frameworks)

- Efficiency, sure we have lots of RAM on servers nowdays but on some platforms it's constrained and if you don't have to burn it, we'd just rather not.

- When we started mapping this out, we ended up with something that looked like a JSON format that we were adding indexes to....and then we were re-inventing SQLite.

The answer is "yes." We had indexes - but it's also a little more complicated than that, as we're storing IPv4 and IPv6 ranges in a single table in a format _designed_ to be indexed a particular way.

In the article, we refer to this as "decimal lexical" formatting, where we're taking the IPs and making them integers but actually treating them as strings. We're doing this in both Redis with sorted sets and then in a single table in SQLite.

I was going to explain all this in the article, but it was too long already, so it will be a future blog post.

The sqlite db is on disk sync'd down to the clients from our service.

The client is responsible for checking with our servers and, if rule updates are found, downloading a new database file. To avoid locking and contention issues, these are each uniquely named, and which DB is "current" is just updated.

Note: This is only in "managed" mode. If you'd rather, you can distribute a SQLite database of the rules alongside your app.

In Redis, the data is a sorted-set that we forced into being lexicographically ordered by setting all the scores to 0. We went through a lot of iterations of it and to be clear it's not _slow_ it's just not as fast as essentially `fopen`

1 - Redis sorted sets - https://redis.io/docs/latest/develop/data-types/sorted-sets/

SQLite for distribution is neat. FWIW - this is at least partially inspired by your datasette project which we may still try and do something with later on the reporting and data exploration side of things.

(article author here) - yes 100% and I hope that came through in the article that this is great solution given our particular use case and that it's not a 1:1 swap out of Redis or Postgres.

Post author and Wafris co-founder here. Conceptually "rate limiting to prevent abuse" (what we're doing here) and "rate limiting for API throttling" have different levels for tolerance.

With that in mind, it's setting higher levels of limiting and doing the math to push that out over many machines/instances/dynos. That helps for things like scraping prevention, etc.

For issues like credential stuffing attacks, you'd want a lower limit but also coupled with mitigations like IP bans, IP reputation, etc. to deal with underlying issue.

fwiw - if you click through to the complete instructions PDF they do link to multiple different studies on the topic: https://fourthievesvinegar.org/wp-content/uploads/2024/09/to...

I'm not a big fan of getting dental work done, much less "DIY home dental" so was pretty skeptical of what they could possibly be doing and was really pleasantly surprised by their tooth seal instructions.

They're taking a somewhat well-known cavity prevention and enamel remineralization treatment that has the unfortunate side effect of turning your teeth black and replicating steps from a study to avoid that.

One of several studies they link to: https://www.scielo.br/j/bdj/a/rHSG9jRQDdY7sCFZzpNXYXy/?lang=...

It's described in the article that they're going to use Wordpress as a sort of "backend" platform for the existing Tumblr front end interface and that this is to mesh with their existing hosted architecture.

Yeah, so in that v2 Ruby/Rails client if you'd rather not use Hub you can do things like push 50,000 IPs into the SQLite DB and then distribute that along with your app (or distribute the db however makes sense for your deployment).

Hey - Mike (Wafris cofounder here)

To your questions: there's an OSS core in that you can use the firewall itself without a connection to Wafris Hub.

We're in a transition where we're moving from our v1 (which required Redis) to a v2 on all clients (which uses locally replicated SQLite) and the docs and clients are in different states.

Some what's driving this is:

1. Solutions like Fastly + Cloudflare are out there, but there's still way too many sites without any "default" type protection.

2. It remains a big issue to try and sync and scale these types of systems if implemented within a framework.

3. It's easy to find stuff to block (look in your logs and it's all sorts of dumb attacks, scrapers, bots, etc.) but much harder to close the loop on discovering what to block. This is really "Wafris Hub" (the web gui) that tries to close the loop on this.

Would love more feedback, either here or mike@wafris.org

Sorry about that, we just put up the refreshed docs site this morning.

https://wafris.org/docs/installation/nodejs/ should work for you now.

In Pratchett's "Going Postal", there's the following quote:

There is a saying, “You can’t fool an honest man,” which is much quoted by people who make a profitable living by fooling honest men. Moist never tried it, knowingly anyway. If you did fool an honest man, he tended to complain to the local Watch, and these days they were harder to buy off. Fooling dishonest men was a lot safer and, somehow, more sporting. And, of course, there were so many more of them. You hardly had to aim.

Setting aside how hard it is to get good estimates of these things, aren't those independent stats?

Calling them "scum" seems pretty harsh. By value, wage theft is the largest economic act of thievery in the economy - I wouldn't call it strictly ethical, but there's a balance to it.

After having seen the rise and fall of a number of "robot to make your pizza", "robot fry cook", etc. startups, I've been pitching a new rule that:

There is an inverse correlation between how much a robot looks like a _robot_ and how useful it is in the real world.

I've a parallel theory that there's a hidden wage decline in service jobs where the move to computerized and card-based payments severely reduced the opportunity for service workers to easily steal from their jobs.

I saw this firsthand with a pizza shop that went to computerized ordering from paper tickets. The drivers could no longer "lose" tickets and pocket the cash payment (resulting in a wage loss of ~$10/hour or more).

I've anecdotally heard similar from lots of folks that used to work fast food and service jobs.

I also came here to try and convince folks to read "Ra" which I thought was fantastic.

Though that being said, I feel like we're flipped on which is more "out there" as Ra feels much less slippery of an idea.

My last visit to NYC there was a pronounced uptick in wall sized LED screens being used for everything from art pieces (at the new Laguardia) to info screens to just shops with massive screens in their storefront in lieu of showing physical goods.

I don't know if this is exactly it, but your idea for designing a system that encourages fewer but much higher-quality interactions is a good one.

This was my thought as well, but someone pointed out to me that regional accent identification captures a large percentage of cadence and inflection differences (specific word choices and turns of phrase obviously would still not be there).

Is the US equivalent of 7-11 something more like WaWa?

For anyone else curious about the exact service area: https://support.google.com/waymo/answer/9059119?hl=en

For a peek behind the modern equivalent of this checkout this podcast ep on the behind the scenes of using AI to create NSFW chatbots - https://www.latent.space/p/nsfw-chatbots

Obviously, the answer is you don't.

My personal concerns are more that we are going to see the botnets get smarter and smarter and more automated.

Just imagine if every script kiddie running a "dumb" botnet suddenly got 2x more effective.

They have a submission button, but it asks for a lot of info, it would be kind of nice if you could just paste a github repo and it would pull out the information.