Hacker News Reader

Comment by bcoates | Hacker News Reader

Isn't a trivial templating system like this a very easy thing to train a bayesian spamfilter on, even without the source template? I guess blogspammers mostly prey on entirely unmaintained sites for something like this to work.

But I'm going to have a hard time avoiding using "fastidious!" as a general expression of approval now.

View on HN

I don't know how easy it is to train a spam filter on, but nested spintax quickly scales to millions of unique output texts. It can be spun at the paragraph, sentence and word level. It's really nasty stuff.

Sure, there can be millions of unique texts, but they all follow certain very similar patterns. That's what would make even a simple naive Bayes filter very effective against them.

I think you're misunderstanding how naïve Bayesian spam filters work. These techniques explicitly do not recognize patterns at all, they just look at vocabulary, and treat each word as a completely independent feature—so they don't even recognize "discount viagra" as a pattern, but merely two independent features "discount" and "viagra".

You can build a naive bayes classifier with n-gram features.

First of all, you can have any sort of features you want. So you could certainly look at things like sentence structure, pairs of words and so on.

More importantly, I used "pattern" in a very general sense: there is strictly more structure to a post generated with this system than a random post or even a human-authored one. So repeated words between posts are a pattern, and a very important one at that.

Naïve Bayesian filtering might not work very well on this kind of text. It basically looks like a regular comment, until you start recognizing that it always follows the same pattern. Your basic Bayesian classifier will throw all of the words in a set before analyzing them, which loses all of the information about patterns and word order. The resulting words are considered "independent" which means that even though the template might generate the words "pretty worth bloggers content online" every time it uses the first template, the naïve Bayesian classifier will never figure that part out.

My suspicion is that existing Bayesian classifiers have pushed the spammers to develop more natural-seeming templates, like this one.

You can run a Bayesian filter over pairs (or even triplets) of words (although this could cause the probabilities to be a bit off, because pairs like "although this" and "this could" are not truly independent). The downside is that as you do this, you drastically increase the size of the model and the amount of training data required.

Bayesian filters can also take more than just the words in the text into account - for example, they can take the submitting IP address (or perhaps /24 or ASN) into account, or a spam classification from external sources.

There are certainly better methods that could be built for recognising unknown templates - a simple known-state Markov model would be sufficient for the cases where templates substitute one word at a time, and you could conceivably use an unsupervised learning algorithm to discover an unknown number of models from a large corpus of comments.

Something like a Hidden Markov Model might be better then? That way you can can keep the information about word order.

Reply on news.ycombinator.com