28 June 2005
Understand how various spam software works
There are four major types of spam fighting technology. We discuss those types here
When considering how to protect your mail system from spam, you'll find that there are too many choices in the marketplace to be able to evaluate them all. Since each solution handles spam differently, it's important to understand the various methods by which spam filters work. There are four major types of spam technology available. I'll discuss each here.
BayesianBayesian filters use complex statistical algorithms using existing information to determine the probability that a message can be trusted. The term 'existing information' is important as it means that this type of solution requires an initial period during which it may be less than effective at capturing spam. However, many people report that, once filters are trained, they do an excellent job of canning spam with a minimal number of false positives. Further, since by its nature, a Bayesian filters learns from its mistakes, it generally requires less ongoing maintenance than other types of filters, and the filter is good at adjusting its parameters to meet the needs of the individual user. On the con side, spammers have found ways to defeat some of the measures used by these filters. If you've ever received a spam email with a large number of nonsensical words, you've seen this in action. By inserting enough valid words into a message, a spammer can fool a Bayesian filter into thinking a message is legit.
Whitelist and blacklistIf an address or domain exists on a whitelist, the message is allowed through; in fact, only messages from addresses on the whitelist are allowed through. If, on the other hand, an address or domain is on a blacklist, it's blocked while all other messages are allowed. There are a number of blacklist services-called RBLs, for RealtimeBlackhole Lists, that compile lists of known spammer addresses. However, RBLs can be problematic in that if they're not maintained, or they're maintained by an overzealous administrator, legit senders might be blocked. The pro side of white and black lists is their simplistic nature. For this kind of spam filter, there is only the dark side and the light side. There is no in between. On the con side, they require a huge amount of maintenance, especially for whitelists, which require an entry every time you want to add a new allowed sender.
Content-basedVery simply put, these kinds of spam filters look for certain words, such as 'Viagra' and kill a message if those words are present. These filters require significant administration in that each time you want to block a new word, you need to create a rule. Further, spammers have found it child's play to get around these kinds of filters. They use a variety of ways to do this. One way is to make the word still readable, but different. For example: 'V.i.a.g.r.a'. You can certainly create a rule that blocks that version too, but spammers have become even sneakier. In some cases, you might look at the word 'V.i.a.g.r.a' and wonder why your filter didn't catch it. If you copy and paste the word into Word and change the font size to something larger, you'll notice that the spammers don't use periods between the letters at all. Instead, they use a variety of characters with a font size of 1 so that they look like a period, but can get past filters.
Challenge/responseIn a desperate move to thwart spammers, some new spam systems require senders to basically prove that they are allowed to send mail to someone. Before a person using this system receives a message, the sender must visit a web site and answer some questions. The pro is that this system virtually eliminates spam. The con is that it's a pain in the neck for legitimate senders.
When considering how to protect your mail system from spam, you'll find that there are too many choices in the marketplace to be able to evaluate them all. Since each solution handles spam differently, it's important to understand the various methods by which spam filters work. There are four major types of spam technology available. I'll discuss each here.
BayesianBayesian filters use complex statistical algorithms using existing information to determine the probability that a message can be trusted. The term 'existing information' is important as it means that this type of solution requires an initial period during which it may be less than effective at capturing spam. However, many people report that, once filters are trained, they do an excellent job of canning spam with a minimal number of false positives. Further, since by its nature, a Bayesian filters learns from its mistakes, it generally requires less ongoing maintenance than other types of filters, and the filter is good at adjusting its parameters to meet the needs of the individual user. On the con side, spammers have found ways to defeat some of the measures used by these filters. If you've ever received a spam email with a large number of nonsensical words, you've seen this in action. By inserting enough valid words into a message, a spammer can fool a Bayesian filter into thinking a message is legit.
Whitelist and blacklistIf an address or domain exists on a whitelist, the message is allowed through; in fact, only messages from addresses on the whitelist are allowed through. If, on the other hand, an address or domain is on a blacklist, it's blocked while all other messages are allowed. There are a number of blacklist services-called RBLs, for RealtimeBlackhole Lists, that compile lists of known spammer addresses. However, RBLs can be problematic in that if they're not maintained, or they're maintained by an overzealous administrator, legit senders might be blocked. The pro side of white and black lists is their simplistic nature. For this kind of spam filter, there is only the dark side and the light side. There is no in between. On the con side, they require a huge amount of maintenance, especially for whitelists, which require an entry every time you want to add a new allowed sender.
Content-basedVery simply put, these kinds of spam filters look for certain words, such as 'Viagra' and kill a message if those words are present. These filters require significant administration in that each time you want to block a new word, you need to create a rule. Further, spammers have found it child's play to get around these kinds of filters. They use a variety of ways to do this. One way is to make the word still readable, but different. For example: 'V.i.a.g.r.a'. You can certainly create a rule that blocks that version too, but spammers have become even sneakier. In some cases, you might look at the word 'V.i.a.g.r.a' and wonder why your filter didn't catch it. If you copy and paste the word into Word and change the font size to something larger, you'll notice that the spammers don't use periods between the letters at all. Instead, they use a variety of characters with a font size of 1 so that they look like a period, but can get past filters.
Challenge/responseIn a desperate move to thwart spammers, some new spam systems require senders to basically prove that they are allowed to send mail to someone. Before a person using this system receives a message, the sender must visit a web site and answer some questions. The pro is that this system virtually eliminates spam. The con is that it's a pain in the neck for legitimate senders.
# posted by vdodon @ 00:38 
