Evolution spam detection with Bogofilter

I’ve not been posting much techie blog stuff recently. I’ll try to rectify the situation, starting with this entry.

My Gnome mail client, Evolution, uses Spamassassin as a junk mail filter. In my experience, with remote checks disabled, it is completely retarded. It seems to detect 0% of spam.
I rigged up Evolution to use Bogofilter instead (hey, Eric S. Raymond wrote it!). The result is a considerable jump in detected spams, and getting better with each new training.

Evolution isn’t really designed to use anything but Spamassassin as the junk mail filter, which I find odd. Especially considering how effective bogofilter is for me compared to it. I worked out a hack to get it going.

Here are the details. I’m using Evolution 2.40 and the menu layouts were recently re-designed so things might be in different places in older versions. The same functionality is there though, just beyond different clicks. I assume you have bogofilter installed.

Turn off junk mail checking: untick Edit->Preferences->Mail Preferences->Junk->Check incoming mail for junk

Add a new message filter rule, call it “bogofilter check”. Move it to the top of the rule list. This will check incoming mail for spam using bogofilter and set the status to Junk. Make it look like this:
evolution screen shot


Add another filter rule, call it “bogofilter teach spam”. Move it to the top of the rule list (above the last rule). This rule will detect when you’ve manually marked a spam as Junk, and have bogofilter re-learn it. Make it look like this:
evolution screen shot


The set up is now done. The process of training bogofilter has one extra step than when using Spamassassin. When you mark an undetected spam as Junk it is moved to the Junk folder. You now need to find and select the message in the Junk folder and manually “Apply Filters” (Message->Apply Filters) (or Cntrl+Y). This is the hacky bit.

If Evolution let us specify message filter rules to be applied to messages when their status is changed this wouldn’t be so much of a hack but it doesn’t, so it is. If Evolution could selectably support Bogofilter as well as Spamassassin”internally” this would be even better. For now, this works for me.

Another disadvantage of this is the inability to have Bogofilter “unlearn” a false positive. To teach Bogofilter that a message it previously thought spam should be known as non-spam, you’d have to save the message out as a file and manually pipe it into “bogofilter -N”

It’s late and I’m very tired. These instructions may well be a figment of my sleep deprived geek imagination. Handle with care.

UPDATED: The screen shot for the “bogofilter check” rule should have “/usr/bin/bogofilter -u”

Comments

Pierre says:

Seems great !

I stopped using evolution some times ago because of crappy spam detection. I will give it another whance using your trick. Thanks very much for the detailed explanation.

dr. yes says:

Shouldn’t it be possible to have bogofilter unlearn false positives using a filter like the above one yet with -n ?

Hey, nice tutorial. I agree completely with your reasoning about evolution supporting more than one anti-spam. Also, it would be really easy if it supported “per-message pipes”, so that you could pipe a message through bogofilter to tell it wether the message is spam or not.

diego.

john says:

my guess is that it would be easy to implement other anti-spam systems via an evolution’s plugin architecture. I should have a look at this some day.

Evan Monroig says:

The man page of bogofilter says that you can unlearn a false positive by launching bogofilter with options ‘-Sn’ and unlearn a false negative by using options ‘-Ns’

Evan

Israel says:

Very nice this tutorial. My best thanks to you.

Andy says:

I’ve got a pretty long list of friends and family in my filter list – they get moved over to a special folder. Same for vendors.

I’m thinking that I insert:

/usr/bin/bogofilter -h

..before moving to their special folders. Wouldn’t that help the baysian filter better understand what my ham looks like?

Andy says:

Oops, I mean /usr/bin/bogofilter -n

[…] 默认插件bogofilter是激活的,一直没怎么注意,后来见别人讨论才知道还需要专门的配置才能让其生效,汗! 具体的设置可以参考: http://johnleach.co.uk/words/archives/2005/09/15/180/ […]

Magnus says:

Very nice this tutorial. It realy helpd me out.

Nice tutorial. Let’s see if it works. Expect feedback in a few weeks. I’m getting that goldmark spam, so there will be enough to teach bogofilter.

Piotr Bogus says:

Thanks, for the tutorial, the only one i found on net, handling evolution and bogofilter. As you mentioned, spamassasin did not find any of my spam, now i hope bogofilter does.

Torbjorn says:

FYI, This tutorial doesnot work on Ubuntu Edgy Eft 6.10, which comes with Evolution 2.82. In this release of Evolutio, you have no option to include any parameters to the program. THere is only a browse button and no textfield you can enter it manually.

livingdaylight says:

AHA…as i came to put my question i see that comment#13 by Torbjorn has already beat me to it. I was going to ask where this ‘Edit Rule’ thing is, and wondered whether it was due to having Evolution 2.8.1 but i couldn’t add bogofilter following your hack. Maybe there is another way? Torbjorn seems to suggest that there isn’t. Maybe things are improved anyway/evolved with v2.8.1 I’ll be putting spamassassin to the test. Please do let us know if you come up with another way though…

Jeremy says:

With Ubuntu 6.10/Evolution 2.82, you need to create a wrapper script to invoke bogofilter correctly.

Like:
$ cat bin/bogofilter-u.sh
#!/bin/sh
exec /usr/bin/bogofilter -u

sheepshearer says:

evo 2.8.1 on Ubuntu 6.10 (edgy) here too – and SpamAssassin won’t even run (due to some well documented Perl dependency).

however, to get bogofilter working you can do as above and then have two scripts:

/usr/bin/bogofilter_s
/usr/bin/bogofilter_u

each rule above then calls those scripts.

it’s a hack, but hey – they take a pipe to STDIN…

create your scripts with:

sudo echo “/usr/bin/bogofilter -s” > /usr/bin/bogofilter_s

sudo echo “/usr/bin/bogofilter -u” > /usr/bin/bogofilter_u

sudo chmod 755 /usr/bin/bogofilter_?

HTH

That worked guys!! Thanks.. I had been trying to get bogofilter to work for over 2 weeks. No one on IRC would help me. Thanks!

BTW, Jeremy’s method worked better than sheepshearer’s. Sheepshearers file was not seen as an executable script file.

Ubuntu Edgy, Evolution 2.8.1

Ningbojoe says:

Thanks for the heads up on this. I was wondering why my Spam detectors were not working and this helped me sort out this problem.
Thanks!!

Todd says:

Excellent article and comments. Thanks.

When I set up bogofilter on my system, I took some time to play around with several different ways of conducting the training (on the command line using bogofilter -s and -n commands.) At several points through each training procedure, I used the command “bogofilter -v

Todd says:

I found bogofilter needed training with examples of _both_ spam and non-spam to work effectively. Maybe this is why Evolution was ineffective: neglect of the non-spam part of the training.

Thanks for an excellent article.

likewhoa says:

>=evolution-2.8.2.x

supports bofofilter now :)
so no need for your custom pipetoprogram filter.

Surendra Singh says:

Important topic and many of you understand it well. However, for Evolution 2.8.1; I could not work out how to configure it based on info above and below the saved screen messages. I have installed bogfilter package already. A precise and stepwise instruction may enhance usefulness this article.

Thanks.

P. Stegmaier says:

I’ve got the most recent update with Ubuntu Precise and using Evolution. Still having problems with spam and wondering if your above hacks would still work on a newer version of Ubuntu and Evolution. Please give me precise directions I can use in the terminal to configure bogofilter and spamassasin. Thanks

Leave a Reply