Cryptolisting
Faster more secure email is possible

Every new year, I try to reduce the number of inputs I get, cleaning out my mailboxes, putting everything unsorted into an archive folder, and so on. So for the past three days I’ve been getting off of mailing lists I don’t read, writing email filters for stuff I do read, and trying to get the avalanche of input I cope with down to a manageable level again.

After being on the Internet almost continuously for 20+ years, I get a lot of spam. I mean, a LOT of spam. One account, that I retired 10 years ago and haven’t used since, has got over 20,000 spams in 2 years. It averages about 3 spam attempts per minute actually, and about 70/day were getting through.

Worse, my main email accounts started to get really cluttered with spam early last year, and I didn’t know why. After a while the problem grew annoying enough to where I moved back to the gmail accounts I use, which were doing a lot better job at filtering out the spam.

I HATE using gmail for email for a lot of reasons. I LIKE email fully integrated with my editor and like reading and sending email to be FAST…

So I decided to rethink my email setup. I’d put off greylisting for a long time - I disliked the idea of delaying an email for an hour, arbitrarily, just to deter “drive by” spam attacks. One of my favorite parts of email is getting email from strangers.

But after trying to figure out what had gone so wrong with my spamassassin config I got fed up, so I put in place a postfix policy daemon called greyfix on my main email server. It helped, quite a lot, but spam was still getting through on the secondary mx exchanger. Greylisting something twice is not a good idea (now there are TWO places where a given email will be arbitrarily delayed), but the only other choice was to go down to one email server and I wasn’t going to do that, so….

Greyfix went on the secondary.

crickets

Wow. The silence of the spams. I have not seen ONE spam get through all day. The mail.log file that used to scroll by too rapidly to read now ambles by at a reasonable pace.

I also instituted a few other things that have (sigh) temporarily made it impossible for me to send mail to - amusingly enough - comcast, my local internet provider. It turns out that they (and a few others I’ve checked), only accept mail from domains inside their servers that are actually the domain name of the host. While a good security measure, it messes me up as mail from taht.net doesn’t work, while teklibre.org does, even though both domains are me…

Moving along…

I was still getting about 4 spam attempts per minute from all over the world on the main server, and that was pretty annoying. It’s a wee, little arm box, and doesn’t LIKE being tapped on so often… So I said to heck with it. I created a whitelist of servers I exchanged mail with regularly - and blocked port 25 on any but that extremely small list of ips on the main server. I figure that - since greylisting works - I might as well force most connections out to my secondary server anyway.

In other words - my main email server has gone “dark” and provides NO opportunity for random spammers - or normal mail exchangers - to get in. This is kind of similar to how many (normal) sysadmins put a main, massively mail-filtering mail server on the outside (DMZ) and the “normal” mailserver inside the firewall, except in my case I’m still allowing some mail servers in and also sending mail directly from the internal server. This makes mail OUT a bit faster, and mail in - mildly slower.

This is a variant of nolisting.

Wow… That increased the amount of legitimate email I get on the main server to 100%. I didn’t have to even look at the logfile anymore, except for errors.

Update: Dang it! Servers that exchange email with me that ALSO implement greylisting temporarily reject mail, too. My primary mailserver gives up after too short a while and forwards the mail to the secondary - which does the right thing and ultimately gets the mail through - and my primary server never gets greylisted by the other server! This means that ALL mails to a greylisting server I try to send from here are now actually getting through slower… I guess I have to think this part through more.

Flush with these successes, I said to myself, “Self, what REALLY bugs you about email that now that you are down and dirty inside your mail system and don’t want to go here again for a couple more years?”

Well, it bugs me that email from my desktop to the server is covered over a secure connection, but interchange between providers - between you and me - is not.

It doesn’t make much sense to me that 12+ years after STARTTLS was invented (and SSL security/certs came much earlier) and everybody sending mail to their email provider uses a secure channel, that nearly nobody - certainly not the big email services - uses cryptography between their services - even when offered!!??

We’ve all come to accept that email is basically insecure. It would certainly help the spam problem, however, if certificates were required. It would establish a higher bar as a line of defense… And people exchanging mail would get a slightly higher level of security overall.

I started digging into the relevant rfcs and found out why that few use STARTTLS for ALL email exchanges is basically an historical accident. It was too buggy in 1999 to use.

Not for the first time I wished Jon Postel had remained alive to guide email to a better landing…

I figure if enough people care, something will happen.

So I dug into that freshly installed greyfix code and figured out that newer versions of postfix actually supply whether the email exchange was encrypted or not.

So I hacked on greyfix for a while, to make it do “cryptolisting”.

What’s Cryptolisting?

It’s not that much different than normal greylisting, except that:

1) new people that send email via an encrypted channel get greylisted for a MUCH shorter time. I figure this is a good bet. Anybody that uses STARTTLS for email interchange has more of a clue.

2) I figure spam sent via encrypted channels is close to nil, so I can further whitelist my internal/main mail server’s acceptable IP addresses with the database I get from the secondary mail server. This will speed up everybody that I get encrypted connections from - a positive feedback loop for people that bother implementing STARTTLS on their outside mail exchanger.

3) I put in a custom header field that I hope to convince gnus to inspect so I can colorize incoming email as to it’s transport security on my end.

4) It puts in an informative delay message in the temporary reject, thus advertising it’s own existence to other sysadmins and giving me room to write a manifesto about getting more crypto in our email… whenever I get around to it.

I’ve cleaned up the code a little, pushed it out to git hub, and created a web site for cryptolisting, so if you want to fool with it, go ahead.

Doing a secondary MX exchanger is a bit trickier than I’d like.

Further cleanup

Now I’m off to clear out my main mailboxes in time for the new year. 3000+ categorized emails left to go. I’ve got a whole bunch of other spam stoppers to put in place AND to figure out how to get mail to comcast again, but I’m really enjoying the silence of the spams… My mail is so quiet and so un-full of distractions…

EHLO? Is there anybody out there??

February 10, 2011
1337 words