Brennan’s Blog

Email has become too unreliable. Over a year ago I started to forward my email into a Gmail account to get a great webmail interface and to shield my inbox from the increasing amount of spam that I was not able to avoid. It seemed to help at first, but with every method employed to block the unwanted messages, the spammers would develop a way to overcome it. Initially you could run an analysis of the words used in an email to create a spam ranking. The male enhancement and stock trading terms were commonly used in spam messages, but gradually the spammers became more clever. The would change the letters to symbols, like an @ instead of an A. But the word analysis eventually adapted and learned to block them.

Now the latest measure used by spammers is to fill the message with a passage from a book, or something pulled randomly from a website through a search engine. By filling the message with good words, the bad words were watered down and the emails would get through. But as you mark those clever messages as spam you increase your chances that genuine messages would be blocked.

What do we do now? Beyond word analysis, efforts like Sender ID have been heavily debated. Alternatives such as SPF are not being fully embraced universally due to concerns that it does not completely solve the problem. One option which has been effective is Real-time Blacks Lists, or RBL.

Services such as OpenRBL.org maintain a dynamic list of IP addresses through a DNS lookup for offenders caught through the use of spam traps. It is a clever solution which has worked well. These spam traps are basically unused email addresses which are not published or in regular use, but reachable through methods used by spammers. I have been running spam traps by placing mailto: links on web pages within HTML comment tags so that it is not visible by a person, but when a spammer scans the page they will pick up those addresses. And when those addresses receive messages, I know it has to be from a spammer. This is one technique used to fill spam traps with offending IP addresses.

While I have used Gmail, I have been unable to send email due to a Gmail server being listed momentarily in the RBL. This can happen due to a user sending mail which lands in spam trap. The blacklists allow you to remove the IP address easily enough, but for a while your host will be blocked. This problem for Gmail will not go away. Google explains that some anti-spam services allow services like Gmail to include the IP address of the actual user so that the user is blocked instead of the Gmail server, but due to their privacy policy on outgoing mail they are not placing that personal information in outgoing mail. And until all mail servers explicitly always allow Gmail through to their users, this problem will remain.

The RBL system is also not going to work for much longer anyway. A new technique to send out millions of spam messages has emerged to combat the effective RBL technique. Due to insecure home computers constantly connected to the internet via broadband connections it is possible for a spam organization to take over a computer and send just 10 emails out, so as to not get that IP address listed with an RBL service. It works because they can compromise thousands of computers which are not sufficiently protected with firewalls and antivirus software.

It seems there is no path back to reliable email. It was such an elegant message delivery system too, with multiple levels of redundancy. If you read up on how it works you will see what I mean. The feature I like most is how mail (SMTP) uses mailbox records (mx) in DNS to ensure reliable delivery. You can configure multiple fallback servers to be used if the primary mail server is unavailable. Unfortunately there was no mechanism built for SMTP to authenticate incoming messages. And despite all of our efforts, placing band-aids onto SMTP is not correcting that difficiency. Email is dead. What's next?