Dealing With Junk Email

Session given at DECUS November 1997
Ned Freed

Overview of Talk

• What is junk email?
• The legal situation surrounding junk email.
• Relay blocking.
• System level spam blocking.
• User level spam blocking and the Sieve language.

• One person's junk mail may be another's vital communication.
• Characterizing it as unsolicited commercial email (UCE) ignores mailings from religious groups, self-help groups, etc.
• Characterizing it as unsolicited bulk mail (UBE) ignores small, targeted mailings.
• Call it junk email or spam and always keep in mind that this may mean different things to different people.

• There are no laws against junk email in the US at present.
• All sorts of legislation is being proposed:
  • Content labelling in the message header.
  • Subject line labelling.
  • Application of junk FAX laws to email.
  • New email-specific prohibitions.
• The ability of US-only law to control spam is unclear.
• There could be a good law tomorrow or a terrible law may be years away.
• The EMA does a good job of tracking things in this area.

• Preventing your site from being used as a relay is the first, best, and most important thing to do.
• This helps others more than it helps you, but we help each other by doing it.
• Instructions for doing spam blocking are available for most popular MTAs:
  • PMDF
  • Sendmail
  • If your MTA can't do relay blocking either front end or replace it with one that can.
  Relay Blocking Fundamentals
  
  
  • Basic concept is simple:
    • Divide sources/destinations of mail into us and them.
    • Don't let our systems be used to transfer mail from them to them.
  • As always, the devil is in the details.
    • Some setups make it hard to tell us from them.
    • Whenever possible use IP address to make the distinction.
    • If using IP addresses isn't possible use domain names derived from the IP address.
    • Some people believe an additional check to make sure the domain name then maps back to the IP addresses is also warranted.
    • Make sure relay blocking doesn't interfere with legitimate use of aliases and mailing lists, or mail sending using POP/IMAP.
  System Level Spam Blocking
  
  

  There are many approaches to system level spam blocking.

  • Origin system blocking. Generally doesn't work because of the prevalence of relays.
    • Blacklist blocking. Somewhat effective, but spammers change addresses and names frequently. Blocking an innocent relay site can also cause more problems than it solves.
    • Domain name blocking. Blocking systems that claim a domain name that doesn't match their IP address doesn't work well -- too many legitimate setups do this.
    • IP address blocking. Blocking connections from IP addresses that don't have a domain name assigned doesn't work well either -- again, too many legitimate users.
  • Origin (MAIL FROM) address blocking works a little better:
    • Blacklist blocking. Some spammers use the same origin repeatedly so this actually works well in isolated cases.
    • Source route blocking. Largely ineffective; most spam doesn't use source routes and quite a bit of legitimate mail does.
    • Percent hack blocking. Entirely ineffective and many legitimate systems use percent hacks.
    • DNS domain blocking. The idea is to check the DNS to see if the domain has A or MX records. Somewhat effective in the past; spammers now routinely use stolen domains.
  • Content blocking at the system level. Rarely done.

  Bottom line is that system level approaches either only work in the short term or require constant maintenance. Use them now but be prepared to give them up when they stop working.
  
  User Level Blocking
  
  

  • Remember that spam is in the eye of the beholder. A spam filter that seems to you to work perfectly may be blocking legitimate email according to someone else.
  • Users know what they want and, if allowed, will set up their own rules.
  • The right solution is to provide users with the tools they need to set up their own spam blocks.
  • What is needed is a standardized, easy to use language for specifying filtering rules, and GUI tools for creating scripts in this language.
  Sieve
  
  
  • Filtering language for email.
  • Intended to become an IETF standard. see Internet Draft for Sieve
  • Operates during final delivery phase of message processing.
  • Filter can specify that messages be:
    • Delivered normally (keep).
    • Silently deleted (discard).
    • Returned to sender (bounce).
    • Forwarded to another address (forward).
    • Filed in an alternate folder (fileinto).
    • Generate an automatic reply (reply).
  • No Turing complete structures in the core language.
  • Core language conditional structure is \verb+if condition {...} else {...}+.
  • Contains, matches, and is are the conditional operators. Matches implements glob-style matching.
  • Any-of and all-of> allow logical combinations.
  • Header and exists allow examination of message headers (including return-path for MAIL FROM information).
  • Requires allows declaration of extensions required by a given filter.
  • Stop used to stop processing.
  • Comments indicated by sharp sign.
  Sieve Examples
  
  

    if any-of (
         header ("from") contains
           ("bart", "homer",
            "smithers", "burns", "lisa"),
         header ("subject") contains ("URGENT")) {
      keep;
    } else {
      reply text: # multi-line message here:
        You are not one of the people I regularly
        correspond with.  I have deleted your message
        due to the large volume of email I regularly
        receive.  If you feel that you need to speak
        with me directly, and cannot find your answer
        in my web pages, please send mail with the
        word "URGENT" in the subject line.  Thank you
        for your time.
        .
        ;
    }
  
   if size over 100K { # this is a comment
                  discard;
               }
  
   if header "from" contains "coyote" {
                 discard;
              } else if  header ("subject") contains
                         ("$$$") {
                 discard;
              } else fileinto "INBOX";
  
   if not exists ("From" "Date") {
                    # Date: is mandatory
                    discard;
                 }
  
   if header "from" contains "coyote@znic.net" {
     bounce "I am not taking mail from you,
             and I don't want
             your birdseed, either!";
   }
  
    if header ("from") matches
       ("*boss@*frobnitzm.edu*") {
      forward "pleeb@xanadu.wv.us";
      stop;
    } else {
      reply text:
        I'm on vacation and not taking any messages;
        try after Sunday.  I have thrown your
        message out. Please resend it later.
        .
      ; discard;
    }
  
   require "dwim";
   if  header   ("subject")   contains-nocase (
     "the secret message") {
                dwim blurdybloop body;
               } stop