SMTP and Firewall Security

It is axiomatic that security analysis has to be contextual. One cannot compare two solutions and say that one of them is "more secure" or "less secure" without knowing the context in which the solutions are being used. For example, we often talk about how having plaintext passwords on the wire is "less secure" than using schemes like, say, APOP or CRAM-MD5. And the conclusion people often take from this is that CRAM-MD5 or APOP are superior in all cases. But in fact this is false, because this assumes a very specific sort of context: Specifically, that unencrypted channels are being used where eavesdropping is possible, and that servers are basically secure against attack. If, on the other hand, a secure channel is used (e.g. SSL or TLS), then there's no exposure of the plaintext password to an eavesdropper, and hence there may be no real advantage of using APOP or CRAM-MD5, since anything that compromises channel security would totally compromise all forms of security, making passwords irrelevant. And more to the point, if an attack on the server is possible, in the case of both APOP and CRAM-MD5 such an attack will expose everyone's password. So, if SSL is being used and server vulnerability is a real issue then plaintext passwords may in fact be "more secure" than APOP or SSL. The context here is crucial in determining which solution is "more secure".

The same sort of reasoning must be applied when someone decides whether or not to restrict the command set of SMTP. So here are some example cases and some corresponding analyses for the specific case of SMTP connections from outside a firewall to inside a firewall. The cases are categorized in terms of the software inside the firewall as well as whether or not there's an intermediate "gateway" system positioned just outside the firewall:

  1. V5 sendmail inside the firewall accepting connections from anywhere. This version of sendmail has known bugs which can be exploited by using nonstandard commands. There are also bugs that can be exploited by using standard commands with weird arguments. Extended SMTP is not supported. As such, a firewall needs to not only severely restrict the command set but also the arguments commands allow. Any setup that does not implement such restrictions will almost certainly be "less secure" than one which does, and there is no real disadvantage to implementing such restrictions in this case.
  1. Current V8 version of sendmail inside the firewall accepting connections from anywhere. This version of sendmail has no known security holes, but given sendmail's long and ongoing history of security holes turning up it would be unreasonable to suppose that no additional bugs will be found and exploited. On the other hand, this version of sendmail supports several SMTP extensions, and the NOTARY extension in particular provides facilities for tracking messages, and being able to track messages more accurately increases your ability to counter various forms of attack and thus increases security. So you end up with a tradeoff -- a firewall that eliminates extended SMTP commands is neither "more secure" nor "less secure" in this situation. A firewall that filters command arguments and understands extended SMTP, on the other hand, probably is "more secure" in all cases than one that doesn't filter at all.

  2. PMDF inside the firewall configured in firewall mode accepting connections from anywhere. Unlike sendmail, PMDF security is achieved by design and implementation, that is, security and reliability were paramount issues considered during the PMDF design and coding process. Specifically, PMDF is coded using range-checked operations on counted strings rather than unchecked null terminated strings, unsafe constructs like unconstrained character pointers that cannot be range-checked reliably are avoided, compiler options to generate runtime range checking on arrays are left enabled, components either run in complete isolation or run with restricted privileges, so that even if bugs do turn up exploiting them cannot compromise the operating system, and so on. PMDF has also undergone extensive security analysis by outside tests on more than one occasion. And perhaps most important, in the 15-year history of PMDF I don't believe there has ever been a security hole that could be exploited through a remote SMTP connection. (This is in sharp contrast to sendmail's beleaguered history. I also doubt very much that most firewalls products can make comparable claims.) As such, the chances that the PMDF system will be compromised through the SMTP port are quite small, and more to the point the chances that employing a restricted SMTP command set and SMTP argument checking will improve PMDF security are nonexistent. Why is this so? Because this sort of protection is geared towards sendmail, not PMDF. And our experience has been that the rare PMDF bug that manifests as an array index error or access violation, which would logically be a strict superset of actually exploitable bugs, haven't been affected or blocked at all by command set restrictions. So the conclusion is clear: In this case a setup that uses a command-restricting firewall is "less secure" than one which does not, because not only does former lose valuable message tracking information which has security implications, the latter doesn't actually increase security in any demonstrable sense.

  3. PMDF on the inside, PMDF on the outside. This is the case that started this discussion. The existance of two systems that have to be compromised of course raises the bar substantially on being able to exploit any security holes that actually exist in the product. But the need to use extended SMTP across the firewall still exists, although logging on the external PMDF system coupled with generation of notifications there offer an somewhat better alternative to losing this information outright. So these additional considerations end up being a wash and so this case pretty much reducing to (3).

There are lots more combinations and permutations that can be analyzed, of course. The point, however, is clear: Sometimes a firewall that restricts the SMTP command set is a good thing and gives you a "more secure" setup, and sometimes it does the exact opposite.


Search: