I've started adding historical notes, to combat fraudulent claims that smtpupd reflects implementor consensus.
John Klensin tossed the previous specifications into a single document, made quite a few changes that he wanted, and published the result as smtpupd-00. He slowly generated new drafts over the next several years. smtpupd-10 was released in February 1999. smtpupd-11 was released in March 2000. smtpupd-12 was released in July 2000.
Some specific examples of Klensin's editing process are described in detail on this page.
SHOULD send an address literal ... optionally followed by
information that will help to identify the client system
in
HELO.
The address literal is fine;
a bracketed IP address is a valid RFC 821 domain,
and any server that rejected it
would be violating RFC 1123, section 5.2.5.
However,
``information that will help to identify the client system''
is a Klensin invention
that creates new interoperability problems.
Clients that send such information
will find themselves completely unable to send mail to some hosts.
smtpupd-10 says that servers ``MUST NOT recognize'' anything other than \015\012 as a line terminator. In reality, for interoperability with existing clients, servers recognize \012 as a line terminator for client requests. Clients do not (and, in practice, cannot) use \012 inside requests except as a line terminator.
smtpupd-10 says that SMTP servers ``MUST reject'' requests containing underscores or other ``illegal character codes.'' In reality, for interoperability with existing clients, servers accept (for example) EHLO requests containing underscores.
smtpupd-10 has an extended Received-line syntax that allows fractional seconds. This breaks some Received-line parsers. This problem was fixed in smtpupd-12.
There are dozens of other interoperability issues where smtpupd-10 does not give adequate warning to future implementors.
Only resolvable, fully-qualified, domain names (FQDNs) are
permitted when domain names are used in SMTP.
This prohibits the behavior of many existing clients:
for example,
a client relaying a message to a smarthost
is violating the protocol if the user typed an address incorrectly.
Obeying Klensin's requirement means checking every address in DNS.
This is an unacceptable policy violation at some corporate sites,
an unacceptable expense for many dumb clients,
and an unacceptable obstacle for the vast majority of users
when their ISPs are temporarily unable to reach the rest of the Internet.
The requirement also prohibits the standard use of domain literals
as specified in RFC 1123.
Modern MTAs send ``double-bounce'' notifications to the local postmaster when a bounce message is undeliverable. smtpupd-10 has three requirements saying that this ``MUST NOT'' be done; only one of the requirements has a relevant exception.
smtpupd-10 says that an SMTP client ``MUST NOT intentionally close the transmission channel until it sends a QUIT command.'' In reality, if an SMTP client is in the middle of sending a message, and is unable to read the next portion of the message from disk, it ruthlessly closes the connection. Sending QUIT in this situation means sending a corrupted message.
smtpupd-10 says that VRFY ``MUST be listed ... in an EHLO response'' if it is supported. In reality, as of 1999, approximately 90% of the servers that support both VRFY and EHLO do not list VRFY in response to EHLO. If a client wants to use VRFY for some reason, it simply tries the VRFY command, as per RFC 821; paying attention to the EHLO response would be foolish. There's no reason to require an announcement.
smtpupd-10 says that SMTP clients ``MUST NOT'' send message data unless the response to DATA is 354. This prohibits the behavior of many existing clients: if a server (in violation of RFC 821) sends (e.g.) a 387 reply, these clients will (as encouraged by RFC 821) treat 387 the same way as 354. The correct rule is that clients must not send message data if the reply begins with 4 or 5.
smtpupd-10 says that servers ``MUST support the EHLO command even if they do not implement any specific extensions.'' In reality, servers that do not support EHLO are still able to receive mail. If a server does not support any extensions, its only benefit from supporting EHLO is to save a small amount of time for clients that start with EHLO. This is enough of a benefit for me to recommend that all servers support EHLO, but there is no interoperability excuse for requiring it.
smtpupd-10 says
Servers MUST NOT return the extended EHLO-style response to a
HELO command.
It is unclear what this means.
Some obvious interpretations prohibit the behavior of existing hosts.
smtpupd-10 says
Whatever mechanisms are used, servers MUST contain provisions
for detecting and stopping trivial loops.
It is horrendously unclear what this means.
Some obvious interpretations prohibit the behavior of existing hosts.
SMTP clients and servers SHOULD NOT copy the full set of RCPT TO
command arguments into the headers, either as part of trace
headers or as informational or private-extension headers.
smtpupd-10 says that this ``requirement'' should be violated
``only under exceptional and well-understood circumstances'';
smtpupd-10 claims that MTAs that use Delivered-To
may not ``interoperate properly'' unless ``great care is taken.''
This is garbage.
Klensin's alleged justification---namely,
that such copying ``defeats the purpose''
of mailing lists and blind carbon copies---is
completely incorrect for a message being
delivered to one recipient at a time.
The normal behavior of part-time dialup hosts and dumb clients is to connect to a relay identified by name (or sometimes IP address). smtpupd-10 says that clients that use relays ``SHOULD'' instead perform MX lookups on the relay name. smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that clients using the normal procedure may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. There is no reason to demand that these clients use MX lookups. This problem seems to have been fixed in smtpupd-11.
Historical notes: The previous specifications don't encourage useless EHLOs.
Klensin's smtpupd-00 says ``SHOULD preferentially utilize EHLO''; Klensin later admitted that this was his idea. In February 1996, and again in May 1996, I asked why. ``A client that doesn't implement any of the extensions can obviously just send HELO,'' Keith Moore said on 24 May 1996. Eric Thomas said he agreed on 26 May 1996.
Klensin's smtpupd-07 says ``MUST preferentially utilize EHLO rather than HELO.'' I objected to this in June 1998. My objection was echoed by Philip Hazel on 28 July 1998. On 4 August 1998, Chris Newman issued a ``Call for Concensus'' [sic] on the screwy idea that ``clients are required to support EHLO, but are not required to preferentially use it.'' My objection was then echoed by Robert Elz on 5 August 1998 and John Myers on 6 August 1998.
The issue was finally raised at the August 1998 DRUMS meeting. Barry Leiba's notes say
Consensus is that "SHOULD use EHLO" be dropped, and a client that doesn't need extensions may use EHLO or HELO at the client's discretion.But the document wasn't changed accordingly.
In January 1999, I objected to the text ``SHOULD preferentially utilize EHLO rather than HELO'' in smtpupd-09:
The poor structure of smtpupd meant that HELO was discouraged in many different locations in the document. Apparently the editor managed to fix a few of these locations but missed several more. I find it disturbing that the editor is so careless in making changes that go against his minority views. I'm willing to make a thorough list of HELO/EHLO references that need fixing, but before I go to the effort I'd like some assurance that the editor will do his job honestly.Klensin responded on 30 January 1999:
Dan, if there are additional places where the EHLO preferences should be toned down as indicated by the minutes, please tell me where they are. If I can get a precise list, especially if it comes with a minimum of gratituous noise, they will be fixed; any omissions are errors, not intentional. This situation is the reason I have asked several times that people provide me with proposed text for changes, not just principles. I will try to fix things according to principles if that is all I get, but I do make mistakes.I took Klensin at his word, and sent out a list of twenty corrections to the document. But the document wasn't changed accordingly.
I announced this web page, again objecting to Klensin's text, in May 1999. The document still wasn't changed.
Chris Newman sent out a Last Call for smtpupd-12 in July 2000. I objected once again. Dave Crocker claimed, falsely, that there was consensus for Klensin's text.
Chris Newman stated that one section of smtpupd-12 ``complies with WG rough concensus [sic]''; he ignored the fact that another section obviously didn't. Then he said
I searched my archives for messages with "HELO" in the subject line. According to such messages, one person has objected to this text on at least two occasions. In all cases there was no support expressed for the objection. The chair concludes there is rough concensus [sic] that the current text is adequate for RFC publication and the issue is permanently closed.What an incredible display of incompetence!
smtpupd-10 says that SMTP servers ``SHOULD support VRFY''; this means ``at least recognition of local mailboxes.'' smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that SMTP servers that always respond 252 to VRFY may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. In reality, VRFY is obsolete. Thousands of the most heavily used mail servers on the Internet always respond 252 to VRFY.
smtpupd-10 says that an SMTP client ``SHOULD'' wait until it receives the reply to QUIT before closing a connection. smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that clients closing the connection immediately after sending QUIT may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. In reality, thousands of SMTP clients close the connection immediately after sending QUIT. There is no reason to wait for the response.
smtpupd-10 says that an SMTP server ``SHOULD attempt to send a line containing a 421 response code'' if it is shut down by the system administrator. smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that servers that simply close the connection may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. In reality, connections are abruptly dropped for a wide variety of reasons, including server shutdowns.
smtpupd-10 says that every ``SMTP-capable host SHOULD support both the alias and the list models of address expansion for multiple delivery.'' smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that servers that don't support mailing lists may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. In reality, firewalls and POP toasters do not, and should not, support mailing lists.
smtpupd-10 says that SMTP servers ``SHOULD reject'' parameters in RSET, DATA, or QUIT. smtpupd-10 says that this ``requirement'' should be violated ``only under exceptional and well-understood circumstances''; smtpupd-10 claims that servers that ignore such parameters may not ``interoperate properly'' unless ``great care is taken.'' This is garbage. In reality, thousands of SMTP servers ignore such parameters.
smtpupd-10 says that all ``fully-capable SMTP implementations'' are ``expected to support'' various client features. This is garbage. In reality, firewalls and POP toasters often run pure SMTP servers. They do not send mail through SMTP, and they are not expected to support any SMTP client features. Installing unused code is a violation of some corporate security policies.
smtpupd-10 claims that, ``in most cases,'' SMTP clients that send all mail through a relay host neglect to retry delivery after failures. This might be true for some dumb clients but it's certainly not true for serious MTAs such as sendmail.
smtpupd-10 defines ``relays'' as receiving mail by SMTP and forwarding mail by SMTP. In reality, many relays support other protocols for transferring Internet mail messages, such as QMTP. (These relays generally do not change message formats, so they do not fit smtpupd-10's ``gateway'' requirements.)
smtpupd-10 says that ``the [message] body, if structured, is defined according to MIME.'' In reality, there are many common non-MIME body structures.
smtpupd-10 begins the same way as the original SMTP specification, RFC 821, written in 1982:
The objective of the Simple Mail Transfer Protocol (SMTP) is to
transfer mail reliably and efficiently.
Unfortunately, SMTP does not achieve this objective.
It is painfully slow;
more importantly,
its design has led to a string of interoperability disasters.
smtpupd-10 requires that years contain at most four digits. In reality, there will be a year 10000. This problem was fixed in smtpupd-12.