D. J. Bernstein
Internet publication
FTP: File Transfer Protocol

Pipelining

RFC 959 generally requires that the client wait for an accepting or rejecting response before sending the next request.

Exceptions: RFC 959 allows the client to send a QUIT request, an ABOR request, or a STAT request without waiting for a response to a previous file transfer request (RETR, NLST, LIST, STOR, APPE, or STOU). Furthermore, ABOR and STAT have special semantics when they are received by the server before the server sends the previous response. However, there are serious problems with the semantics in each case; the client can more easily achieve the same results in other ways.

A far more important exception is pipelining, where the client sends several requests at once, and the server interprets the requests without regard to timing: i.e., as if each request had been sent after the response accepting or rejecting the previous request. This can save many round-trip times in FTP, noticeably speeding up clients.

Pipelining is prohibited by RFC 959, and it is in conflict with the special semantics of ABOR and STAT. (On the other hand, section 4.1.2.10 of RFC 1123 says ``a single READ from the control connection may include more than one command,'' and section 4.1.3.3 of RFC 1123 says that pipelining behavior is permitted for ABOR and STAT.) In practice, clients that attempt to pipeline may lose synchronization with the server, producing disastrous results:

However, clients may pipeline all requests to servers that declare support for pipelining. It is the client's responsibility to avoid deadlock. Of course, clients need to wait for responses to some requests, notably PASV, to decide what to do next. (Waiting for a PASV response does not waste any round-trip times: the subsequent RETR request cannot begin communicating data until after the client has reacted to the PASV response.)

For example, here is a maximally pipelined FTP file retrieval:

     Client makes TCP connection to server's FTP port.

     Server sends TCP SYN ACK.

     Client sends TCP ACK.

     Server: 220 Features: p .

     Client: USER anonymous
     Client: PASS joe@
     Client: TYPE I
     Client: PASV

     Server: 331 Please send PASS with your email address.
     Server: 230 Thanks.
     Server: 200 Good idea! I always recommend TYPE I.
     Server: 227 =1,2,3,4,50,103

     Client makes separate TCP connection to server's port 12903.

     Server sends TCP SYN ACK.

     Client sends TCP ACK.
     Client: RETR index.html
     Client: QUIT

     Server: 150 I see that file...
     Server sends index.html on separate TCP connection.
     Server closes separate TCP connection.
     Server: 226 RETR successful.
     Server: 221 Bye.
     Server closes FTP connection.

     Client closes separate TCP connection.
     Client closes FTP connection.
The client is occupied for only 5 round trip times, if index.html is short. This is not too bad, given FTP's horrible design. Note that the server must turn off Nagle's delay, i.e., enable TCP_NODELAY.

Asynchrononus QUIT

RFC 959 permits the client to send a QUIT request while a file transfer is in progress. The server is required to treat the QUIT request as if it had appeared after the response to the file transfer.

However, clients must not use this feature, except of course with pipelining servers. Microsoft FTP Service 4.0 simply discards the QUIT request, letting the client sit idle until it times out.

Asynchrononus ABOR

An ABOR request asks the server to stop any file transfer in progress. The client is required to send the TELNET string \377\364, and then send the TELNET string \377\362 with the TCP Urgent bit set, immediately before the ABOR request.

The client is permitted to send an ABOR request while waiting for a response to a file transfer request. The semantics of ABOR depend on ABOR's timing; ABOR is useless if the server does not notice the ABOR request immediately. However, the response to ABOR is still required to come after the response accepting or rejecting the file transfer request.

There is no reason for clients to use ABOR. The easiest way to abort the data transfer is to simply close the data connection.

Some servers do not support ABOR. I do not recommend that servers support ABOR; it is much more important for servers to support pipelining.

Servers that support ABOR typically respond with code 226 if a file transfer is in progress (after code 426 rejecting the file transfer request), or with code 225 if no file transfer is in progress.

Asynchrononus STAT

The client is permitted to send a STAT request while a file transfer is in progress. The client is required to send the TELNET string \377\364, and then send the TELNET string \377\362 with the TCP Urgent bit set, immediately before the STAT request.

Many servers accept asynchronous STATs. The response text shows the number of bytes transferred so far, in some human-readable format. This response comes before the response to the file transfer request. (If the client sends the STAT a moment before the server finishes the file transfer, the response will come after the response to the file transfer request. RFC 959 does not provide a bulletproof way for the client to tell which response belongs to the STAT and which response belongs to the file transfer.)

Clients must not attempt to use this feature. Microsoft FTP Service 4.0 simply discards the STAT request, producing a disastrous loss of synchronization. Clients that want to know the progress of a file transfer can monitor the transfer locally.

Of course, clients can send STAT at any moment to pipelining servers. The server will respond after it has finished responding to the previous request.