Geeky Bob

For this installment in my series about FTP Clients, I'd like to take a look at the MOVEit Freely Command-Line Secure FTP Client ("FTPS.EXE") from Ipswitch. For this blog post I used MOVEit Freely FTP Client version 5.0.0.0, and it is available from one of the following URLs:

http://www.ipswitchft.com/products/moveit/client/freely/

http://www.ipswitchft.com/Products/MOVEitFreely/

At the time of this blog post, Ipswitch is providing the MOVEit Freely FTP client for free, although you are required to fill out a registration page with a short questionnaire. For more information on the license for the MOVEit Freely command-line FTP client, please see Ipswitch's web site.

If you're like me and you like to script a lot of batch jobs on your servers, the MOVEit Freely command-line FTP client can be quite handy. The command set for the MOVEit Freely FTP client is a greatly-enhanced superset of the commands that are available with the command-line FTP.EXE client that is built-in to Windows, with added features that make additional functionality possible, such as SSL, passive FTP, resumable downloads, etc. There is a manual available with the MOVEit Freely FTP client, and I highly recommend using the manual as a reference when writing automation scripts because there are a lot of options that are available to you.

Active and Passive FTP

One of the great things about the MOVEit Freely command-line FTP client is the ability to use either Passive or Active connections, and you can switch between the two connection types using the "passive" command in the FTP session. This helps immensely when working with firewalls and such. The following example shows what that might look like:

CMD>ftps.exe ftp.example.com 220 Microsoft FTP Service Connected to ftp.example.com. User: administrator 331 Password required for administrator. Password: ******** 230 User logged in. ftp> passive Passive mode On . ftp> put foobar.txt 227 Entering Passive Mode (192,168,0,1,224,39). 150 Opening ASCII mode data connection. 226 Transfer complete. ftp: 8 bytes sent in 0.06Seconds 0.13Kbytes/sec. ftp> passive Passive mode Off . ftp> put foobar.txt 200 PORT command successful. 125 Data connection already open; Transfer starting. 226 Transfer complete. ftp: 8 bytes sent in 0.01Seconds 0.78Kbytes/sec. ftp> bye 221 Goodbye. CMD>

Using FTP over SSL (FTPS)

The MOVEit Freely command-line FTP client supports both Implicit and Explicit FTPS, so the choice is up to you which one to use, but I generally use Explicit FTPS since Implicit FTPS should be considered obsolete. The SSL mode is specified using the "-e:" parameter on the command-line, and the 5.0.0.0 version of the MOVEit Freely command-line FTP supports the following values for that parameter:

Parameter	FTPS Mode	Description
off	n/a	Specifies that no encryption will be used on either the control channel or data channel. Note: This is the default behavior.
on	Explicit	Specifies that both the control channel and data channel will use encryption over an explicit FTPS connection. Notes: This uses the AUTH TLS, PBSZ 0, and PROT P commands when establishing a connection. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 below.)
on-ccc	Explicit	Specifies the control channel will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data channel connections will still be encrypted. Notes: This uses the AUTH TLS, PBSZ 0, and PROT P commands to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated. (See Note 2 below.) The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.
tls-p	Explicit	Specifies that both control and data channel will use encryption over an explicit FTPS connection. Notes: This uses the AUTH TLS and PROT P commands when establishing a connection. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)
tls-c	Explicit	Specifies that only the control connection will use encryption over an explicit FTPS connection. Data channel connections will be unencrypted. Notes: This uses only the AUTH TLS to enable encryption when establishing a connection. You must manually send a PBSZ command before you can use the "prot on" and "prot off" to specify whether encryption will be used. (See Note 1 and Note 3 below.)
tls-c-ccc	Explicit	Specifies that only the control connection will use encryption over an explicit FTPS connection during login, but the control channel will switch to unencrypted after a login has been established. Data connections will be unencrypted. Notes: This uses only the AUTH TLS to enable encryption when establishing a connection, then uses the CCC command after the username and password are successfully negotiated. The USER and PASS commands are the only commands that will be encrypted; all other FTP commands are unencrypted.
implicit	Implicit	Specifies that both the control channel and data channel will use encryption over an implicit FTPS connection, which can only be on port 990 for the FTP7 service. Notes: The implicit FTPS connection will encrypt both the control channel and data channel without the use of an AUTH command. You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)
implicit-ccc	Implicit	Specifies that the control channel would use encryption over an implicit FTPS connection during login and switch the control channel to unencrypted after login, but this is not supported in FTP7. The implicit FTPS connection will succeed, but the FTP7 service will return an error when the CCC command is sent. You can ignore the error and continue to use the session. Notes: Implicit FTPS connections require encryption for the command channel. (See Note 4 below.) You can use "prot on" and "prot off" to specify whether encryption will be used. (See Note 3 below.)

The following notes should be considered:

1. Encryption of the data connection can be changed during the session with MOVEit Freely's "prot" command. You can use the "prot on" to specify that data connections will be encrypted, and "prot off" to specify that data connections will be unencrypted; these commands will respectively send the PROT P and PROT C commands over FTP.
2. The "on-ccc", "tls-c-ccc", and "implicit-ccc" parameters are useful with firewalls that inspect FTP traffic. Switching the control channel back to unencrypted allows the firewall to inspect and possibly modify the FTP commands. For example, firewalls that are performing Network Address Translation (NAT) may need to modify the PORT and PASV commands.
3. "RFC 2228 - FTP Security Extensions" states that FTP clients are required to send an FTP PBSZ command before sending an FTP PROT command, and unfortunately the MOVEit Freely 5.0.0.0 FTP client does not send this command, so you get a "503 Bad sequence of commands" error. You can work around this by issuing a literal command to the server using the FTP client's "quote" command, which appears to work. See the following example for more information. The following example shows what that might look like:

   Note: I turned on debugging for this example with the "-d" option so you can see the sequence of commands.

   CMD>ftps.exe -e:tls-c -d ftp.example.com 220 Microsoft FTP Service ---> AUTH TLS 234 AUTH command ok. Expecting TLS Negotiation. Connected to ftp.example.com. User: administrator ---> USER administrator 331 Password required for administrator. Password: ******** ---> PASS (hidden) 230 User logged in. ---> SYST 215 Windows_NT ftp> prot on ---> PROT P 503 Bad sequence of commands. Data connections will still NOT be encrypted ftp> quot PBSZ 0 ---> PBSZ 0 200 PBSZ command successful. ftp> prot on ---> PROT P 200 PROT command successful. Data connections will be encrypted ftp> bye ---> QUIT 221 Goodbye. CMD>

4. The FTP7 service treats implicit FTPS connections as though the SSL policy for the control is set to "Require".

One last note about FTPS, if you are using a certificate with trust issues, you will see the following prompt displayed:

You can get around this certificate prompt when writing scripts by using the "-z" switch. The following example shows what that might look like:

Note: For this example I bypassed a certificate prompt with the "-z" switch, and I specified passive FTP with the "passive" command.

CMD>ftps.exe -z -e:on ftp.example.com 220 Microsoft FTP Service 234 AUTH command ok. Expecting TLS Negotiation. Connected to ftp.example.com. User: administrator 331 Password required for administrator. Password: ******** 230 User logged in. 200 PBSZ command successful. 200 PROT command successful. 215 Windows_NT ftp> passive Passive mode On . ftp> ls -l 227 Entering Passive Mode (192,168,0,1,224,97). 150 Opening ASCII mode data connection. 03-10-08 10:41AM <DIR> App_Data 09-04-08 11:41AM <DIR> aspnet_client 09-04-08 11:41AM <DIR> bin 12-17-02 11:47AM 2360 default.aspx ftp: 128 bytes received in 0.03Seconds 83.25Kbytes/sec. 226 Transfer complete. ftp> bye 221 Goodbye. CMD>

Using FTP Virtual Hosts

Since everything is happening from a command-line, you can use both FTP7's Virtual Hosts and the actual FTP HOST command. Once again, see my Virtual Hosts and Host Names in FTP7 blog post for more information about FTP Virtual Host Names and FTP True Host Names, and see https://datatracker.ietf.org/drafts/draft-hethmon-mcmurray-ftp-hosts/ for more information about status of the FTP HOST command.

In any event, FTP7 virtual hosts are supported by using the "ftp.example.com|username" syntax when specifying your username, and when you connect to the FTP7 server it will route your requests to the correct FTP virtual host site. The following example shows what that might look like:

CMD>ftps.exe ftp.example.com 220 Microsoft FTP Service Connected to ftp.example.com. User: ftp.contoso.com|administrator 331 Password required for ftp.contoso.com|administrator. Password: ******** 230-Directory has 104,857,600 bytes of disk space available. 230 User logged in. ftp> bye 221 Goodbye. CMD>

True FTP hosts can be used by specifying the FTP HOST command before the client sends the USER and PASS credentials. This is accomplished in two parts:

1. You need to suppress the automatic username prompt MOVEit client by using the "-n" switch on the command-line.
2. You need to specify the host name using the MOVEit client's "quote" command, which allows you to send custom FTP commands. The syntax for this would be "quote HOST ftp.example.com".

The following example shows what that might look like:

CMD>ftps.exe -n ftp.example.com 220 Microsoft FTP Service Connected to ftp.example.com. ftp> quote HOST ftp.contoso.com 220 Host accepted. ftp> USER administrator 331 Password required for administrator. Password: ******** 230-Directory has 104,857,600 bytes of disk space available. 230 User logged in. ftp> bye 221 Goodbye. CMD>

Scorecard for the MOVEit Freely command-line FTP client

This concludes our quick look at some of the features that are available with the MOVEit Freely command-line FTP client, and here's the scorecard results:

Note: Keeping with my standard disclaimer, there are a great number of additional features that the MOVEit Freely command-line FTP client provides - I'm just keeping the focus on those topic areas that apply to FTP7.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

For this next installment in my FTP Clients series, I'd like to take a look at the FileZilla FTP client. For this blog post I was using FileZilla version 3.1.6.

There are a lot of places where you can find FileZilla, but the best location is the official FileZilla web site at http://filezilla-project.org/. The FileZilla FTP client is free, so you can't beat the price. ;-]

The user interface is pretty straight-forward: you have separate windows for your local and remote files/folders, as well as a logging window that lists the FTP commands that are sent and the FTP server's responses.

FileZilla has a great Site Manager feature, which allows you to store commonly-used connections to FTP sites.

Using FTP over SSL (FTPS)

When creating a connection to an FTPS server, FileZilla has two options: FTPS and FTPES. It's important to have this option configured correctly, otherwise you will run into problems when trying access a site using FTPS. If you'll recall from my "FTP Clients - Part 2: Explicit FTPS versus Implicit FTPS" blog post, Explicit FTPS allows the client to initiate SSL/TLS whenever it wants, but for most FTP clients that will be when logging in to your FTP site, and in that regard it may almost seem like Implicit FTPS, but behind the scenes the FTP client and server are communicating differently.

In the case of FTP7, the following rules apply:

• If you enable FTPS and you assign the FTP site to port 990, you are using Implicit SSL - FileZilla refers to this as FTPS.
• If you enable FTPS and you assign the FTP site to any port other than port 990, you are using Explicit SSL - FileZilla refers to this as FTPES.

Using FTP Virtual Hosts

Because FileZilla's site manager allows you to specify the virtual host name as part of the user credentials, FileZilla works great with FTP7's virtual host names. All that you need to do is use the "ftp.example.com|username" syntax when specifying your username, and when you connect to the FTP7 server it will route your requests to the correct FTP virtual host site.

Unfortunately, even though FileZilla allows you to send custom commands, you cannot send custom commands outside of an established FTP session, so you can't send the FTP HOST command as part of your login, therefore true FTP hosts are not supported.

Directory Browsing

Using MS-DOS or UNIX directory listings in FTP7 didn't have any impact on whether FileZilla could render directory listings, nor did configuring any of the other options such as four-digit years, etc. When I create FTP connections in FileZilla's site manager it defaults to auto-detecting the FTP server type, which makes the directory browsing behavior transparent to the client. (Behind the scenes FileZilla is sending an FTP SYST command, which allows FileZilla to detect the operating system.)

You can customize the server type in the advanced settings for your FTP connection, so you can match up your FTP7 directory listing options and the server type that FileZilla expects, but personally I have had no problems with auto-detection so I prefer to use that option.

On a side note, if you intentionally misconfigure FileZilla's server type settings, you can cause FileZilla to behave strangely. For example, choosing a VMS server type and configuring FTP7 to use MS-DOS directory listings will not work, but then again - I wouldn't expect that to work. ;-]

Scorecard for FileZilla

So - that concludes our quick round-trip for some of FileZilla's features, and here's the scorecard results:

Note: As with all of the FTP clients in this blog series, there are a great number of additional features that FileZilla provides - I'm just keeping the focus on a few specific topic areas that apply to FTP7.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

In my "FTP Clients - Part 1: Web Browser Support" blog post, I mentioned creating a secured Global Listener FTP Site when you're working with FTP virtual hosts, but I didn't really explain what I meant by that or why you would want to do this. With that in mind, today's blog post is to describe how and why you might want to create a Global Listener FTP Site.

To start things off, the concept is really simple - a Global Listener FTP Site is an FTP site with no virtual host binding and anonymous access disabled. It's kind of like having a "Default FTP Site" with restricted access. Here's why this is a good idea when you're working with FTP virtual hosts - some clients default to anonymous, like web browsers, and if anonymous succeeds then the FTP client doesn't have the opportunity to enter the FTP virtual host name, so you can't get to the virtual host site.

To refresh everyone's memory, there are two different methods for binding multiple FTP host names to IP addresses in FTP7:

• FTP Virtual Host Names - This uses the "ftp.example.com|username" syntax as part of the client login in order to route FTP requests to the correct FTP site. This syntax is compatible with FTP almost every FTP clients, and should be thought of as a backwards-compatible method for binding multiple FTP host names to a single IP address.
• FTP True Host Names - This uses the FTP HOST command, which is still only an IETF draft at the moment. In the future this may be the way that FTP clients and servers automatically communicate with each other, like the "Host: www.example.com" header does for HTTP, but that may still have a few years at the very least.

Unless your FTP client allows sending custom FTP commands, you won't be able to use FTP True Host Names, so if you want to host several FTP sites on the same IP address then your only option is to use FTP Virtual Host Names. The trouble is, as I mentioned earlier, that some FTP clients (like web browsers) try to log in using anonymous first. If all of your FTP sites are bound to a virtual host name, the FTP client will get a "550-No such host is known" error from the FTP server, because the anonymous user did not specify a virtual host name as part of the USER command. On some clients you could fix that by specifying "ftp.example.com|anonymous" as your anonymous user name, but in most cases the login attempt will just fail.

If you create an FTP site that has no virtual host name, then the FTP service will have some place to send these default anonymous requests. When this FTP site does not have anonymous access enabled, the client will be prompted for their username, which will allow you to enter the "ftp.example.com|username" syntax to specify the virtual host name.

Please note that creating a Global Listener FTP site is really more of a workaround for the way that some FTP clients behave - it's certainly not required, and it only applies to situations where you are using FTP Virtual Host Names. For example, if you are using user isolation to restrict users to specific paths on a single FTP site, the Global Listener FTP site would be completely unnecessary.

More Information

Note: See my Virtual Hosts and Host Names in FTP7 blog post for more information about FTP Virtual Host Names and FTP True Host Names, and see https://datatracker.ietf.org/drafts/draft-hethmon-mcmurray-ftp-hosts/ for more information about status of the FTP HOST command.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

In part 2 of my series on FTP clients, I thought it would be best to have a discussion about the differences between Implicit FTPS and Explicit FTPS. In my recent "FTP Clients - Part 1: Web Browser Support" blog post, I referenced Implicit and Explicit FTPS with a link to my Using FTP Over SSL walkthrough. But it occurred to me that some people may not understand the difference between the two, and my upcoming blog posts are going to build upon that knowledge, so I thought that a quick discussion of these two technologies would be prudent.

FTP over SSL (FTPS)

One of the many limitations of the File Transfer Protocol (FTP) is a general lack of security; e.g. user names and passwords are transmitted in clear text, data is transferred with no encryption, etc. In order to address this situation, FTP over SSL (FTPS) was introduced in Requests for Comments (RFC) article 2228 - FTP Security Extensions, and expanded in RFC 4217 - Securing FTP with TLS to address Transport Layer Security (TLS).

Following up on these RFC articles, the FTP service for Windows Server 2008 added support for FTPS, and the FTP SSL Settings Feature in the IIS Manager allows you to configure your FTPS settings to allow or require SSL, enforce 128-bit SSL, or customize your control/data channel SSL settings.

Explicit FTPS

Explicit FTPS is really what RFCs 2228 and 4217 envisioned; basically the way this works is an FTP client connects over the control/command channel (usually on port 21), and then the client can negotiate SSL for either the command/control channel or the data channel using new FTP commands like AUTH, PROT, CCC, etc.

The FTP service for Windows Server 2008 allows customized settings for both the command/control channel and the data channel through the Advanced SSL Policy dialog:

There are several ways that Explicit FTPS might be implemented depending on your business needs:

Control Channel	Data Channel	Notes
Allow	Allow	This configuration allows the client to decide whether any part of the FTP session should be encrypted.
Require only for credentials	Allow	This configuration protects your FTP client credentials from electronic eavesdropping, and allows the client to decide whether data transfers should be encrypted.
Require only for credentials	Require	This configuration requires that the client's credentials must be secure, and then allows the client to decide whether FTP commands should be encrypted. However, all data transfers must be encrypted.
Require	Require	This configuration is the most secure - the client must negotiate SSL using the FTPS-related commands before other FTP commands are allowed, and all data transfers must be encrypted.

Implicit FTPS

Implicit FTPS takes SSL one step further than simply requiring that SSL-related commands must be sent first like you can with Explicit SSL; with Implicit FTPS, an SSL handshake must be negotiated before any FTP commands can be sent by the client. In addition, even though Explicit FTPS allows the client to arbitrarily decide whether to use SSL, Implicit FTPS requires that the entire FTP session must be encrypted. Basically the way that Implicit FTPS works is that an FTP client connects to the command/control channel, in this case using port 990, and immediately performs an SSL handshake; after SSL has been negotiated, additional FTP commands for the session can be sent by the FTP client.

Using FTPS in FTP service for Windows Server 2008 follows the Internet Assigned Numbers Authority (IANA) specification that the Implicit FTPS command/control channel is on port 990 and the Implicit FTPS data channel is on port 989.

Using FTPS in Windows Server 2008

Here's the way that you specify which type of FTP over SSL (FTPS) that you are using in Windows Server 2008:

• If you enable FTPS and you assign the FTP site to the default port of 21, you are using Explicit SSL.
• If you enable FTPS and you assign the FTP site to port 990, you are using Implicit SSL.
• In point of fact, if you enable FTPS and you assign the FTP site to any port other than port 990, you are always using Explicit SSL.

Note: If you are using FTP on any ports other than the defaults of 21/20 and 990/989, you must make sure that those ports are not already assigned by IANA to another protocol. For more information, see the list of assigned port numbers on IANA's web site.

Parting Thoughts

Choosing whether to use Explicit FTPS over Implicit FTPS is a personal choice, and generally this choice may depend on your business needs or your FTP client. In several FTP clients that I've tested, the FTP client chooses one form of FTPS over another as the default method, and the FTP client may require some manual configuration to use the other.

Shortly after shipping the FTP service for Windows Server 2008 we discovered an issue where the FTP service was not cleaning up Implicit SSL connections properly, and we issued a hotfix rollup package for the FTP service that is discussed in Microsoft Knowledge Base article 955136.

I hope this helps to clear things up a bit. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Since we've been testing a lot of FTP clients with our new FTP server for IIS 7, I thought that it would be a good idea to discuss some of the highlights and pitfalls that we have run into when testing various clients.

I thought that I'd begin this series with an examination of several web browsers, which are really not the best FTP clients around - web browsers are mostly just "putting a pretty face" on an FTP site rather than functioning as an FTP client. In any event, here's a summary table of different features that I tested with a few web browsers:

See the individual client notes below for more information on each client.

Firefox 3.0.2

Directory Browsing:

Some older Firefox versions only worked with UNIX directory listings - you could not use MS-DOS directory listings. That said, I was able to use version 3.0.2 with MS-DOS directory listings but only when I did not enable four-digit years; if you enable four-digit years, Firefox will only display blank directory listings.
 

Outside of that problem, Firefox has a great user experience for browsing FTP sites that's very reminiscent of Apache's Fancy Indexing feature for directories that have no home page.

FTP SSL/TLS:

SSL connections cannot be created by this client.

Virtual Hosts:

You can create a connection to an FTP7 virtual host only if a "Global Listener" FTP site is created that requires authentication, after which you can enter the "sitename|username" syntax to connect to the virtual host. There is no method for entering custom commands, so HOST does not work.

Observations:

One really odd thing that Firefox does is allow you to keep hitting the "Up to a higher level directory" link and it just keeps appending ".." to the path. That's really odd, and it doesn't seem to break anything, but it sure is annoying.



One last annoyance is that my default installation of Firefox cached the directory listings from the FTP server in between sessions. So I used Firefox to browse an FTP site, then completely closed Firefox, then when I came back I saw directory listings from my previous session instead of updated directory listings until I forced a refresh.

Internet Explorer 7.0

Directory Browsing:

Internet Explorer works with both MS-DOS and UNIX directory listings. But that being said, Internet Explorer's FTP browsing experience is completely no-frills - it's an ugly "What You Have Is What You See" approach to viewing your FTP site. (In fact, it looks a lot like the built-in IIS directory browsing pages.)



That being said, Internet Explorer works with both MS-DOS and UNIX directory listings, and has no trouble with four-digit years like Firefox.

FTP SSL/TLS:

SSL connections cannot be created by this client.

Virtual Hosts:

You can create a connection to an FTP7 virtual host only if a "Global Listener" FTP site is created that requires authentication, after which you can enter the "sitename|username" syntax to connect to the virtual host. There is no method for entering custom commands, so HOST does not work.

Observations:

A truly great feature of Internet Explorer's FTP browsing experience is the "Open FTP Site in Windows Explorer" integration, where you can switch from the web browser representation to a full Read/Write Windows Explorer view of your site.



One last annoyance is that Internet Explorer cached the directory listings from the FTP server in between sessions just like Firefox. If I completely closed Internet Explorer and reopened it I saw FTP directory listings from my previous session instead of updated directory listings until I forced a refresh. (Something about that just doesn't seem right. :-O)

Google Chrome 0.2.149 (Beta)

Directory Browsing:

In case you thought that Internet Explorer's FTP browsing experience was pretty basic, Google Chrome's browsing experience is even simpler than IE's.



That being said, Google Chrome seemed to work with both MS-DOS and UNIX directory listings, and had no trouble with four-digit years.

FTP SSL/TLS:

SSL connections cannot be created by this client.

Virtual Hosts:

You can create a connection to an FTP7 virtual host only if a "Global Listener" FTP site is created that requires authentication, after which you can enter the "sitename|username" syntax to connect to the virtual host. There is no method for entering custom commands, so HOST does not work.

Observations:

Part of me is inclined to cut Google Chrome a little slack on their display since this is their first browser and it's still only in beta. ;-)

Opera 9.5.2

Directory Browsing:

Much like Firefox, Opera has a rich FTP browsing experience:



The only downside to the directory browsing experience is slow the speed that it took to connect to the server and render the directory listing. I eventually got tired of waiting and stopped testing it before I could play around with the MS-DOS versus UNIX and other directory browsing settings.

FTP SSL/TLS:

SSL connections cannot be created by this client.

Virtual Hosts:

You can create a connection to an FTP7 virtual host only if a "Global Listener" FTP site is created that requires authentication, after which you can enter the "sitename|username" syntax to connect to the virtual host. There is no method for entering custom commands, so HOST does not work.

Observations:

As mentioned in the directory browsing notes earlier, Opera's FTP browsing experience was incredibly slow for me. I didn't make any changes to Opera at first - I simply used the default installation and it would take an inordinate amount of time to connect to the server; often the Opera browser would simply switch to "Not Responding" and I would have to end the task from Task Manager.

Summary

As far as web browsers are concerned, they're not great FTP clients. That being said, here's my thoughts on their respective experiences:

• Directory Browsing:
  • Best: Firefox 3.0.2, Opera 9.5.2
  • Worst: Google Chrome 0.2.149 (Beta), Internet Explorer 7.0
• Performance:
  • Best: Firefox 3.0.2, Google Chrome 0.2.149 (Beta), Internet Explorer 7.0
  • Worst: Opera 9.5.2
• Shell Integration:
  • Only Internet Explorer 7.0 had shell integration, which enables true Read/Write FTP capabilities
• FTP SSL/TLS:
  • None of the four web browsers that I tested had FTP SSL/TLS capabilities
• Virtual Hosts:
  • All four web browsers that I tested could use FTP7 virtual hosts

In the next part of this series, I'll start taking a look at some specific FTP clients.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Today Microsoft released the newest version of our FTP server for Windows Server 2008 Release Candidate 0 (RC0)!

Listed below are the links for the download pages for each of the individual installation packages:

• FTP 7 (x86) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=87847
• FTP 7 (x64) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=89114

I added the following text to my "Installing and Troubleshooting FTP7" walkthrough, but I’m going to repost that here because it sums up nicely what our new FTP server has to offer:

This new FTP service incorporates many new features that enable web authors to publish content better than before, and offers web administrators more security and deployment options.

• Integration with IIS 7.0: IIS 7.0 has a brand-new administration interface and configuration store, and the new FTP service is tightly integrated with this new design. The old IIS 6 metabase is gone, and a new configuration store that is based on the .NET XML-based *.config format has taken its place. In addition, IIS 7.0 has a new administration tool, and the new FTP server plugs seamlessly into that paradigm.
• Support for new Internet standards: One of the most significant features in the new FTP server is support for FTP over SSL. The new FTP server also supports other Internet improvements such as UTF8 and IPv6.
• Shared hosting improvements: By fully integrating into IIS 7.0, the new FTP server makes it possible to host FTP and Web content from the same site by simply adding an FTP binding to an existing Web site. In addition, the FTP server now has virtual host name support, making it possible to host multiple FTP sites on the same IP address. The new FTP server also has improved user isolation, now making it possible to isolate users through per-user virtual directories.
• Extensibility and custom authentication: The new FTP server supports developer extensibility, making it possible for software vendors to write custom providers for FTP authentication. Microsoft is using this extensibility feature to implement two new methods for using non-Windows accounts for FTP authentication for IIS Managers and .NET Membership.
• Improved logging support: FTP logging has been enhanced to include all FTP-related traffic, unique tracking for FTP sessions, FTP sub-statuses, additional detail fields in FTP logs, and much more.
• New supportability features: IIS 7.0 has a new option to display detailed error messages for local users, and the FTP server supports this by providing detailed error responses when logging on locally to an FTP server. The FTP server also logs detailed information using Event Tracing for Windows (ETW), which provides additional detailed information for troubleshooting.

Additional information about new features in FTP7 is available in the "What's New for Microsoft and FTP?" topic on Microsoft's http://www.iis.net/ web site.

The following prerequisites are required to install this new version:

1. You must be using Windows Server 2008 (code name "Longhorn") Release Candidate 0 (RC0) or later.
2. Internet Information Services 7.0 must be installed.
3. If you are going to manage the new FTP server using the IIS 7.0 user interface, the administration tool will need to be installed.
4. You must install the new FTP server as an administrator.
5. IIS 7.0 supports a new shared configuration environment, which must be disabled on each server in a web farm before installing the new FTP server for each node. Note: Shared configuration can be re-enabled after the FTP server had been installed.
6. The FTP server that is shipped on the Windows Server 2008 DVD must be uninstalled before installing the new FTP server.

To help you get started using the new FTP server, the following walkthroughs have been published on the http://www.iis.net/ web site:

• Installing FTP7:
  • Installing and Troubleshooting FTP7
• Working with FTP Sites:
  • Creating a New FTP Site
  • Add FTP publishing to an existing Web site
• Configuring Security Features:
  • Configure FTP over SSL
  • Configure FTP User Isolation
  • Configure FTP virtual host names
  • Configuring FTP Firewall Settings
• Advanced Administrative Features:
  • Using FSRM Folder Quotas with FTP
  • Configure IIS Manager Authentication
  • Administering the New FTP Service using the IIS 7 Configuration Files

Special thanks go to:

• Jaroslav, Emily, Daniel, Umer – for building it
• Suditi, Ciprian, Jeong – for testing it
• Andrew, Carlos, Brian – for making it look pretty
• Reagan, Claudia, Rick – for documenting it

Thanks!

I received the following question from Mickey Binder in response to my blog entries about the new FTP service for IIS7:

"When using virtual hosts will it always be necessary to provide DOMAIN|Username or is it me doing something wrong. I can see in your documentation how it should be setup, I just wonder why it still needs the DOMAIN in the user login as this should already be specified by the host I connect to (Like with the http virtual hosts)."

I thought that was a great question, and I think that it deserves a detailed response. The short answer is that you need to provide the host name somehow when connecting to an FTP site, and FTP simply does not work the way that HTTP does. I'll explain why in detail with this blog post.

FTP and HTTP Host Names

HTTP provides a way to pass the host name in the headers that are passed between the client and server, but FTP currently does not provide this same functionality. Looking at one of my older blog posts, I pointed out that there are three ways that you can create unique bindings for a Web/HTTP site: IP address, port, or host header. Based on its current design, FTP can create unique bindings by IP address or port, but the FTP protocol currently does not allow for something like host headers.

Here's why - I'm sure most everyone is aware that HTTP packets consist of a set of headers and possibly a block of data. Here's an example of a simple GET request:

GET /default.aspx HTTP/1.0 [crlf]
Accept: */* [crlf]
[crlf]

When HTTP 1.1 was published in RFC 2068 and RFC 2616 it defined a header for specifying a "host" name in a separate name/value pair:

GET /default.aspx HTTP/1.1 [crlf]
Host: example.com [crlf]
Accept: */* [crlf]
[crlf] 

This allows multiple virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, the FTP protocol currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then IIS and all FTP clients would need to be updated to accommodate the changes to FTP.

I realize that most everyone is aware that when you attempt to connect to an HTTP or FTP server from a client, the client looks up the IP address using a name server and then creates a connection to the server by IP address. What many people may not realize is that the server is basically unaware of the host name that the client used - at connection time the server is really only concerned with inbound data on the IP address.

That being said, the connection protocol could provide a mechanism for specifying the host name. For example, as I mentioned above the HTTP/1.1 protocol provides a mechanism for specifying host names using a host header and FTP does not. In fact, if you were to open Network Monitor or similar tool and capture an FTP connection between a client and a server, you would be able to see that the host name is not passed as part of the FTP conversation between the client and server.

Host Names in FTP for IIS7

For the new FTP service for IIS7 we wanted to find a way to have host names in FTP, so we approached the situation with two possible ideas:

1. Find a backwards-compatible method for specifying host names for FTP
2. See if we could get the FTP protocol updated for host names

In the end, we chose to do both.

FTP Virtual Host Name Support

First of all we have "Virtual Host" names; this is the "ftp.example.com|username" syntax that led to the original question. Here's the way that this can be used in a hosting environment. Let's say that a web hoster has a customer with several Web/FTP sites that he needs to update. The web hoster gives all of his customers the same ftp address, like "ftp.example.com". When setting up the bindings for the Web/FTP sites the web hoster configures each site for the appropriate HTTP/FTP bindings, using host names like "www.contoso.com" & "www.fabrikam.com" for the HTTP bindings and "ftp.contoso.com" & "ftp.fabrikam.com" for FTP bindings. (All of the Web names would be set up in DNS, of course, but the FTP names are somewhat optional as I'll explain later.)

Since HTTP provides host headers to pass the host name, the Web client & Web server will both know which site they're working with. Since FTP doesn't have a built-in way to pass a host name, the customer will connect to the "ftp.example.com" name that his web hoster gave him and log in using the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax. The FTP service for IIS7 will then internally route the FTP activity to the correct site based on the FTP bindings. If the FTP names "ftp.contoso.com" & "ftp.fabrikam.com" were registered in DNS, the client would still need to specify the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax when logging in because the FTP activity did not actually pass the host name in any way.

The great thing about the "Virtual Host" names solution is that it is backwards-compatible because any client should be able to send the "ftp.example.com|username" syntax.

FTP True Host Name Support

We also wanted to see if the FTP protocol could be updated to allow sending a host name as part of the FTP session like HTTP does. After some research I discovered that Robert Elz and Paul Hethmon had provided a detailed discussion of a "HOST" command for FTP in their Internet draft titled "Extensions to FTP" during their work with the FTPEXT Working Group at the IETF. I contacted the two of them and with their approval Paul and I submitted a new IETF draft detailing a HOST command for FTP, which is posted at the following URL:

http://www.ietf.org/internet-drafts/draft-hethmon-mcmurray-ftp-hosts-00.txt

Here's the way that the HOST command works: the FTP server and FTP client both need to know that the HOST command is supported. The FTP client connects to an FTP server using either a DNS name or IP address and sends a FEAT command and sees that HOST is supported, so the client sends "HOST ftp.example.com" before sending USER and PASS, which allows the FTP server to route the request to the correct FTP site based on the bindings. (An FTP client could skip the FEAT command completely and simply attempt a HOST command and process the FTP reply, but that's not the best approach.)

The FTP HOST command solution is not backwards-compatible, however, because an FTP client needs to be able to send the "HOST ftp.example.com" syntax. Some FTP clients allow sending custom commands, which enables this functionality. For example, if you are using the FTP.EXE command-line tool that comes in Windows, you can type "quote HOST ftp.example.com" when connecting to the new FTP server for IIS7 and it will route the request to the appropriate site. In an ideal world, FTP clients will start negotiating the HOST feature behind the scenes and you should never know that this is occurring, which is how almost all Web browsers currently work. (e.g. When you enter "www.example.com" in the address bar of a Web browser it will automatically add the host header to the HTTP request.)

The Final Word

So the long answer to the original question is that you have two ways of specifying a host with the new FTP service for IIS7:

• FTP "Virtual Host" names are supported using "ftp.example.com|username" syntax
• True FTP host names are supported using the "HOST ftp.example.com" syntax

So my thanks to Mickey Binder for his great question, and I hope this helps other people understand this concept a little better.

A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using VBScript.

This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using C#. To do so, start a new Windows Console Application project using C# in Visual Studio 2005 on a computer running Windows Server 2008 with the new FTP7 server installed. You will need to add a reference to the AppHostAdminLibrary by manually browsing to the nativerd.dll file that's located in the %WinDir%\System32\InetSrv folder. After you've added the reference, replace all of the C# code from the project template with the following C# code:

using System;
using System.Collections.Generic;
using System.Text;
using AppHostAdminLibrary;

namespace FtpDumpSessions
{
  class FtpDumpSessions
  {
    static void Main(string[] args)
    {
      AppHostWritableAdminManager objAdminManager =
        new AppHostWritableAdminManager();

      // get the collection of sites
      IAppHostElement objSitesElement =
        objAdminManager.GetAdminSection(
        "system.applicationHost/sites",
        "MACHINE/WEBROOT/APPHOST");
      uint intSiteCount =
        objSitesElement.Collection.Count;
      Console.WriteLine(
        "Site count: {0}",
        intSiteCount);

      try
      {
        // loop through the sites collection
        for (int intSite = 0;
          intSite < intSiteCount;
          ++intSite)
        {
          // get a site
          IAppHostElement objFtpSite =
            objSitesElement.Collection[intSite];

          // get the FTP section
          IAppHostElement objFtpSiteElement =
            objFtpSite.ChildElements["ftpServer"];

          // get the sessions collection
          IAppHostElement objFtpSessions =
            objFtpSiteElement.ChildElements["sessions"];
          uint intSessionCount =
            objFtpSessions.Collection.Count;
          Console.WriteLine(
            "\tFTP sessions for {0}: {1}",
            objFtpSite.Properties["name"].Value, intSessionCount);

          // loop through the sessions
          for (int intSession = 0;
            intSession < intSessionCount;
            ++intSession)
          {
            IAppHostElement objFtpSession =
              objFtpSessions.Collection[intSession];
            // loop through each session's properties
            for (int intProperty = 0;
              intProperty < objFtpSession.Properties.Count;
              ++intProperty)
            {
              Console.WriteLine(
                "\t\t{0}: {1}",
                objFtpSession.Properties[intProperty].Name,
                objFtpSession.Properties[intProperty].Value);
            }
          }
        }
      }
      catch (System.Exception ex)
      {
        Console.WriteLine(
          "\r\nError: {0}",
          ex.Message);
      }
    }
  }
}

When you compile and run the project, you should see a listing of all users connected to your FTP7 sites.

That's about it for this post - have fun!

A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using C#.

This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using VBScript. To do so, copy the following VBScript code to Windows Notepad and save the file as "ftp_sessions.vbs" on a computer running Windows Server 2008 with the new FTP7 server installed:

Option Explicit

Dim objAdminManager, objSiteCollection, objFtpSiteElement
Dim objSite, objFtpSession, objFtpSessions, objFtpProperty
Dim intSite, intFtpSession, intFtpProperty
Dim intSiteCount, intFtpSessionCount, intFtpPropertyCount

Set objAdminManager = WScript.CreateObject("Microsoft.ApplicationHost.AdminManager")

' get the collection of sites
Set objSiteCollection = objAdminManager.GetAdminSection( _
  "system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST" )

intSiteCount = CInt(objSiteCollection.Collection.Count)

WScript.Echo String(40,"*")
WScript.Echo "Site count: " & intSiteCount
WScript.Echo String(40,"*")

' loop through the sites collection
For intSite = 0 To intSiteCount-1

  ' get a site
  Set objSite = objSiteCollection.Collection.Item(intSite)
  
  ' get the FTP section
  Set objFtpSiteElement = objSite.ChildElements.Item("ftpServer")
  
  ' get the sessions collection
  Set objFtpSessions = objFtpSiteElement.ChildElements.Item("sessions")
  intFtpSessionCount = CInt(objFtpSessions.Collection.Count)

  WScript.Echo String(40,"=")
  WScript.Echo "FTP sessions for " & _
    objSite.Properties.Item("name").Value & _
    ": " & intFtpSessionCount
  WScript.Echo String(40,"=")

  ' loop through the sessions
  For intFtpSession = 0 To intFtpSessionCount - 1
    Set objFtpSession = objFtpSessions.Collection.Item(intFtpSession)
    intFtpPropertyCount = CInt(objFtpSession.Properties.Count)
    ' loop through each session's properties
    For intFtpProperty = 0 To intFtpPropertyCount - 1
      Set objFtpProperty = objFtpSession.Properties.Item(intFtpProperty)
      WScript.Echo CStr(objFtpProperty.Name) & ": " & CStr(objFtpProperty.Value)
    Next
    WScript.Echo String(40,"-")
  Next
Next

To make sure that you don't see any message box pop-ups, run the script from the command-line using the following syntax:

cscript.exe ftp_sessions.vbs

That's about it for this post - have fun!

It's been a long time in development, but Microsoft has released a beta version of a new FTP service that we have completely rewritten for Windows Server code name "Longhorn". This new FTP service contains many great new features, such as:

• Integration of Web sites and FTP sites, allowing you to optionally host Web content and Web authoring on the same site by adding FTP settings.
• Adoption of the IIS 7.0 administration and configuration files, allowing .NET-style configuration of your FTP sites and using the new IIS 7.0 administration tool to manage your FTP settings.
• Data security through Secure Sockets Layer (SSL) and FTPS
• Support for additional Internet FTP standards such as UTF8 and IPv6.
• Hosting multiple FTP sites on the same IP address through the use of FTP virtual host names.
• Enhanced user isolation that allows you to create complex heirarchies of virtual directories to lock users into specific content locations.
• Improved supportability through improved W3C logging, detailed local error messages, and ETW tracing.

There are way too many features to list in such a short space, so a good place for addtional information is the "What’s New for Microsoft and FTP?" article. I encourage you to download the beta version of the new FTP service today, and listed below are the links for the download pages for each of the individual installation packages:

• FTP 7 (x86) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=87847
• FTP 7 (amd64) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=89114

After installing the new FTP server, the following walkthroughs on the www.iis.net web site should help get you started:

• Add FTP publishing to an existing Web site
• Creating a new FTP site
• Configure FTP over SSL
• Configure FTP user isolation
• Configure FTP virtual host names
• Administering the New FTP Service using the IIS 7 Configuration Files

In closing, the beta version of this new FTP service has a lot to offer, and we've put a lot of time and effort into making what we think is a great start for the future of FTP for IIS. About the only piece of bad news that I have for anyone is that this new FTP service will not work on Windows Server 2003 with IIS 6.0.

All in all, our team is excited to see people start testing with this beta version. And lest I forget, my special thanks go to:

• Jaro, Emily, Daniel – for building it
• Suditi, Cip, Jeong – for testing it
• Janani, Andrew, Reagan – for making it look pretty

Thanks everyone!