Geeky Bob

I received the following question from Mickey Binder in response to my blog entries about the new FTP service for IIS7:

"When using virtual hosts will it always be necessary to provide DOMAIN|Username or is it me doing something wrong. I can see in your documentation how it should be setup, I just wonder why it still needs the DOMAIN in the user login as this should already be specified by the host I connect to (Like with the http virtual hosts)."

I thought that was a great question, and I think that it deserves a detailed response. The short answer is that you need to provide the host name somehow when connecting to an FTP site, and FTP simply does not work the way that HTTP does. I'll explain why in detail with this blog post.

FTP and HTTP Host Names

HTTP provides a way to pass the host name in the headers that are passed between the client and server, but FTP currently does not provide this same functionality. Looking at one of my older blog posts, I pointed out that there are three ways that you can create unique bindings for a Web/HTTP site: IP address, port, or host header. Based on its current design, FTP can create unique bindings by IP address or port, but the FTP protocol currently does not allow for something like host headers.

Here's why - I'm sure most everyone is aware that HTTP packets consist of a set of headers and possibly a block of data. Here's an example of a simple GET request:

GET /default.aspx HTTP/1.0 [crlf]
Accept: */* [crlf]
[crlf]

When HTTP 1.1 was published in RFC 2068 and RFC 2616 it defined a header for specifying a "host" name in a separate name/value pair:

GET /default.aspx HTTP/1.1 [crlf]
Host: example.com [crlf]
Accept: */* [crlf]
[crlf] 

This allows multiple virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, the FTP protocol currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then IIS and all FTP clients would need to be updated to accommodate the changes to FTP.

I realize that most everyone is aware that when you attempt to connect to an HTTP or FTP server from a client, the client looks up the IP address using a name server and then creates a connection to the server by IP address. What many people may not realize is that the server is basically unaware of the host name that the client used - at connection time the server is really only concerned with inbound data on the IP address.

That being said, the connection protocol could provide a mechanism for specifying the host name. For example, as I mentioned above the HTTP/1.1 protocol provides a mechanism for specifying host names using a host header and FTP does not. In fact, if you were to open Network Monitor or similar tool and capture an FTP connection between a client and a server, you would be able to see that the host name is not passed as part of the FTP conversation between the client and server.

Host Names in FTP for IIS7

For the new FTP service for IIS7 we wanted to find a way to have host names in FTP, so we approached the situation with two possible ideas:

1. Find a backwards-compatible method for specifying host names for FTP
2. See if we could get the FTP protocol updated for host names

In the end, we chose to do both.

FTP Virtual Host Name Support

First of all we have "Virtual Host" names; this is the "ftp.example.com|username" syntax that led to the original question. Here's the way that this can be used in a hosting environment. Let's say that a web hoster has a customer with several Web/FTP sites that he needs to update. The web hoster gives all of his customers the same ftp address, like "ftp.example.com". When setting up the bindings for the Web/FTP sites the web hoster configures each site for the appropriate HTTP/FTP bindings, using host names like "www.contoso.com" & "www.fabrikam.com" for the HTTP bindings and "ftp.contoso.com" & "ftp.fabrikam.com" for FTP bindings. (All of the Web names would be set up in DNS, of course, but the FTP names are somewhat optional as I'll explain later.)

Since HTTP provides host headers to pass the host name, the Web client & Web server will both know which site they're working with. Since FTP doesn't have a built-in way to pass a host name, the customer will connect to the "ftp.example.com" name that his web hoster gave him and log in using the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax. The FTP service for IIS7 will then internally route the FTP activity to the correct site based on the FTP bindings. If the FTP names "ftp.contoso.com" & "ftp.fabrikam.com" were registered in DNS, the client would still need to specify the appropriate "ftp.contoso.com|customer" or "ftp.fabrikam.com|customer" syntax when logging in because the FTP activity did not actually pass the host name in any way.

The great thing about the "Virtual Host" names solution is that it is backwards-compatible because any client should be able to send the "ftp.example.com|username" syntax.

FTP True Host Name Support

We also wanted to see if the FTP protocol could be updated to allow sending a host name as part of the FTP session like HTTP does. After some research I discovered that Robert Elz and Paul Hethmon had provided a detailed discussion of a "HOST" command for FTP in their Internet draft titled "Extensions to FTP" during their work with the FTPEXT Working Group at the IETF. I contacted the two of them and with their approval Paul and I submitted a new IETF draft detailing a HOST command for FTP, which is posted at the following URL:

http://www.ietf.org/internet-drafts/draft-hethmon-mcmurray-ftp-hosts-00.txt

Here's the way that the HOST command works: the FTP server and FTP client both need to know that the HOST command is supported. The FTP client connects to an FTP server using either a DNS name or IP address and sends a FEAT command and sees that HOST is supported, so the client sends "HOST ftp.example.com" before sending USER and PASS, which allows the FTP server to route the request to the correct FTP site based on the bindings. (An FTP client could skip the FEAT command completely and simply attempt a HOST command and process the FTP reply, but that's not the best approach.)

The FTP HOST command solution is not backwards-compatible, however, because an FTP client needs to be able to send the "HOST ftp.example.com" syntax. Some FTP clients allow sending custom commands, which enables this functionality. For example, if you are using the FTP.EXE command-line tool that comes in Windows, you can type "quote HOST ftp.example.com" when connecting to the new FTP server for IIS7 and it will route the request to the appropriate site. In an ideal world, FTP clients will start negotiating the HOST feature behind the scenes and you should never know that this is occurring, which is how almost all Web browsers currently work. (e.g. When you enter "www.example.com" in the address bar of a Web browser it will automatically add the host header to the HTTP request.)

The Final Word

So the long answer to the original question is that you have two ways of specifying a host with the new FTP service for IIS7:

• FTP "Virtual Host" names are supported using "ftp.example.com|username" syntax
• True FTP host names are supported using the "HOST ftp.example.com" syntax

So my thanks to Mickey Binder for his great question, and I hope this helps other people understand this concept a little better.

A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using VBScript.

This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using C#. To do so, start a new Windows Console Application project using C# in Visual Studio 2005 on a computer running Windows Server 2008 with the new FTP7 server installed. You will need to add a reference to the AppHostAdminLibrary by manually browsing to the nativerd.dll file that's located in the %WinDir%\System32\InetSrv folder. After you've added the reference, replace all of the C# code from the project template with the following C# code:

using System;
using System.Collections.Generic;
using System.Text;
using AppHostAdminLibrary;

namespace FtpDumpSessions
{
  class FtpDumpSessions
  {
    static void Main(string[] args)
    {
      AppHostWritableAdminManager objAdminManager =
        new AppHostWritableAdminManager();

      // get the collection of sites
      IAppHostElement objSitesElement =
        objAdminManager.GetAdminSection(
        "system.applicationHost/sites",
        "MACHINE/WEBROOT/APPHOST");
      uint intSiteCount =
        objSitesElement.Collection.Count;
      Console.WriteLine(
        "Site count: {0}",
        intSiteCount);

      try
      {
        // loop through the sites collection
        for (int intSite = 0;
          intSite < intSiteCount;
          ++intSite)
        {
          // get a site
          IAppHostElement objFtpSite =
            objSitesElement.Collection[intSite];

          // get the FTP section
          IAppHostElement objFtpSiteElement =
            objFtpSite.ChildElements["ftpServer"];

          // get the sessions collection
          IAppHostElement objFtpSessions =
            objFtpSiteElement.ChildElements["sessions"];
          uint intSessionCount =
            objFtpSessions.Collection.Count;
          Console.WriteLine(
            "\tFTP sessions for {0}: {1}",
            objFtpSite.Properties["name"].Value, intSessionCount);

          // loop through the sessions
          for (int intSession = 0;
            intSession < intSessionCount;
            ++intSession)
          {
            IAppHostElement objFtpSession =
              objFtpSessions.Collection[intSession];
            // loop through each session's properties
            for (int intProperty = 0;
              intProperty < objFtpSession.Properties.Count;
              ++intProperty)
            {
              Console.WriteLine(
                "\t\t{0}: {1}",
                objFtpSession.Properties[intProperty].Name,
                objFtpSession.Properties[intProperty].Value);
            }
          }
        }
      }
      catch (System.Exception ex)
      {
        Console.WriteLine(
          "\r\nError: {0}",
          ex.Message);
      }
    }
  }
}

When you compile and run the project, you should see a listing of all users connected to your FTP7 sites.

That's about it for this post - have fun!

A few weeks ago my friend Jaroslav posted a blog entry about viewing the current FTP7 sessions using Javascript, and I followed that up with a blog post about viewing the current FTP7 sessions using C#.

This blog entry follows up on those postings by showing you how to view the current FTP7 sessions using VBScript. To do so, copy the following VBScript code to Windows Notepad and save the file as "ftp_sessions.vbs" on a computer running Windows Server 2008 with the new FTP7 server installed:

Option Explicit

Dim objAdminManager, objSiteCollection, objFtpSiteElement
Dim objSite, objFtpSession, objFtpSessions, objFtpProperty
Dim intSite, intFtpSession, intFtpProperty
Dim intSiteCount, intFtpSessionCount, intFtpPropertyCount

Set objAdminManager = WScript.CreateObject("Microsoft.ApplicationHost.AdminManager")

' get the collection of sites
Set objSiteCollection = objAdminManager.GetAdminSection( _
  "system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST" )

intSiteCount = CInt(objSiteCollection.Collection.Count)

WScript.Echo String(40,"*")
WScript.Echo "Site count: " & intSiteCount
WScript.Echo String(40,"*")

' loop through the sites collection
For intSite = 0 To intSiteCount-1

  ' get a site
  Set objSite = objSiteCollection.Collection.Item(intSite)
  
  ' get the FTP section
  Set objFtpSiteElement = objSite.ChildElements.Item("ftpServer")
  
  ' get the sessions collection
  Set objFtpSessions = objFtpSiteElement.ChildElements.Item("sessions")
  intFtpSessionCount = CInt(objFtpSessions.Collection.Count)

  WScript.Echo String(40,"=")
  WScript.Echo "FTP sessions for " & _
    objSite.Properties.Item("name").Value & _
    ": " & intFtpSessionCount
  WScript.Echo String(40,"=")

  ' loop through the sessions
  For intFtpSession = 0 To intFtpSessionCount - 1
    Set objFtpSession = objFtpSessions.Collection.Item(intFtpSession)
    intFtpPropertyCount = CInt(objFtpSession.Properties.Count)
    ' loop through each session's properties
    For intFtpProperty = 0 To intFtpPropertyCount - 1
      Set objFtpProperty = objFtpSession.Properties.Item(intFtpProperty)
      WScript.Echo CStr(objFtpProperty.Name) & ": " & CStr(objFtpProperty.Value)
    Next
    WScript.Echo String(40,"-")
  Next
Next

To make sure that you don't see any message box pop-ups, run the script from the command-line using the following syntax:

cscript.exe ftp_sessions.vbs

That's about it for this post - have fun!

It's been a long time in development, but Microsoft has released a beta version of a new FTP service that we have completely rewritten for Windows Server code name "Longhorn". This new FTP service contains many great new features, such as:

• Integration of Web sites and FTP sites, allowing you to optionally host Web content and Web authoring on the same site by adding FTP settings.
• Adoption of the IIS 7.0 administration and configuration files, allowing .NET-style configuration of your FTP sites and using the new IIS 7.0 administration tool to manage your FTP settings.
• Data security through Secure Sockets Layer (SSL) and FTPS
• Support for additional Internet FTP standards such as UTF8 and IPv6.
• Hosting multiple FTP sites on the same IP address through the use of FTP virtual host names.
• Enhanced user isolation that allows you to create complex heirarchies of virtual directories to lock users into specific content locations.
• Improved supportability through improved W3C logging, detailed local error messages, and ETW tracing.

There are way too many features to list in such a short space, so a good place for addtional information is the "What’s New for Microsoft and FTP?" article. I encourage you to download the beta version of the new FTP service today, and listed below are the links for the download pages for each of the individual installation packages:

• FTP 7 (x86) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=87847
• FTP 7 (amd64) Installation Package
  http://go.microsoft.com/fwlink/?LinkId=89114

After installing the new FTP server, the following walkthroughs on the www.iis.net web site should help get you started:

• Add FTP publishing to an existing Web site
• Creating a new FTP site
• Configure FTP over SSL
• Configure FTP user isolation
• Configure FTP virtual host names
• Administering the New FTP Service using the IIS 7 Configuration Files

In closing, the beta version of this new FTP service has a lot to offer, and we've put a lot of time and effort into making what we think is a great start for the future of FTP for IIS. About the only piece of bad news that I have for anyone is that this new FTP service will not work on Windows Server 2003 with IIS 6.0.

All in all, our team is excited to see people start testing with this beta version. And lest I forget, my special thanks go to:

• Jaro, Emily, Daniel – for building it
• Suditi, Cip, Jeong – for testing it
• Janani, Andrew, Reagan – for making it look pretty

Thanks everyone!

In IIS 4.0 and IIS 5.0, if you created a virtual directory that had a name that was identical to a user name, when the user logged in to the FTP site they would automatically be changed to their folder. When multiple users will access the same FTP content, you could create another virtual directory that is identical to the other user name and point it to the same content.

This allowed sharing a single FTP site across several users and content sets without advertising user names or content folders. Even though a user could type "CD /" from an FTP prompt, they would not be able to see the virtual directories from other user accounts on that server because virtual directories are not listed when a user types "ls -l" or "dir" from an FTP prompt at the root. That being said, this security feature still doesn't go far enough from a security perspective.

One of the great IIS 6.0 features is User Isolation, which is discussed in the Hosting Multiple FTP Sites with FTP User Isolation (IIS 6.0) topic on MSDN. As a quick review, there are three different isolation modes that you can choose when creating an IIS 6.0 FTP site:

• "Do Not Isolate Users"
  No user isolation; FTP works like IIS 4.0 or IIS 5.0.
  (Not covered in this post.)
• "Isolate Users"
  Simple user isolation through folders.
  (Described below.)
• "Isolate Users Using Active Directory"
  Requires A.D. configuration.
  (Not covered in this post.)

To configure the "Isolate Users" mode, you first need to create your FTP site and choose the "Isolate Users" option when prompted for FTP User Isolation. Once the FTP site has been created, you need to create a physical folder named "LocalUser" for local user accounts or named after your domain under your FTP server's root folder. To isolate users to a specific folder, you use these rules that I copied from the MSDN topic that I listed earlier in this post:

• For anonymous users, the home directory is LocalUser\Public under the FTP root directory.
• For local users, the home directory is LocalUser\UserName under the FTP root directory.
• For users that log on with Domain\UserName, the home directory is Domain\UserName under the FTP root directory.

This is very easy to configure, and when a user logs in to your FTP server they will be restricted to their physical folder under the FTP root. Typing "CD /" from an FTP prompt will always restrict the user within their own site.

That being said, because physical directories are required for this configuration it may seem like a step backward when you consider that you used to be able to create multiple virtual directories that pointed to content in varying locations and for multiple user accounts. Not to worry, however, because Windows provides a way around this limitation using NTFS junctions.

For those of you that are not familiar with NTFS junctions, there are several topics that discuss this. (For example, see Inside Win2K NTFS, Part 1.) A junction is somewhat like a symbolic directory link in the UNIX world, where a junction looks like a folder but points to content that is physically located somewhere else. There are two tools that you can use to create junctions, LINKD from the Windows Resource Kit, and JUNCTION from www.sysinternals.com. Using these tools with IIS 6.0 can allow you the freedom to deploy FTP folder structures much like you did with IIS 4/5 while utilizing the user isolation features in IIS 6.

Here's an example - when configuring an IIS 6.0 FTP site, I used the following steps:

1. I chose the "Isolate Users" option when creating my FTP site.
2. I created the "LocalUser" physical folder under my FTP site's root folder.
3. I created junctions under the "LocalUser" physical folder that were named after user accounts and pointed to various content folders.

When a user logs in to my FTP site using their user account, they are automatically dropped in their content folder via the junction. Since you can create multiple junctions that point to the same content folder, you can create junctions for every user account that will work with a set of content.

I hope this helps!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/