Just a short, simple blog for Bob to share his thoughts.
24 May 2013 • by Bob • IIS, FTP
I get a lot of questions about Active versus Passive FTP Connections, specifically when people are configuring their FTP firewall settings as described in my articles like Configuring FTP Firewall Settings in IIS and FTP Firewall Support, and I get related questions when people are trying to figure out why they can't use the command-line FTP.EXE utility that ships with Windows over the Internet. With all of this in mind, I thought that I would put together a quick blog that explains Active and Passive FTP connections and why those matter when you are connecting to an FTP server over the Internet.
Here is the briefest description that I can think of to describe the different between Active and Passive FTP:
That sums up the whole concept into two bullet points. Unfortunately, neither of those bullet points are the least bit significant to you if you don't understand what they actually mean, so I will describe them in detail below.
The following output shows the communication between an FTP client and FTP server using Active FTP to retrieve a simple directory listing, and I manually color-coded the output so that the client and server's responses would be a little easier to see:
OPEN ftp.contoso.com Resolving ftp.contoso.com... Connect socket #1920 to 169.254.10.1, port 21... 220 Microsoft FTP Service HOST ftp.contoso.com 220 Host accepted. USER robert 331 Password required for robert. PASS ********** 230 User logged in. PWD 257 "/" is current directory. PORT 169,254,10,2,72,50 200 PORT command successful. LIST 125 Data connection already open; Transfer starting. drwxrwxrwx 1 owner group 0 Feb 15 19:26 aspnet_client -rwxrwxrwx 1 owner group 689 Jan 31 22:27 default.htm 226 Transfer complete. Transferred 106 bytes in 0.008 seconds QUIT 221 Goodbye.
In the beginning of this exchange, the client connects to the server from one of its ephemeral data ports to the server's port for the FTP command channel. After negotiating the FTP host, username, and password, the client retrieves the name of the current directory. So far all of the client/server communication has taken place over the FTP command channel, and up to now the conversation is identical to Passive FTP, but that is about to change.
The client's next task is to request a directory listing, which is denoted by the LIST
command. The server will return its response to the client over the data channel, so before FTP client can send the LIST
command, the client has to specify whether to use Active or Passive. In this example, the client has specified Active FTP by sending a PORT
command. The syntax for this command is PORT A1,A2,A3,A4,P1,P2
, where A1
through A4
are octets of the client's IPv4 address, and P1
/P2
are two bytes that make up a 16-bit (0-65535) port address on the client. (Note: if you are using IPv6, there is a similar EPRT
command that works with IPv6 addresses.)
Here's what the information in the PORT
command means: the FTP client is essentially telling the FTP server, "For the upcoming data transfer, you need to talk to me at this IP address on this port." This means that the FTP client is actively in control of how the subsequent data communication is going to take place.
If we analyze this information, you can easily see why Active FTP will often fail to work over the Internet. As a relevant example, if you were to use the FTP.EXE client that ships with Windows, it can only use Active FTP. So when a client computer requests something from the server that needs to use the data channel, the client computer sends its IP address via a PORT
command. If the FTP client is behind a firewall or NAT server, then the client is going to send its internal, LAN-based address, to which the FTP server will more than likely fail to connect. For example, if you are on a LAN that uses a NAT server and you have a 192.168.0.nnn IPv4 address, that IP address is invalid over the Internet, so the server will never be able to establish a data connection to your client to send the data. (Note: This is the reason why many customers contact me with the following problem description: "I can use FTP.EXE to connect to my server, and everything works until I try to retrieve a directory listing, then it hangs until the connection times out." What is actually happening is the FTP server is trying to connect to the FTP client's IP address and port that were specified by the PORT command, but the connection does not succeed because the server cannot connect to the private IP address of the client.)
The following output shows the communication between an FTP client and FTP server using Passive FTP to retrieve the same directory listing as my previous example, and once again I manually color-coded the output so that the client and server's responses would be a little easier to see:
OPEN ftp.contoso.com Resolving ftp.contoso.com... Connect socket #2076 to 169.254.10.1, port 21... 220 Microsoft FTP Service HOST ftp.contoso.com 220 Host accepted. USER robert 331 Password required for robert. PASS ********** 230 User logged in. PWD 257 "/" is current directory. PASV 227 Entering Passive Mode (169,254,10,1,197,19). LIST Connect socket #2104 to 169.254.10.1, port 50451... 150 Opening ASCII mode data connection. drwxrwxrwx 1 owner group 0 Feb 15 19:26 aspnet_client -rwxrwxrwx 1 owner group 689 Jan 31 22:27 default.htm 226 Transfer complete. Transferred 106 bytes in 0.008 seconds QUIT 221 Goodbye.
As I mentioned in my earlier example, the beginning of this conversation is identical to Active FTP: the client connects from one of its ephemeral data ports to the server's port for the FTP command channel. After negotiating the FTP host, username, and password, the client retrieves the name of the current directory as in my earlier example - and here's where the difference begins.
Once again the client's next task is to request the directory listing, which is still denoted by the LIST
command. But in this second example, the client has specified Passive FTP by sending a PASV
command. The server responds to this command with a reply that is in the format of 227 Entering Passive Mode (A1,A2,A3,A4,P1,P2)
, where A1
through A4
are octets of the server's IPv4 address, and P1
/P2
are two bytes that make up a 16-bit (0-65535) port address on the server. (Note: if you are using IPv6, there is a similar EPSV
command that works with IPv6 addresses.)
Here's what the information in the response to the PASV
command means: the FTP client is essentially telling the FTP server, "For the upcoming data transfer, you need to tell me which IP address and port I should use to talk to you." This means that the FTP client is passively allowing the server to control how the subsequent data communication is going to take place.
If we analyze this information, you can easily see why Passive FTP often works over the Internet; when the FTP server is in control of the communication parameters, it doesn't matter whether the FTP client is behind a NAT server, because the server is telling the client how it should communicate with the server.
All of this leads to an obvious question: what happens when both the server and the client are behind NAT servers or firewalls? This is where a little bit of configuration comes into play. If you read my Configuring FTP Firewall Settings in IIS and FTP Firewall Support articles, you would notice that you can configure the IIS FTP service to tell the FTP client which IP address to use; when your FTP server is located behind a firewall, you would configure your FTP server to send the external IP address of your firewall, and then you would configure your firewall to route FTP requests on that IP address to your FTP server. Since the FTP server is sending the external IP address of your firewall, the client knows how to communicate to the FTP server even though it is behind a firewall, even if your server is using a LAN-based internal IP address.
Having explained everything in my preceding examples, you should now understand what I meant earlier when I described the difference between Active and Passive FTP with these two simple points:
I hope this clears up some questions you might have about Active versus Passive FTP, why you sometimes need to configure your firewall settings for your FTP service, and why the built-in FTP client for Windows seldom works over the Internet.
That wraps it up for today's blog post. ;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
02 May 2013 • by Bob • IIS, FTP, Extensibility
In Part 1 of this blog series about adding custom FTP providers with the IIS Configuration Editor, I showed you how to add a custom FTP provider with a custom setting for the provider that is stored in your IIS configuration settings. For my examples, I showed how to do this by using both the AppCmd.exe application from a command line and by using the IIS Configuration Editor. In part 2 of this blog series, I will show you how to use the IIS Configuration Editor to add custom FTP providers to your FTP sites.
As a brief review from Part 1, the following XML excerpt illustrates what the provider's settings should resemble when added to your IIS settings:
<system.ftpServer>
<providerDefinitions>
<add name="FtpXmlAuthorization"
type="FtpXmlAuthorization, FtpXmlAuthorization, version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73" />
<activation>
<providerData name="FtpXmlAuthorization">
<add key="xmlFileName"
value="C:\inetpub\FtpUsers\Users.xml" />
</providerData>
</activation>
</providerDefinitions>
</system.ftpServer>
The above example shows the settings that are added globally to register an FTP provider. Note that this example only contains the settings for my custom provider; you would normally see the settings for the IisManagerAuth and AspNetAuth providers that ship with the FTP service in the providerDefinitions collection.
To actually use a provider for an FTP site, you would need to add the provider to the settings for the FTP site in your IIS settings. So for part 2 of this blog series, we will focus on how to add a custom provider to an FTP site by using the IIS Configuration Editor.
Having said all of that, the rest of this blog is broken down into the following sections:
Before continuing, I should reiterate that custom FTP providers fall into two categories: providers that are used for authentication and providers that are used for everything else. This distinction is important, because the settings are stored in different sections of your IIS settings. With that in mind, let's take a look at the settings for an example FTP site.
The following example shows an FTP site with several custom FTP providers added:
<site name="ftp.contoso.com" id="2">
<application path="/">
<virtualDirectory path="/"
physicalPath="c:\inetpub\www.contoso.com\wwwroot" />
</application>
<bindings>
<binding protocol="ftp"
bindingInformation="*:21:ftp.contoso.com" />
</bindings>
<ftpServer>
<security>
<ssl controlChannelPolicy="SslAllow"
dataChannelPolicy="SslAllow" />
<authentication>
<customAuthentication>
<providers>
<add name="MyCustomFtpAuthenticationProvider" />
</providers>
</customAuthentication>
</authentication>
</security>
<customFeatures>
<providers>
<add name="MyCustomFtpHomeDirectoryProvider" />
<add name="MyCustomFtpLoggingProvider" />
</providers>
</customFeatures>
<userIsolation mode="Custom" />
</ftpServer>
</site>
If you look at the above example, you will notice the following providers have been added:
As I mentioned earlier, you will notice that the settings for FTP custom providers are stored in different sections of the ftpServer collection depending on whether they are used for authentication or some other purpose.
Open the IIS Manager and click on the Configuration Editor at feature the server level:
Click the Section drop-down menu, expand the the system.applicationHost collection, and then highlight the sites node:
If you click on the Collection row, an ellipsis [...] will appear:
When you click the ellipsis [...], IIS will display the Collection Editor dialog box for your sites; both HTTP and FTP sites will be displayed:
Expand the ftpServer node, which is where all of the site-level settings for an FTP site are kept.
As I mentioned earlier, custom FTP providers fall into two categories: providers that are used for authentication and everything else. Because of this distinction, the following steps show you how to add a provider to the correct section of your settings depending on the provider's purpose.
Expand the customFeatures node, which is located under the ftpServer node for an FTP site; this collection defines the custom providers for an FTP site that are not used for authentication, for example: home directory providers, logging providers, etc. When you highlight the providers row, an ellipsis [...] will appear:
When you click the ellipsis [...], IIS will display the Collection Editor dialog box for your custom features (providers). When you click Add in the Actions pane, you need to enter the name of an FTP provider that you entered by following the instructions in Part 1 of this blog series:
Once you enter the name of your FTP provider in the Collection Editor dialog box for your custom features, you can close that dialog box. The Collection Editor for your sites will reflect the updated provider count for your FTP site:
Important Note: If you are adding a custom FTP Home Directory Provider, you have to configure the mode for FTP's User Isolation features. To do so, you need to expand the userIsolation node, which is located under the ftpServer node for an FTP site. Once you have done so, click the mode drop-down menu and choose Custom from the list of choices:
When you close the Collection Editor dialog box for your sites, you need to click Apply in the Actions pane to commit the changes to your IIS settings:
First and foremost - there is built-in support for adding custom authentication providers in IIS Manager; to see the steps to do so, see the FTP Custom Authentication <customAuthentication> article on the IIS.NET website. However, if you want to add a custom FTP authentication provider to an FTP site by using the IIS Configuration Editor, you can do so by using the following steps.
Expand the security node under the ftpServer node for an FTP site, then expand the authentication node, and then expand the customAuthentication node; this collection defines the custom authentication providers for an FTP site. When you highlight the providers row, an ellipsis [...] will appear:
When you click the ellipsis [...], IIS will display the Collection Editor dialog box for your custom authentication providers. When you click Add in the Actions pane, you need to enter the name of an FTP authentication provider that you entered by following the instructions in Part 1 of this blog series:
Once you enter the name of your FTP authentication provider in the Collection Editor dialog box for your custom authentication providers, you can close that dialog box. The Collection Editor for your sites will reflect the updated authentication provider count for your FTP site:
When you close the Collection Editor dialog box for your sites, you need to click Apply in the Actions pane to commit the changes to your IIS settings:
As I mentioned in part 1 of this series, I admit that this might seem like a lot of steps to go through, but it's not that difficult once you understand how the configuration settings are organized and you get the hang of using the IIS Configuration Editor to add or modify these settings.
In the Add a custom FTP provider to an FTP site that is not used for authentication section of this blog, I added a step to specify Custom as the User Isolation mode. Since this is something of an advanced feature, there is no user interface for enabling custom user isolation; this was a design decision to keep people from breaking their FTP sites. Here's why: if you enable custom user isolation and you don't install a custom Home Directory provider for FTP, all users will be denied access to your FTP site.
That being said, once you have enabled custom user isolation, the option to disable custom user isolation will "magically" appear in the FTP User Isolation feature in the IIS Manager. To see this for yourself, you would first need to follow the steps to custom user isolation in the Add a custom FTP provider to an FTP site that is not used for authentication section of this blog.
Once you have enabled custom user isolation, highlight your FTP site in the list of Sites pane of IIS Manager, then open the FTP User Isolation feature:
When you open the FTP User Isolation feature, you will see that an option for Custom now appears in the list of user isolation modes:
This option will appear as long as custom user isolation is enabled. If you change the user isolation mode to something other than Custom, this option will continue appear in the list of user isolation modes until you navigate somewhere else in IIS Manager. Once you have changed the user isolation mode to one of the built-in modes and you navigate somewhere else, the Custom option will not show up in the FTP User Isolation feature until you follow the steps to re-enable custom user isolation.
If you want additional information about configuring the settings for FTP providers, you can find detailed reference documentation at the following URLs:
Each of these articles contain "How-To" steps, detailed information about each of the configuration settings, and code samples for AppCmd.exe, C#/VB.NET, and JavaScript/VBScript.
As always, let me know if you have any questions. ;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
31 March 2013 • by Bob • Extensibility, FTP, IIS
I've written a lot of walkthroughs and blog posts about creating custom FTP providers over the past several years, and I usually include instructions for adding these custom providers to IIS. When you create a custom FTP authentication provider, IIS has a user interface for adding that provider to FTP. But if you are adding a custom home directory or logging provider, there is no dedicated user interface for adding those types of FTP providers. In addition, if you create a custom FTP provider that requires settings that are stored in your IIS configuration, there is no user interface to add or manage those settings.
With this in mind, I include instructions in my blogs and walkthroughs that describe how to add those type of providers by using AppCmd.exe from a command line. For example, if you take a look at my How to Use Managed Code (C#) to Create an FTP Authentication and Authorization Provider using an XML Database walkthrough, I include the following instructions:
Adding the Provider
cd %SystemRoot%\System32\Inetsrv
appcmd.exe set config -section:system.ftpServer/providerDefinitions /+"[name='FtpXmlAuthorization',type='FtpXmlAuthorization,FtpXmlAuthorization,version=1.0.0.0,Culture=neutral,PublicKeyToken=426f62526f636b73']" /commit:apphost
appcmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpXmlAuthorization']" /commit:apphost
appcmd.exe set config -section:system.ftpServer/providerDefinitions /+"activation.[name='FtpXmlAuthorization'].[key='xmlFileName',value='C:\Inetpub\XmlSample\Users.xml']" /commit:apphost
This example adds a custom FTP provider, and then it adds a custom setting for that provider that is stored in your IIS configuration settings.
That being said, there is actually a way to add custom FTP providers with settings like the ones that I have just described through the IIS interface by using the IIS Configuration Editor. This feature was first available through the IIS Administration Pack for IIS 7.0, and is built-in for IIS 7.5 and IIS 8.0.
Before I continue, if would probably be prudent to take a look at the settings that we are trying to add, because these settings might help you to understand the rest of steps in this blog. Here is an example from my applicationhost.config file for three custom FTP authentication providers; the first two providers are installed with the FTP service, and the third provider is a custom provider that I created with a single provider-specific configuration setting:
<system.ftpServer>
<providerDefinitions>
<add name="IisManagerAuth" type="Microsoft.Web.FtpServer.Security.IisManagerAuthenticationProvider, Microsoft.Web.FtpServer, version=7.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
<add name="AspNetAuth" type="Microsoft.Web.FtpServer.Security.AspNetFtpMembershipProvider, Microsoft.Web.FtpServer, version=7.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
<add name="FtpXmlAuthorization" type="FtpXmlAuthorization, FtpXmlAuthorization, version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73" />
<activation>
<providerData name="FtpXmlAuthorization">
<add key="xmlFileName" value="C:\inetpub\FtpUsers\Users.xml" />
</providerData>
</activation>
</providerDefinitions>
</system.ftpServer>
With that in mind, in part 1 of this blog series, I will show you how to use the IIS Configuration Editor to add a custom FTP provider with provider-specific configuration settings.
Step 1 - Open the IIS Manager and click on the Configuration Editor at feature the server level:
Step 2 - Click the Section drop-down menu, expand the the system.ftpServer collection, and then highlight the providerDefinitions node:
Step 3 - A default installation IIS with the FTP service should show a Count of 2 providers in the Collection row, and no settings in the activation row:
Step 4 - If you click on the Collection row, an ellipsis [...] will appear, and when you click that, IIS will display the Collection Editor dialog for FTP providers. By default you should see just the two built-in providers for the IisManagerAuth and AspNetAuth providers:
Step 5 - When you click Add in the Actions pane, you can enter the registration information for your provider. At a minimum you must provide a name for your provider, but you will need to enter either the clsid for a COM-based provider or the type for a managed-code provider:
Step 6 - When you close the Collection Editor dialog, the Count of providers in the Collection should now reflect the provider that we just added; click Apply in the Actions pane to save the changes:
Step 7 - If you click on the activation row, an ellipsis [...] will appear, and when you click that, IIS will display the Collection Editor dialog for provider data; this is where you will enter provider-specific settings. When you click Add in the Actions pane, you must specify the name for your provider's settings, and this name must match the exact name that you provided in Step 5 earlier:
Step 8 - If you click on the Collection row, an ellipsis [...] will appear, and when you click that, IIS will display the Collection Editor dialog for the activation data for an FTP provider. At a minimum you must provide a key for your provider, which will depend on the settings that your provider expects to retrieve from your configuration settings. (For example, in the XML file that I provided earlier, my FtpXmlAuthorization
provider expects to retrieve the path to an XML that contains a list of users, roles, and authorization rules.) You also need to enter the either the value or encryptedValue for your provider; although you can specify either setting, should generally specify the value when the settings are not sensitive in nature, and specify the encryptedValue for settings like usernames and passwords:
Step 9 - When you close the Collection Editor dialog for the activation data, the Count of key/value pairs in the Collection should now reflect the value that we just added:
Step 10 - When you close the Collection Editor dialog for the provider data, the Count of provider data settings in the activation row should now reflect the custom settings that we just added; click Apply in the Actions pane to save the changes:
That's all that there is to adding a custom FTP provider with provider-specific settings; I admit that it might seem like a lot of steps until you get the hang of it.
In the next blog for this series, I will show you how to add custom providers to FTP sites by using the IIS Configuration Editor.
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
28 March 2013 • by Bob • IIS, Scripting, FTP
A customer asked me a question a little while ago that provided me the opportunity to recycle some code that I had written many years ago. In so doing, I also made a bunch of updates to the code to make it considerably more useful, and I thought that it would make a great blog.
Here's the scenario: a customer had hundreds of user accounts created, and he wanted to use the FTP service's User Isolation features to restrict each user to a specific folder on his FTP site. Since it would take a long time to manually create a folder for each user account, the customer wanted to know if there was a way to automate the process. As it turns out, I had posted a very simple script in the IIS.net forums several years ago that did something like what he wanted; and that script was based off an earlier script that I had written for someone else back in the IIS 6.0 days.
One quick reminder - FTP User Isolation uses a specific set of folders for user accounts, which are listed in the table below.
User Account Types | Home Directory Syntax |
---|---|
Anonymous users | %FtpRoot%\LocalUser\Public |
Local Windows user accounts
(Requires Basic authentication.) |
%FtpRoot%\LocalUser\%UserName% |
Windows domain accounts
(Requires Basic authentication.) |
%FtpRoot%\%UserDomain%\%UserName% |
Note: %FtpRoot% is the root directory for your FTP site: for example, C:\Inetpub\Ftproot.
That being said, I'm a big believer in recycling code, so I found the last version of that script that I gave to someone and I made a bunch of changes to it so it would be more useful for the customer. What that in mind, here's the resulting script, and I'll explain a little more about what it does after the code sample.
Option Explicit ' Define the root path for the user isolation folders. ' This should be the root directory for your FTP site. Dim strRootPath : strRootPath = "C:\Inetpub\wwwroot\" ' Define the name of the domain or the computer to use. ' Leave this blank for the local computer. Dim strComputerOrDomain : strComputerOrDomain = "" ' Define the remaining script variables. Dim objFSO, objCollection, objUser, objNetwork, strContainerName ' Create a network object; used to query the computer name. Set objNetwork = WScript.CreateObject("WScript.Network") ' Create a file system object; used to creat folders. Set objFSO = CreateObject("Scripting.FileSystemObject") ' Test if the computer name is null. If Len(strComputerOrDomain)=0 Or strComputerOrDomain="." Then ' If so, define the local computer name as the account repository. strComputerOrDomain = objNetwork.ComputerName End If ' Verify that the root path exists. If objFSO.FolderExists(strRootPath) Then ' Test if the script is using local users. If StrComp(strComputerOrDomain,objNetwork.ComputerName,vbTextCompare)=0 Then ' If so, define the local users container path. strContainerName = "LocalUser" ' And define the users collection as local. Set objCollection = GetObject("WinNT://.") Else ' Otherwise, use the source name as the path. strContainerName = strComputerOrDomain ' And define the users collection as remote. Set objCollection = GetObject("WinNT://" & strComputerOrDomain & "") End If ' Append trailing backslash if necessary. If Right(strRootPath,1)<>"\" Then strRootPath = strRootPath & "\" ' Define the adjusted root path for the container folder. strRootPath = strRootPath & strContainerName & "\" ' Test if the container folder already exists. If objFSO.FolderExists(strRootPath)=False Then ' Create the container folder if necessary. objFSO.CreateFolder(strRootPath) End If ' Specify the collection filter for user objects only. objCollection.Filter = Array("user") ' Loop through the users collection. For Each objUser In objCollection ' Test if the user's account is enabled. If objUser.AccountDisabled = False Then ' Test if the user's folder already exists. If objFSO.FolderExists(strRootPath & "\" & objUser.Name)=False Then ' Create the user's folder if necessary. objFSO.CreateFolder(strRootPath & "\" & objUser.Name) End If End If Next End If
I documented this script in great detail, so it should be self-explanatory for the most part. But just to be on the safe side, here's an explanation of what this script is doing when you run it on your FTP server:
That's all for now. ;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
31 January 2013 • by Bob • FTP
For this installment in my series about FTP clients, I want to take a look at BitKinex 3, which is an FTP client from Barad-Dur, LLC. For this blog I used BitKinex 3.2.3, and it is available from the following URL:
At the time of this blog post, BitKinex 3 is available for free, and it contains a bunch of features that make it an appealing FTP and WebDAV client.
![]() |
Fig. 1 - The Help/About dialog in BitKinex 3. |
When you open BitKinex 3, it shows four connection types (which it refers to as Data Sources): FTP, HTTP/WebDAV, SFTP/SSH, and My Computer. The main interface is analogous to what you would expect in a Site Manager with other FTP clients - you can define new data sources (connections) to FTP sites and websites:
![]() |
Fig. 2 - The main BitKinex 3 window. |
Creating an FTP data source is pretty straight-forward, and there are a fair number of options that you can specify. What's more, data sources can have individual options specified, or they can inherit from a parent note.
![]() |
Fig. 3 - Creating a new FTP data source. |
![]() |
Fig. 4 - Specifying the options for an FTP data source. |
Once a data source has connected, a child window will open and display the folder trees for your local and remote content. (Note: there are several options for customizing how each data source can be displayed.)
![]() |
Fig. 5 - An open FTP data source. |
BitKinex 3 has support for command-line automation, which is pretty handy if you do a lot of scripting like I do. Documentation about automating BitKinex 3 from the command line is available on the BitKinex website at the following URL:
BitKinex Command Line Interface
That being said, the documentation is a bit sparse and there are few examples, so I didn't attempt anything ambitious from a command line during my testing.
BitKinex 3 has built-in support for FTP over SSL (FTPS) supports both Explicit and Implicit FTPS. To specify the FTPS mode, you need to choose the correct mode from the Security drop-down menu for your FTP data source.
![]() |
Fig. 6 - Specifying the FTPS mode. |
Once you have established an FTPS connection through BitKinex 3, the user experience is the same as it is for a standard FTP connection.
True FTP hosts are not supported natively, and even though BitKinex 3 allows you to send a custom command after a data source has been opened, I could not find a way to send a custom command before sending user credentials, so true FTP hosts cannot be used.
BitKinex 3's login settings allow you to specify the virtual host name as part of the user credentials by using syntax like "ftp.example.com|username" or "ftp.example.com\username", so you can use virtual FTP hosts with BitKinex 3.
![]() |
Fig. 7 - Specifying an FTP virtual host. |
This concludes my quick look at a few of the FTP features that are available with BitKinex 3, and here are the scorecard results:
Client Name | Directory Browsing | Explicit FTPS | Implicit FTPS | Virtual Hosts | True HOSTs | Site Manager | Extensibility |
---|---|---|---|---|---|---|---|
BitKinex 3.2.3 | Rich | Y | Y | Y | N | Y | N/A |
Note: I could not find anyway to extend the functionality of BitKinex 3; but as I mentioned earlier, it does support command-line automation. |
That wraps it up this blog - BitKinex 3 is pretty cool FTP client with a lot of options, and I think that my next plan of action is to try out the WebDAV features that are available in BitKinex 3. ;-)
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
31 January 2013 • by Bob • Scripting, FTP, Extensibility
I was recently creating a new authentication provider using FTP extensibility, and I ran into a weird behavior that I had seen before. With that in mind, I thought my situation would make a great blog subject because someone else may run into it.
Here are the details of the situation: let's say that you are developing a new FTP provider for IIS, and your code changes never seem to take effect. Your provider appears to be working, it's just that any new functionality is not reflected in your provider's behavior. You restart the FTP service as a troubleshooting step, but that does not appear to make any difference.
I'll bypass mentioning any other troubleshooting tasks and cut to the chase - if you read my Changing the Identity of the FTP 7 Extensibility Process blog post a year ago, you will recall that I mentioned that all custom FTP extensibility providers are executed through COM+ in a DLLHOST.exe process. When you restart the FTP service, that should clean up the DLLHOST.EXE process that is being used for FTP extensibility. However, if you are developing custom FTP providers and the DLLHOST.EXE process is not terminated by the FTP service, you may find yourself in a situation where you have a DLLHOST.EXE process in memory that contains an older copy of your provider, which will not be removed from memory until the DLLHOST.EXE process for FTP extensibility has been forcibly terminated.
If you have read some of my earlier blog posts or walkthroughs on IIS.NET, you may have noticed that I generally like to use a few pre-build and post-build commands in my FTP projects; usually I add these commands in order to to automatically register/unregister my FTP providers in the Global Assembly Cache (GAC).
With a little modification and some command-line wizardry, you can automate the termination of any orphaned DLLHOST.EXE processes that are being used for FTP extensibility. With that in mind, here are some example pre-build/post-build commands that will unregister/reregister your provider in the GAC, restart the FTP service, and terminate any orphaned FTP extensibility DLLHOST.EXE processes.
Note: The following syntax was written using Visual Studio 2010; you would need to change "%VS100COMNTOOLS%" to "%VS90COMNTOOLS%" for Visual Studio 2008 or "%VS110COMNTOOLS%" for Visual Studio 2012.
Pre-build Commands:
net stop ftpsvc
call "%VS100COMNTOOLS%\vsvars32.bat">nul
cd /d "$(TargetDir)"
gacutil.exe /uf "$(TargetName)"
for /f "usebackq tokens=1,2* delims=," %%a in (`tasklist /fi "MODULES eq Microsoft.Web.FtpServer.*" /fi "IMAGENAME eq DLLHOST.EXE" /fo csv ^| find /i "dllhost.exe"`) do taskkill /f /pid %%b
Post-build Commands:
call "%VS100COMNTOOLS%\vsvars32.bat">nul
gacutil.exe /if "$(TargetPath)"
net start ftpsvc
The syntax is a little tricky for the FOR statement, so be carefully when typing or copying/pasting that into your projects. For example, you need to make sure that all of the code from the FOR statement through the TASKKILL command are on the same line in your project's properties.
When you compile your provider, Visual Studio should display something like the following:
------ Rebuild All started: Project: FtpBlogEngineNetAuthentication, Configuration: Release Any CPU ------
The Microsoft FTP Service service is stopping.
The Microsoft FTP Service service was stopped successfully.
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.1
Copyright (c) Microsoft Corporation. All rights reserved.
Assembly: FtpBlogEngineNetAuthentication, Version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73, processorArchitecture=MSIL
Uninstalled: FtpBlogEngineNetAuthentication, Version=1.0.0.0, Culture=neutral, PublicKeyToken=426f62526f636b73, processorArchitecture=MSIL
Number of assemblies uninstalled = 1
Number of failures = 0
SUCCESS: The process with PID 12656 has been terminated.
FtpBlogEngineNetAuthentication -> C:\Users\dude\Documents\Visual Studio 2010\Projects\FtpBlogEngineNetAuthentication\FtpBlogEngineNetAuthentication\bin\Release\FtpBlogEngineNetAuthentication.dll
Microsoft (R) .NET Global Assembly Cache Utility. Version 4.0.30319.1
Copyright (c) Microsoft Corporation. All rights reserved.
Assembly successfully added to the cache
The Microsoft FTP Service service is starting.
The Microsoft FTP Service service was started successfully.
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========
If you analyze the output from the build process, you will see that the commands in my earlier samples stopped the FTP service, removed the existing assembly from the GAC, terminated any orphaned DLLHOST.EXE processes, registered the newly-built DLL in the GAC, and then restarted the FTP service.
By utilizing these pre-build/post-build commands, I have been able to work around situations where a DLLHOST.EXE process is being orphaned and caching old assemblies in memory.
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
30 November 2012 • by Bob • FTP, SSL
For this installment in my series about FTP clients, I want to take a look at Beyond Compare 3 from Scooter Software. At its heart, Beyond Compare is a file/folder comparison tool, so it might seem an unlikely candidate for an FTP client, but it has a lot of great FTP features packed into it.
![]() |
Fig. 1 - The Help/About dialog in Beyond Compare 3. |
Note: For this blog I used Beyond Compare version 3.3.5.
Like many self-proclaimed computer geeks, over the years I have collected a lot of various utilities that perform specific actions that I need to take care of. Sometimes I discover these tools when Binging my way through the Internet, and other times they come highly recommended from other people. In this specific situation, Beyond Compare falls into the latter category - dozens of people had recommended Beyond Compare to me before I tried it out, and after falling in love with it I have recommended it to dozens of my friends. At the time I was using Microsoft WinDiff to compare files, which is still a great application to do simple comparisons, but Beyond Compare does so much more.
![]() |
Fig. 2 - The Start New Session screen. |
![]() |
Fig. 3 - Comparing the files within two folders. |
![]() |
Fig. 4 - Comparing the HTML content of two files. |
I could go on about Beyond Compare as a comparison tool, but that's really outside the scope of this blog since I am supposed to be talking about FTP features. Needless to say, if you're looking for a good comparison tool, you might want to download the trial edition of Beyond Compare 3 and give it a try.
That being said, let's get back to the business at hand. Beyond Compare 3 has a collection of FTP Profiles, which you can think of as analogous to a site manager in more traditional FTP clients.
![]() |
Fig. 5 - Opening Beyond Compare 3's FTP Profiles. |
Inside the FTP Profiles dialog, you can specify a wealth of connection options for remote FTP sites that you would expect to find in any other FTP client.
![]() |
Fig. 6 - Specifying FTP connection options. |
Once you have established an FTP connection through Beyond Compare 3, you can view your local files and the files in your remote FTP site side-by-side, and then you can perform comparisons, updates, merges, etc.
![]() |
Fig. 7 - Viewing local and remote files. |
Beyond Compare 3 has built-in support for Explicit FTP over SSL (FTPS), which you specify when you are creating the FTP profile for a site.
![]() |
Fig. 8 - Specifying an Explicit FTPS connection. |
Once you have established an Explicit FTPS connection through Beyond Compare 3, the user experience is the same as it is for a standard FTP connection.
![]() |
Fig. 9 - Comparing files over FTPS. |
That being said, at first glance Beyond Compare 3 did not appear to support Implicit FTPS. For me this was not a deal-breaker by any stretch of the imagination since Explicit FTPS is preferred. (Even though Implicit FTPS is supported by IIS7 through IIS8, it is really an outdated protocol.)
10 January 2013 Update: I heard from Craig Peterson at Scooter Software that Beyond Compare 3 does support Implicit FTPS, but it does so implicitly. (No pun intended. ;-]) When you connect using FTP over SSL on port 990, it will automatically use implicit FTPS.
Beyond Compare 3 has built-in support for the HOST command, so you can use true FTP host names when using Beyond Compare 3 to connect to FTP7 and FTP8 sites that are configured with host names. This feature is enabled by default, but if you needed to disable it for some reason, that feature can be accessed on the Connection tab of Beyond Compare 3's FTP Profiles dialog.
![]() |
Fig. 10 - Specifying support for the FTP HOST command. |
Beyond Compare 3's login settings allow you to specify the virtual host name as part of the user credentials by using syntax like "ftp.example.com|username" or "ftp.example.com\username", but since Beyond Compare 3 allows you to use true FTP hosts this is really a moot point. Just the same, there's nothing to stop you from disabling the HOST command for a connection and specifying an FTP virtual host as part of your username, although I'm not sure why you would want to do that.
![]() |
Fig. 11 - Specifying a virtual FTP host. |
This concludes our quick look at some of the FTP features that are available with Beyond Compare 3, and here are the scorecard results:
Client Name | Directory Browsing | Explicit FTPS | Implicit FTPS | Virtual Hosts | True HOSTs | Site Manager | Extensibility |
---|---|---|---|---|---|---|---|
Beyond Compare 3.3.5 | Rich | Y | Y | Y | Y | Y | N/A 1 |
As noted earlier, Beyond Compare 3 supports the FTP HOST command, and is enabled by default for new connections. 1 Note: I could not find anyway to extend the functionality of Beyond Compare 3, but it does have a scripting interface; see their Automating with Scripts and Scripting Reference pages for more details. |
So there you have it - Beyond Compare 3 contains many of the features that would make up a great GUI-based FTP client with first-class support for all of the features that I have been examining in detail throughout my blog series about FTP clients. And as I have done with all of my blogs thus far, I included the following disclaimer in all of my preceding posts: there are a great number of additional features that Beyond Compare 3 provides - but once again I only focused on a few specific topic areas that apply to FTP7 and FTP8. For example, one particular feature that I might to experiment with in the future is Beyond Compare 3's support for FTP SSL Client Certificates. But I'll leave that for another day. ;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
03 October 2012 • by Bob • IIS, Scripting, FTP, Extensibility, IIS, Scripting, FTP, Extensibility
I was recently contacted by someone who was trying to use Windows Management Instrumentation (WMI) code to stop and restart FTP websites by using code that he had written for IIS 6.0; his code was something similar to the following:
Option Explicit On Error Resume Next Dim objWMIService, colItems, objItem ' Attach to the IIS service. Set objWMIService = GetObject("winmgmts:\root\microsoftiisv2") ' Retrieve the collection of FTP sites. Set colItems = objWMIService.ExecQuery("Select * from IIsFtpServer") ' Loop through the sites collection. For Each objItem in colItems ' Restart one single website. If (objItem.Name = "MSFTPSVC/1") Then Err.Clear objItem.Stop If (Err.Number <> 0) Then WScript.Echo Err.Number objItem.Start If (Err.Number <> 0) Then WScript.Echo Err.Number End If Next
The problem that the customer was seeing is that this query did not return the list of FTP-based websites for IIS 7.0 or IIS 7.5 (called IIS7 henceforth), although changing the class in the query from IIsFtpServer to IIsWebServer would make the script work with HTTP-based websites those versions of IIS7.
The problem with the customer's code was that he is using WMI to manage IIS7; this relies on our old management APIs that have been deprecated, although part of that model is partially available through the metabase compatibility feature in IIS7. Here's what I mean by "partially": only a portion of the old ADSI/WMI objects are available, and unfortunately FTP is not part of the objects that can be scripted through the metabase compatibility feature in IIS7.
That being said, what the customer wants to do is still possible through scripting in both IIS7 and IIS8, and the following sample shows how to loop through all of the sites, determine which sites have FTP bindings, and then stop/start FTP for each site. To use this script, copy the code into a text editor like Windows Notepad and save it with a name like "RestartAllFtpSites.vbs" to your system, then double-click the file to run it.
' Temporarily disable breaking on runtime errors. On Error Resume Next ' Create an Admin Manager object. Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager") adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST" ' Test for commit path support. If Err.Number <> 0 Then Err.Clear ' Create a Writable Admin Manager object. Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager") adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST" If Err.Number <> 0 Then WScript.Quit End If ' Resume breaking on runtime errors. On Error Goto 0 ' Retrieve the sites collection. Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST") Set sitesCollection = sitesSection.Collection ' Loop through the sites collection. For siteCount = 0 To CInt(sitesCollection.Count)-1 isFtpSite = False ' Determine if the current site is an FTP site by checking the bindings. Set siteElement = sitesCollection(siteCount) Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection For bindingsCount = 0 To CInt(bindingsCollection.Count)-1 Set bindingElement = bindingsCollection(bindingsCount) If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then isFtpSite = True Exit For End If Next ' If it's an FTP site, start and stop the site. If isFtpSite = True Then Set ftpServerElement = siteElement.ChildElements.Item("ftpServer") ' Create an instance of the Stop method. Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance() ' Execute the method to stop the FTP site. stopFtpSite.Execute() ' Create an instance of the Start method. Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance() ' Execute the method to start the FTP site. startFtpSite.Execute() End If Next
And the following code sample shows how to stop/start a single FTP site. To use this script, copy the code into a text editor like Windows Notepad, rename the site name appropriately for one of your FTP sites, save it with a name like "RestartContosoFtpSite.vbs" to your system, then double-click the file to run it.
' Temporarily disable breaking on runtime errors. On Error Resume Next ' Create an Admin Manager object. Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager") adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST" ' Test for commit path support. If Err.Number <> 0 Then Err.Clear ' Create a Writable Admin Manager object. Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager") adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST" If Err.Number <> 0 Then WScript.Quit End If ' Resume breaking on runtime errors. On Error Goto 0 ' Retrieve the sites collection. Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST") Set sitesCollection = sitesSection.Collection ' Locate a specific site. siteElementPos = FindElement(sitesCollection, "site", Array("name", "ftp.contoso.com")) If siteElementPos = -1 Then WScript.Echo "Site was not found!" WScript.Quit End If ' Determine if the selected site is an FTP site by checking the bindings. Set siteElement = sitesCollection(siteElementPos) Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection For bindingsCount = 0 To CInt(bindingsCollection.Count)-1 Set bindingElement = bindingsCollection(bindingsCount) If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then isFtpSite = True Exit For End If Next ' If it's an FTP site, start and stop the site. If isFtpSite = True Then Set ftpServerElement = siteElement.ChildElements.Item("ftpServer") ' Create an instance of the Stop method. Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance() ' Execute the method to stop the FTP site. stopFtpSite.Execute() ' Create an instance of the Start method. Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance() ' Execute the method to start the FTP site. startFtpSite.Execute() End If ' Locate and return the index for a specific element in a collection. Function FindElement(collection, elementTagName, valuesToMatch) For i = 0 To CInt(collection.Count) - 1 Set elem = collection.Item(i) If elem.Name = elementTagName Then matches = True For iVal = 0 To UBound(valuesToMatch) Step 2 Set prop = elem.GetPropertyByName(valuesToMatch(iVal)) value = prop.Value If Not IsNull(value) Then value = CStr(value) End If If Not value = CStr(valuesToMatch(iVal + 1)) Then matches = False Exit For End If Next If matches Then Exit For End If End If Next If matches Then FindElement = i Else FindElement = -1 End If End Function
I hope this helps!
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
01 September 2012 • by Bob • FTP, IIS, Windows
The folks in the TechEd group have uploaded the video from my "What's New with Internet Information Services (IIS) 8: Performance, Scalability, and Security Features" presentation to YouTube, so you can view the video online.
You can also download the slides and the WMV/MP4 for my presentation at the following URL:
http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/WSV332
One quick side note: around 38:55 during the video, I had just asked the audience if anyone had used the IIS Configuration Editor, when a tremendous thunderclap resounded outside - this prompted a great laugh from audience members. After the presentation had ended, a couple people came up and jokingly asked how I had managed to stage that so well.
28 August 2012 • by Bob • FTP, Extensibility
I recently received a question from a customer about troubleshooting custom FTP providers, and I recommended using the FTP service's Event Tracing for Windows (ETW) features in order to help troubleshoot the problem. I've helped a lot of customers use this little-known feature of the FTP service, so I thought that it would make a great subject for a quick blog.
By way of explanation, the FTP service in IIS 7.5 and IIS 8.0 allows developers to write their own custom functionality, and over the past several years I have written several walkthroughs and blogs that illustrate how you can create your own custom FTP providers:
That being said, sometimes things go wrong, and when that happens, I use some FTP ETW troubleshooting tricks that I'd like to share.
Several years ago I wrote a blog about FTP and ETW Tracing, where I described how to turn on the FTP service's ETW tracing through a batch file, and then it used Log Parser to render the output in a datagrid for analysis. In the interests of completeness, here is the batch file again:
@echo off rem====================================================================== echo Verifying that LogParser.exe is in the path... LogParser -h >nul 2>nul if errorlevel 1 ( echo. echo Error: echo. echo LogParser.exe is was not found. It is required for parsing traces. echo. echo Recommended actions: echo. echo - If LogParser is installed then fix the PATH echo variable to include the LogParser directory echo. echo - If LogParser is not installed, then install echo it from the following location: echo. echo http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07 echo. goto :EOF ) else ( echo Done. echo. ) rem====================================================================== echo Starting the ETW session for full FTP tracing... logman start "ftp" -p "IIS: Ftp Server" 255 5 -ets echo. echo Now reproduce your problem. echo. echo After you have reproduced your issue, hit any key to close the FTP echo tracing session. Your trace events will be displayed automatically. echo. pause>nul rem====================================================================== echo. echo Closing the ETW session for full FTP tracing... logman stop "ftp" -ets rem====================================================================== echo. echo Parsing the results - this may take a long time depending on the size of the trace... LogParser "select EventTypeName, UserData from ftp.etl" -e 2 -o:DATAGRID -compactModeSep " | " -rtp 20 |
When you save and run this batch file, it will display something like the following:
C:\FTP_ETW.cmd Verifying that LogParser.exe is in the path... Done. Starting the ETW session for full FTP tracing... The command completed successfully. Now reproduce your problem. After you have reproduced your issue, hit any key to close the FTP tracing session. Your trace events will be displayed automatically. |
When you see this displayed, you will need to reproduce your problem, and FTP's ETW tracing will record the troubleshooting information.
Once you have reproduced your problem, hit a key to end the ETW session, and you will see the following message displayed:
Closing the ETW session for full FTP tracing... The command completed successfully. Parsing the results - this may take a long time depending on the size of the trace... |
The batch file will eventually call Log Parser to parse the ETW events, and a dialog like the following will be displayed:
Now that you know how to set up FTP's ETW tracing, let's examine what you should be looking for in the tracing information.In all of the examples in this blog, I am using the XML-based authentication provider that is documented in the How to Use Managed Code (C#) to Create an FTP Authentication Provider using an XML Database walkthrough.
The following illustration highlights several lines that show the FTP service starting its authentication process, loading my custom authentication provider, and ending the authentication process after I have successfully logged in:
This example shows what everything looks like when it works as expected, so now let's look at what happens when something goes wrong.
If I use the same provider, but I enter my username or password incorrectly, I will see the following lines in the trace:
This example informs you that the provider was loaded successfully, but the logon failed. The error code that is returned is 0x8007052E - this hexadecimal 32-bit value can be split into 16-bit values:
Logon failure: unknown user name or bad password.
"If I continue to use the same provider as earlier, and I delete the XML file that my provider uses, then I will receive the following error:
Once again, this example informs you that the provider was loaded successfully, but an error occurred. In this specific case you see the actual details that the XML file exists, and that is an error that is returned by a throw() statement in the provider. The error code that is returned is 0x80070057 - and once again this hexadecimal 32-bit value can be split into 16-bit values:
The parameter is incorrect.
"If I replace the missing XML file for the provider, but I remove all of the permissions to the file, I get the following error:
As in the previous examples, this informs you that the provider was loaded successfully, but an error occurred. You can't look up the 0x80131500 error code by using "NET HELPMSG" from a command-prompt, but that doesn't matter since the error description informs you of the problem - access to the path where the file is located was denied.
If I enter a bad provider name, I get the following error:
Unlike the previous examples, this informs you that the provider was not loaded successfully. The description for this error informs you that it could not load the provider, and it gives you the assembly information. In addition to the error description, the error code that is returned by the FTP service is 0x80070002 - and once again this hexadecimal 32-bit value can be split into 16-bit values:
The system cannot find the file specified.
"So now let's look at a common perplexing problem:
This example shows the same 0x8007052E error code that we looked at in a previous example, but you'll notice that any reference to the provider is conspicuously absent from the trace - this means that the FTP service made no attempt to load the custom authentication provider. In this specific case, even though I had correctly registered my custom FTP authentication provider on the system, I had not added or enabled the custom authentication provider for my FTP site.
In this blog I showed you how to troubleshoot several different errors with FTP custom authentication providers by using FTP's ETW features.
As a parting thought, I should point out that the most-common error that I run into when creating my own providers is the last example. Believe it or not, I nearly always miss a step when I am creating a new provider and I forget to add a setting here or there which will cause the FTP service to completely ignore my provider. A perfect example is when I am writing custom home directory providers - I always remember to add the provider to the global list of FTP providers, and I usually remember to add the provider to the list of custom features for my FTP site, but I forget to configure my FTP site to use custom user isolation and my provider is ignored. (Darn, darn, darn...)
;-]
Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/