www.geekybob.com

Just a short, simple blog for Bob to share his thoughts.

Programmatically Starting and Stopping FTP Sites in IIS 7 and IIS 8

03 October 2012 • by Bob • IIS, Scripting, FTP, Extensibility, IIS, Scripting, FTP, Extensibility

I was recently contacted by someone who was trying to use Windows Management Instrumentation (WMI) code to stop and restart FTP websites by using code that he had written for IIS 6.0; his code was something similar to the following:

Option Explicit
On Error Resume Next

Dim objWMIService, colItems, objItem

' Attach to the IIS service.
Set objWMIService = GetObject("winmgmts:\root\microsoftiisv2")
' Retrieve the collection of FTP sites.
Set colItems = objWMIService.ExecQuery("Select * from IIsFtpServer")
' Loop through the sites collection.
For Each objItem in colItems
    ' Restart one single website.
    If (objItem.Name = "MSFTPSVC/1") Then
        Err.Clear
        objItem.Stop
        If (Err.Number <> 0) Then WScript.Echo Err.Number
        objItem.Start
        If (Err.Number <> 0) Then WScript.Echo Err.Number
    End If
Next

The problem that the customer was seeing is that this query did not return the list of FTP-based websites for IIS 7.0 or IIS 7.5 (called IIS7 henceforth), although changing the class in the query from IIsFtpServer to IIsWebServer would make the script work with HTTP-based websites those versions of IIS7.

The problem with the customer's code was that he is using WMI to manage IIS7; this relies on our old management APIs that have been deprecated, although part of that model is partially available through the metabase compatibility feature in IIS7. Here's what I mean by "partially": only a portion of the old ADSI/WMI objects are available, and unfortunately FTP is not part of the objects that can be scripted through the metabase compatibility feature in IIS7.

That being said, what the customer wants to do is still possible through scripting in both IIS7 and IIS8, and the following sample shows how to loop through all of the sites, determine which sites have FTP bindings, and then stop/start FTP for each site. To use this script, copy the code into a text editor like Windows Notepad and save it with a name like "RestartAllFtpSites.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Loop through the sites collection.
For siteCount = 0 To CInt(sitesCollection.Count)-1
    isFtpSite = False
    ' Determine if the current site is an FTP site by checking the bindings.
    Set siteElement = sitesCollection(siteCount)
    Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
    For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
        Set bindingElement = bindingsCollection(bindingsCount)
        If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
            isFtpSite = True
            Exit For
        End If
    Next
    ' If it's an FTP site, start and stop the site.
    If isFtpSite = True Then
        Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
        ' Create an instance of the Stop method.
        Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
        ' Execute the method to stop the FTP site.
        stopFtpSite.Execute()
        ' Create an instance of the Start method.
        Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
        ' Execute the method to start the FTP site.
        startFtpSite.Execute()
    End If
Next

And the following code sample shows how to stop/start a single FTP site. To use this script, copy the code into a text editor like Windows Notepad, rename the site name appropriately for one of your FTP sites, save it with a name like "RestartContosoFtpSite.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Locate a specific site.
siteElementPos = FindElement(sitesCollection, "site", Array("name", "ftp.contoso.com"))
If siteElementPos = -1 Then
    WScript.Echo "Site was not found!"
    WScript.Quit
End If

' Determine if the selected site is an FTP site by checking the bindings.
Set siteElement = sitesCollection(siteElementPos)
Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
    Set bindingElement = bindingsCollection(bindingsCount)
    If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
        isFtpSite = True
        Exit For
    End If
Next

' If it's an FTP site, start and stop the site.
If isFtpSite = True Then
    Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
    ' Create an instance of the Stop method.
    Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
    ' Execute the method to stop the FTP site.
    stopFtpSite.Execute()
    ' Create an instance of the Start method.
    Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
    ' Execute the method to start the FTP site.
    startFtpSite.Execute()
End If

' Locate and return the index for a specific element in a collection.
Function FindElement(collection, elementTagName, valuesToMatch)
   For i = 0 To CInt(collection.Count) - 1
      Set elem = collection.Item(i)
      If elem.Name = elementTagName Then
         matches = True
         For iVal = 0 To UBound(valuesToMatch) Step 2
            Set prop = elem.GetPropertyByName(valuesToMatch(iVal))
            value = prop.Value
            If Not IsNull(value) Then
               value = CStr(value)
            End If
            If Not value = CStr(valuesToMatch(iVal + 1)) Then
               matches = False
               Exit For
            End If
         Next
         If matches Then
            Exit For
         End If
      End If
   Next
   If matches Then
      FindElement = i
   Else
      FindElement = -1
   End If
End Function

I hope this helps!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Programmatically Starting and Stopping FTP Sites in IIS 7 and IIS 8

03 October 2012 • by Bob • IIS, Scripting, FTP, Extensibility, IIS, Scripting, FTP, Extensibility

I was recently contacted by someone who was trying to use Windows Management Instrumentation (WMI) code to stop and restart FTP websites by using code that he had written for IIS 6.0; his code was something similar to the following:

Option Explicit
On Error Resume Next

Dim objWMIService, colItems, objItem

' Attach to the IIS service.
Set objWMIService = GetObject("winmgmts:\root\microsoftiisv2")
' Retrieve the collection of FTP sites.
Set colItems = objWMIService.ExecQuery("Select * from IIsFtpServer")
' Loop through the sites collection.
For Each objItem in colItems
    ' Restart one single website.
    If (objItem.Name = "MSFTPSVC/1") Then
        Err.Clear
        objItem.Stop
        If (Err.Number <> 0) Then WScript.Echo Err.Number
        objItem.Start
        If (Err.Number <> 0) Then WScript.Echo Err.Number
    End If
Next

The problem that the customer was seeing is that this query did not return the list of FTP-based websites for IIS 7.0 or IIS 7.5 (called IIS7 henceforth), although changing the class in the query from IIsFtpServer to IIsWebServer would make the script work with HTTP-based websites those versions of IIS7.

The problem with the customer's code was that he is using WMI to manage IIS7; this relies on our old management APIs that have been deprecated, although part of that model is partially available through the metabase compatibility feature in IIS7. Here's what I mean by "partially": only a portion of the old ADSI/WMI objects are available, and unfortunately FTP is not part of the objects that can be scripted through the metabase compatibility feature in IIS7.

That being said, what the customer wants to do is still possible through scripting in both IIS7 and IIS8, and the following sample shows how to loop through all of the sites, determine which sites have FTP bindings, and then stop/start FTP for each site. To use this script, copy the code into a text editor like Windows Notepad and save it with a name like "RestartAllFtpSites.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Loop through the sites collection.
For siteCount = 0 To CInt(sitesCollection.Count)-1
    isFtpSite = False
    ' Determine if the current site is an FTP site by checking the bindings.
    Set siteElement = sitesCollection(siteCount)
    Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
    For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
        Set bindingElement = bindingsCollection(bindingsCount)
        If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
            isFtpSite = True
            Exit For
        End If
    Next
    ' If it's an FTP site, start and stop the site.
    If isFtpSite = True Then
        Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
        ' Create an instance of the Stop method.
        Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
        ' Execute the method to stop the FTP site.
        stopFtpSite.Execute()
        ' Create an instance of the Start method.
        Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
        ' Execute the method to start the FTP site.
        startFtpSite.Execute()
    End If
Next

And the following code sample shows how to stop/start a single FTP site. To use this script, copy the code into a text editor like Windows Notepad, rename the site name appropriately for one of your FTP sites, save it with a name like "RestartContosoFtpSite.vbs" to your system, then double-click the file to run it.

' Temporarily disable breaking on runtime errors.
On Error Resume Next

' Create an Admin Manager object.
Set adminManager = CreateObject("Microsoft.ApplicationHost.AdminManager")
adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"

' Test for commit path support.
If Err.Number <> 0 Then
    Err.Clear
    ' Create a Writable Admin Manager object.
    Set adminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
    adminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
    If Err.Number <> 0 Then WScript.Quit
End If

' Resume breaking on runtime errors.
On Error Goto 0

' Retrieve the sites collection.
Set sitesSection = adminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
Set sitesCollection = sitesSection.Collection

' Locate a specific site.
siteElementPos = FindElement(sitesCollection, "site", Array("name", "ftp.contoso.com"))
If siteElementPos = -1 Then
    WScript.Echo "Site was not found!"
    WScript.Quit
End If

' Determine if the selected site is an FTP site by checking the bindings.
Set siteElement = sitesCollection(siteElementPos)
Set bindingsCollection = siteElement.ChildElements.Item("bindings").Collection
For bindingsCount = 0 To CInt(bindingsCollection.Count)-1
    Set bindingElement = bindingsCollection(bindingsCount)
    If StrComp(CStr(bindingElement.Properties.Item("protocol").Value),"ftp",vbTextCompare)=0 Then
        isFtpSite = True
        Exit For
    End If
Next

' If it's an FTP site, start and stop the site.
If isFtpSite = True Then
    Set ftpServerElement = siteElement.ChildElements.Item("ftpServer")
    ' Create an instance of the Stop method.
    Set stopFtpSite = ftpServerElement.Methods.Item("Stop").CreateInstance()
    ' Execute the method to stop the FTP site.
    stopFtpSite.Execute()
    ' Create an instance of the Start method.
    Set startFtpSite = ftpServerElement.Methods.Item("Start").CreateInstance()
    ' Execute the method to start the FTP site.
    startFtpSite.Execute()
End If

' Locate and return the index for a specific element in a collection.
Function FindElement(collection, elementTagName, valuesToMatch)
   For i = 0 To CInt(collection.Count) - 1
      Set elem = collection.Item(i)
      If elem.Name = elementTagName Then
         matches = True
         For iVal = 0 To UBound(valuesToMatch) Step 2
            Set prop = elem.GetPropertyByName(valuesToMatch(iVal))
            value = prop.Value
            If Not IsNull(value) Then
               value = CStr(value)
            End If
            If Not value = CStr(valuesToMatch(iVal + 1)) Then
               matches = False
               Exit For
            End If
         Next
         If matches Then
            Exit For
         End If
      End If
   Next
   If matches Then
      FindElement = i
   Else
      FindElement = -1
   End If
End Function

I hope this helps!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Video: What's New with Internet Information Services (IIS) 8: Performance, Scalability, and Security Features

01 September 2012 • by Bob • FTP, IIS, Windows

The folks in the TechEd group have uploaded the video from my "What's New with Internet Information Services (IIS) 8: Performance, Scalability, and Security Features" presentation to YouTube, so you can view the video online.

You can also download the slides and the WMV/MP4 for my presentation at the following URL:

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/WSV332

One quick side note: around 38:55 during the video, I had just asked the audience if anyone had used the IIS Configuration Editor, when a tremendous thunderclap resounded outside - this prompted a great laugh from audience members. After the presentation had ended, a couple people came up and jokingly asked how I had managed to stage that so well.

Smile

Windows Cache 1.3 for PHP 5.4

26 July 2012 • by Bob • IIS, PHP, WinCache

The IIS team has officially signed off on the Windows Cache Extension (WinCache) version 1.3 for PHP 5.4, and the files have been uploaded to SourceForge. This version addresses all of the problems that were identified with WinCache 1.1 that customers were seeing after they upgraded their systems from PHP 5.3 to PHP 5.4.

With that in mind, you can download WinCache 1.3 for for PHP 5.4 from the following URL:

http://sourceforge.net/projects/wincache/files/wincache-1.3.4/

You can discuss WinCache 1.1 and WinCache 1.3 in the Windows Cache Extension for PHP forum on Microsoft's IIS.net website.

Source Code Availability

Since WinCache is an open source project, the IIS team has uploaded the pre-release source code for WinCache at the following URL:

http://pecl.php.net/package/WinCache

For the instructions on how to build the extension yourself, please refer to the Building WinCache Extension documentation.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Error: Class Not Registered (0x80040154) when Querying FTP Runtime State

29 June 2012 • by Bob • FTP, IIS

I had a great question from a customer earlier today, and I thought that it was worth blogging about. The problem that he was running into was that he was seeing the following error when he was trying to query the runtime state for the FTP service in an application that he was writing:

Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))

He was using Visual Basic, and his code looked okay to me, so for the moment I was stumped.

I'm more of a C# guy, and I remembered that I had written the following blog many years ago:

Viewing current FTP7 sessions using C#

I copied the code from that blog into a new Visual Studio project, and I got the same error that he was seeing when I ran my code - this had me a little more confused. Have you ever said to yourself, "Darn - I know that worked the other day...?" ;-]

I knew that there is more than one way to access the runtime state, so I rewrote my sample application using two different approaches:

Method #1:

AppHostAdminManager objAdminManager = new AppHostAdminManager();
IAppHostElement objSitesElement =
  objAdminManager.GetAdminSection("system.applicationHost/sites",
  "MACHINE/WEBROOT/APPHOST");
uint intSiteCount = objSitesElement.Collection.Count;
for (int intSite = 0; intSite < intSiteCount; ++intSite)
{
    IAppHostElement objFtpSite = objSitesElement.Collection[intSite];
    Console.WriteLine("Name: " + objFtpSite.Properties["name"].StringValue);
    IAppHostElement objFtpSiteElement = objFtpSite.ChildElements["ftpServer"];
    IAppHostPropertyCollection objProperties = objFtpSiteElement.Properties;
    try
    {
        IAppHostProperty objState = objProperties["state"];
        string ftpState = objState.StringValue;
        Console.WriteLine("State: " + ftpState);
    }
    catch (System.Exception ex)
    {
        Console.WriteLine("\r\nError: {0}", ex.Message);
    }
}

Method #2:

ServerManager manager = new ServerManager();
foreach (Site site in manager.Sites)
{
    Console.WriteLine("Name: " + site.Name);
    ConfigurationElement ftpServer = site.GetChildElement("ftpServer");
    try
    {
        foreach (ConfigurationAttribute attrib in ftpServer.Attributes)
        {
            Console.WriteLine(attrib.Name + ": " + attrib.Value);
        }
    }
    catch (System.Exception ex)
    {
        Console.WriteLine("\r\nError: {0}", ex.Message);
    }
}

Both of these methods returned the same COM error, so this was getting weird for me. Hmm...

The FTP runtime state is exposed through a COM interface, and that is implemented in a DLL that is named "ftpconfigext.dll". That file should be registered when you install IIS, and I re-registered it on my system just for good measure, but that didn't resolve the issue.

I had a brief conversation with one of my coworkers, Eok Kim, about the error that I was seeing. He also suggested re-registering the DLL, but something else that he said about searching the registry for the InprocServer32 entry made me wonder if the whole problem was related to the bitness of my application.

To make a long story short - that was the whole problem.

Both the customer and I were creating 32-bit .NET applications, and the COM interface for the FTP runtime state is implemented in a 64-bit-only DLL. Once we both changed our projects to compile for 64-bit platforms, we were both able to get the code to run. (Coincidentally, all I had was a 32-bit system when I wrote my original blog, so I probably would have run into this sooner if I had owned a 64-bit system way back then. ;-])

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Using URL Rewrite with QDIG

28 June 2012 • by Bob • IIS, URL Rewrite

One of the applications that I like to use on my websites it the Quick Digital Image Gallery (QDIG), which is a simple PHP-based image gallery that has just enough features to be really useful without a lot of work on my part to get it working. (Simple is always better - ;-].) Here's a screenshot of QDIG in action with some Bing photos:

(Click to enlarge photo.)

The trouble is, QDIG creates some really heinous query string lines; see the URL line in the following screenshot for an example:

(Click to enlarge photo.)

I don't know about you, but in today's SEO-friendly world, I hate long and convoluted query strings. Which brings me to one of my favorite subjects: URL Rewrite for IIS

If you've been around IIS for a while, you probably already know that there are a lot of great things that you can do with the IIS URL Rewrite module, and one of the things that URL Rewrite is great at is cleaning up complex query strings into something that's a little more intuitive.

It would take way to long to describe all of the steps to create the following rules with the URL Rewrite interface, so I'll just include the contents of my web.config file for my QDIG directory - which is a physical folder called "QDIG" that is under the root of my website:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <!-- Rewrite the inbound URLs into the correct query string. -->
        <rule name="RewriteInboundQdigURLs"  stopProcessing="true">
          <match url="Qif/(.*)/Qiv/(.*)/Qis/(.*)/Qwd/(.*)" />
          <conditions>
            <add input="{REQUEST_FILENAME}"  matchType="IsFile" negate="true" />
            <add input="{REQUEST_FILENAME}"  matchType="IsDirectory" negate="true" />
          </conditions>
          <action type="Rewrite"  url="/QDIG/?Qif={R:1}&amp;Qiv={R:2}&amp;Qis={R:3}&amp;Qwd={R:4}"  appendQueryString="false" />
        </rule>
      </rules>
      <outboundRules>
        <!-- Rewrite the outbound URLs into user-friendly URLs. -->
        <rule name="RewriteOutboundQdigURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="A, Img, Link"  pattern="^(.*)\?Qwd=([^=&amp;]+)&amp;(?:amp;)?Qif=([^=&amp;]+)&amp;(?:amp;)?Qiv=([^=&amp;]+)&amp;(?:amp;)?Qis=([^=&amp;]+)(.*)" />
          <action type="Rewrite"  value="/QDIG/Qif/{R:3}/Qiv/{R:4}/Qis/{R:5}/Qwd/{R:2}" />
        </rule>
        <!-- Rewrite the outbound relative QDIG URLs for the correct path. -->
        <rule name="RewriteOutboundRelativeQdigFileURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="Img"  pattern="^\.\/qdig-files/(.*)$" />
          <action type="Rewrite"  value="/QDIG/qdig-files/{R:1}" />
        </rule>
        <!-- Rewrite the outbound relative file URLs for the correct path. -->
        <rule name="RewriteOutboundRelativeFileURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="Img"  pattern="^\.\/(.*)$" />
          <action type="Rewrite"  value="/QDIG/{R:1}" />
        </rule>
        <preConditions>
          <!-- Define a precondition so the outbound rules only apply to HTML responses. -->
          <preCondition name="ResponseIsHTML">
            <add input="{RESPONSE_CONTENT_TYPE}"  pattern="^text/html" />
          </preCondition>
        </preConditions>
      </outboundRules>
    </rewrite>
  </system.webServer>
</configuration>

Here's the breakdown of what all of the rules do:

Once you have these rules in place, you get nice user-friendly URLs in QDIG:

(Click to enlarge photo.)

I should also point out that these rules also support changing the style from thumbnails to file names to file numbers, etc.

(Click to enlarge photo.)

All of that being said, there is one thing that these rules do not support - and that's nested folders under my QDIG application. I don't like to use folders under my QDIG folder - I like to use separate folders with the QDIG file in it, because this makes each gallery self-contained and easily transportable. That being said, after I had written the text for this blog, I tried to use a subfolder under my QDIG application and that didn't work. By looking at what was going on, I'm pretty sure that it would be pretty trivial to write some URL Rewrite rules that would accommodate using subfolders, but that's another project for another day. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Advanced Log Parser Charts Part 4 - Adding Custom Input Formats

25 May 2012 • by Bob • IIS, LogParser

In Part 4 of this series, I'll show you how you can do a couple of cool things:

For the data source for my custom plug-in, I thought that it would be cool to consume the data from FTP 7's Runtime Status (RSCA). If you've followed some of my old blogs, you would have seen that around five years ago I wrote the following pair of blogs about programmatically viewing FTP 7 sessions:

I'm going to recycle some of the FTP RSCA concepts from those blogs in order to create my COM plug-in.

Log Parser Input Formats

If you're like me, you already realize that the existing features of Log Parser simply rock. But what most people don't realize is that Log Parser lets you extend the functionality by adding new input formats, so you can consume the data from any place where you feel compelled to sit down and write your own Log Parser module.

As a quick reminder, Log Parser supports the following built-in input formats:

This last input format, COM, is how you interface with Log Parser in order to create your own input formats. When you install Log Parser, there are a few COM-based samples in the Log Parser directory, and you can take a look at those when you get the chance.

A Brief Introduction to Creating a COM plug-ins for Log Parser

To start with, your COM plug-in has to support a few public methods - and each of these will be more clear when I create my plug-in later:

Method Name Description
OpenInput Opens your data source and sets up any initial environment settings.
GetFieldCount Returns the number of fields that your plug-in will provide.
GetFieldName Returns the name of a specified field.
GetFieldType Returns the datatype of a specified field.
GetValue Returns the value of a specified field.
ReadRecord Reads the next record from your data source.
CloseInput Closes your data source and cleans up any environment settings.

After you've created and registered your COM plug-in, you will call it by using something like the following syntax:

logparser "SELECT * FROM FOO" -i:COM -iProgID:BAR

In this example, FOO is some data source that makes sense to your plug-in, and BAR is the COM class name for your plug-in.

Creating the COM plug-in for FTP RSCA Data

I'm going to demonstrate how to create a COM component as a scriptlet, and then I'll call that from Log Parser to process the data. I chose to use a scriptlet for this demo because they are quick to design and they're easily portable. Since no compilation is required, updates take place on the fly. All of that being said, if I were writing a real COM plug-in for Log Parser, I would use C# or C++.

To create the sample COM plug-in, copy the following code into a text file, and save that file as "MSUtil.LogQuery.FtpRscaScriptlet.sct" to your computer. (Note: The *.SCT file extension tells Windows that this is a scriptlet file.)

<SCRIPTLET>
  <registration
    Description="FTP RSCA for Log Parser Scriptlet"
    Progid="MSUtil.LogQuery.FtpRscaScriptlet"
    Classid="{4e616d65-6f6e-6d65-6973-526f62657274}"
    Version="1.00"
    Remotable="False" />
  <comment>
  EXAMPLE 1: logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  EXAMPLE 2: logparser "SELECT * FROM 1" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  </comment>
  <implements id="Automation" type="Automation">
    <method name="OpenInput">
      <parameter name="strValue"/>
    </method>
    <method name="GetFieldCount" />
    <method name="GetFieldName">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="GetFieldType">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="ReadRecord" />
    <method name="GetValue">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="CloseInput">
      <parameter name="blnAbort"/>
    </method>
  </implements>
  <SCRIPT LANGUAGE="VBScript">

Option Explicit

Dim objAdminManager,objSessionDictionary
Dim objSitesSection,objSitesCollection
Dim objSiteElement,objFtpServerElement
Dim objSessionsElement,objSessionElement
Dim intSiteElementPos,intSession,intRecordIndex
Dim clsSession

intRecordIndex = -1

' --------------------------------------------------------------------------------
' Open an input session that reads FTP RSCA data and stores it in a dictionary object.
' --------------------------------------------------------------------------------

Public Function OpenInput(strValue)
  Set objSessionDictionary = CreateObject("Scripting.Dictionary")
  Set objAdminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
  objAdminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
  Set objSitesSection = objAdminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
  Set objSitesCollection = objSitesSection.Collection
  If IsNumeric(strValue) Then
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("id", strValue))
  Else
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("name", strValue))
  End If
  If intSiteElementPos > -1 Then
    Set objSiteElement = objSitesCollection.Item(intSiteElementPos)
    Set objFtpServerElement = objSiteElement.ChildElements.Item("ftpServer")
    Set objSessionsElement = objFtpServerElement.ChildElements.Item("sessions").Collection
    For intSession = 0 To CLng(objSessionsElement.Count)-1
       Set objSessionElement = objSessionsElement.Item(intSession)
       Set clsSession = New Session
       clsSession.CurrentDateTime = GetUtcDate()
       clsSession.ClientIp = objSessionElement.GetPropertyByName("clientIp").Value
       clsSession.SessionId = objSessionElement.GetPropertyByName("sessionId").Value
       clsSession.SessionStartTime = objSessionElement.GetPropertyByName("sessionStartTime").Value
       clsSession.UserName = objSessionElement.GetPropertyByName("userName").Value
       clsSession.CurrentCommand = objSessionElement.GetPropertyByName("currentCommand").Value
       clsSession.PreviousCommand = objSessionElement.GetPropertyByName("previousCommand").Value
       clsSession.CommandStartTime = objSessionElement.GetPropertyByName("commandStartTime").Value
       clsSession.BytesSent = objSessionElement.GetPropertyByName("bytesSent").Value
       clsSession.BytesReceived = objSessionElement.GetPropertyByName("bytesReceived").Value
       clsSession.LastErrorStatus = objSessionElement.GetPropertyByName("lastErrorStatus").Value
       objSessionDictionary.Add intSession,clsSession
    Next
  End If
End Function

' --------------------------------------------------------------------------------
' Close the input session.
' --------------------------------------------------------------------------------

Public Function CloseInput(blnAbort)
  intRecordIndex = -1
  objSessionDictionary.RemoveAll
End Function

' --------------------------------------------------------------------------------
' Return the count of fields.
' --------------------------------------------------------------------------------

Public Function GetFieldCount()
  GetFieldCount = 11
End Function

' --------------------------------------------------------------------------------
' Return the specified field's name.
' --------------------------------------------------------------------------------

Public Function GetFieldName(intFieldIndex)
  Select Case intFieldIndex
    Case 0
      GetFieldName = "currentDateTime"
    Case 1
      GetFieldName = "clientIp"
    Case 2
      GetFieldName = "sessionId"
    Case 3
      GetFieldName = "sessionStartTime"
    Case 4
      GetFieldName = "userName"
    Case 5
      GetFieldName = "currentCommand"
    Case 6
      GetFieldName = "previousCommand"
    Case 7
      GetFieldName = "commandStartTime"
    Case 8
      GetFieldName = "bytesSent"
    Case 9
      GetFieldName = "bytesReceived"
    Case 10
      GetFieldName = "lastErrorStatus"
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's type.
' --------------------------------------------------------------------------------

Public Function GetFieldType(intFieldIndex)
  Const TYPE_INTEGER   = 1
  Const TYPE_REAL      = 2
  Const TYPE_STRING    = 3
  Const TYPE_TIMESTAMP = 4
  Const TYPE_NULL      = 5
  
  Select Case intFieldIndex
    Case 0
      GetFieldType = TYPE_STRING
    Case 1
      GetFieldType = TYPE_STRING
    Case 2
      GetFieldType = TYPE_STRING
    Case 3
      GetFieldType = TYPE_STRING
    Case 4
      GetFieldType = TYPE_STRING
    Case 5
      GetFieldType = TYPE_STRING
    Case 6
      GetFieldType = TYPE_STRING
    Case 7
      GetFieldType = TYPE_STRING
    Case 8
      GetFieldType = TYPE_INTEGER
    Case 9
      GetFieldType = TYPE_INTEGER
    Case 10
      GetFieldType = TYPE_INTEGER
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's value.
' --------------------------------------------------------------------------------

Public Function GetValue(intFieldIndex)
  If objSessionDictionary.Count > 0 Then
    Select Case intFieldIndex
      Case 0
        GetValue = objSessionDictionary(intRecordIndex).CurrentDateTime
      Case 1
        GetValue = objSessionDictionary(intRecordIndex).ClientIp
      Case 2
        GetValue = objSessionDictionary(intRecordIndex).SessionId
      Case 3
        GetValue = objSessionDictionary(intRecordIndex).SessionStartTime
      Case 4
        GetValue = objSessionDictionary(intRecordIndex).UserName
      Case 5
        GetValue = objSessionDictionary(intRecordIndex).CurrentCommand
      Case 6
        GetValue = objSessionDictionary(intRecordIndex).PreviousCommand
      Case 7
        GetValue = objSessionDictionary(intRecordIndex).CommandStartTime
      Case 8
        GetValue = objSessionDictionary(intRecordIndex).BytesSent
      Case 9
        GetValue = objSessionDictionary(intRecordIndex).BytesReceived
      Case 10
        GetValue = objSessionDictionary(intRecordIndex).LastErrorStatus
    End Select
  End If
End Function
  
' --------------------------------------------------------------------------------
' Read the next record, and return true or false if there is more data.
' --------------------------------------------------------------------------------

Public Function ReadRecord()
  If objSessionDictionary.Count > 0 Then
    If intRecordIndex < (objSessionDictionary.Count-1) Then
      intRecordIndex = intRecordIndex + 1
      ReadRecord = True
    Else
      ReadRecord = False
    End If
  End If
End Function

' --------------------------------------------------------------------------------
' Return the current UTC date/time.
' --------------------------------------------------------------------------------

Private Function GetUtcDate()
  Dim dtmNow,dtmUtc,strUtc
  Dim objShell,lngActiveTimeBias
  dtmNow = Now()
  Set objShell = CreateObject("WScript.Shell")
  lngActiveTimeBias = CLng(objShell.RegRead("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation\ActiveTimeBias"))
  dtmUtc = DateAdd("n",lngActiveTimeBias,dtmNow)
  strUtc = Year(dtmUtc) & "-" & _
    Right("0" & Month(dtmUtc),2) & "-" & _
    Right("0" & Day(dtmUtc),2) & "T" & _
    Right("0" & Hour(dtmUtc),2) & ":" & _
    Right("0" & Minute(dtmUtc),2) & ":" & _
    Right("0" & Second(dtmUtc),2) & ".000Z"
  GetUtcDate = strUtc
End Function

' --------------------------------------------------------------------------------
' Return an element's position in a collection.
' --------------------------------------------------------------------------------

Private Function FindElement(objCollection, strElementTagName, arrValuesToMatch)
  Dim i,elem,matches,j,prop,value
  For i = 0 To CInt(objCollection.Count) - 1
    Set elem = objCollection.Item(i)
    If elem.Name = strElementTagName Then
      matches = True
      For j = 0 To UBound(arrValuesToMatch) Step 2
        Set prop = elem.GetPropertyByName(arrValuesToMatch(j))
        value = prop.Value
        If Not IsNull(value) Then
          value = CStr(value)
        End If
        If Not value = CStr(arrValuesToMatch(j + 1)) Then
          matches = False
          Exit For
        End If
      Next
      If matches Then
        Exit For
      End If
    End If
  Next
  If matches Then
    FindElement = i
  Else
    FindElement = -1
  End If
End Function

' --------------------------------------------------------------------------------
' Define a generic class for holding session data.
' --------------------------------------------------------------------------------

Class Session
  Public CurrentDateTime
  Public ClientIp
  Public SessionId
  Public SessionStartTime
  Public UserName
  Public CurrentCommand
  Public PreviousCommand
  Public CommandStartTime
  Public BytesSent
  Public BytesReceived
  Public LastErrorStatus
End Class

  </SCRIPT>

</SCRIPTLET>

After you've saved the scriptlet code to your computer, you will register it by using the following syntax:

regsvr32 MSUtil.LogQuery.FtpRscaScriptlet.sct

At the very minimum, you can now use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

Next, we'll analyze what the scriptlet does.

Examining the COM plug-in in Detail

Here are the different parts of the scriptlet and what they do:

This wraps up the description of how the scriptlet works as a COM plug-in, in the next part of my blog we'll look at how to actually use it.

Using the COM plug-in with Log Parser

Earlier I showed you how you can use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

This will return output that resembles something like the following:

currentDateTime clientIp sessionId sessionStartTime userName currentCommand previousCommand commandStartTime bytesSent bytesReceived lastErrorStatus
---------------- -------- --------- ---------------- -------- -------------- --------------- ---------------- --------- ------------- ---------------
2012-05-25T11:42:11.000Z 10.121.75.26 3950d1e5-3e94-4734-a89a-9768c52aa924 2012-05-25T10:08:09.861Z robert PASS USER 2012-05-25T11:42:06.080Z 6049 1193 0
2012-05-25T11:42:11.000Z 10.121.75.26 d1591fa8-3b09-4afd-b2c0-950421ba79fe 2012-05-25T10:08:18.184Z robert RETR NLST 2012-05-25T11:42:07.172Z 5887 1169 0
2012-05-25T11:42:11.000Z 10.121.75.26 0f92b5ed-920a-441d-a15d-39056a36f2a4 2012-05-25T10:08:22.327Z robert NOOP NLST 2012-05-25T11:41:40.917Z 5857 1163 0
2012-05-25T11:42:11.000Z 10.121.75.26 16925f0d-1fc5-4cb7-be19-ab33face2da9 2012-05-25T10:08:48.756Z   NLST SYST 2012-05-25T11:41:44.770Z 6026 1192 0
2012-05-25T11:42:11.000Z 10.121.75.26 aeb68389-869b-4afc-8c81-47b578e74824 2012-05-25T10:08:54.214Z   USER HOST 2012-05-25T11:41:42.087Z 5864 1168 0
2012-05-25T11:42:11.000Z 10.121.75.26 4ed55569-ee25-47d1-8388-12cdb90a1c07 2012-05-25T10:12:31.555Z alice RETR NLST 2012-05-25T11:42:01.789Z 5780 1138 0
2012-05-25T11:42:11.000Z 10.121.75.26 d6b16bb4-cb65-492d-a9fa-fbd6b72de0f3 2012-05-25T10:12:54.591Z bob NOOP NLST 2012-05-25T11:41:46.563Z 5748 1130 0
                     
Statistics:                    
-----------                    
Elements processed: 7                  
Elements output: 7                  
Execution time: 0.12 seconds                  

That information is something of a jumbled mess, and we can clean that up a bit by simply choosing the fields that we might be interested in:

userName currentCommand commandStartTime
-------- -------------- ----------------
robert PASS 2012-05-25T11:42:06.080Z
robert RETR 2012-05-25T11:42:07.172Z
robert NOOP 2012-05-25T11:41:40.917Z
  NLST 2012-05-25T11:41:44.770Z
  USER 2012-05-25T11:41:42.087Z
alice RETR 2012-05-25T11:42:01.789Z
bob NOOP 2012-05-25T11:41:46.563Z
     
Statistics:    
-----------    
Elements processed: 7  
Elements output: 7  
Execution time: 0.12 seconds  

Now let's look at some interesting data - one of the main focuses for this blog series is charting  with Log Parser, so let's look at doing something useful with the data. To start with, here's how to create a pie chart that counts the number of sessions by user name:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"User Sessions"

This will generate a chart like the following:

Here's a variation on that script that illustrates how to create a pie chart that counts the number of authenticated sessions versus anonymous sessions:

logparser "SELECT CASE UserName WHEN '' THEN 'Anonymous' ELSE 'Authenticated' END AS AuthType,COUNT(*) AS Sessions INTO AuthenticatedSessions.gif FROM 1 GROUP BY AuthType" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"Authenticated Sessions"

This will generate a chart like the following:

We can also do line, bar, and column charts with the data:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:Column3D -o:CHART -values:on -legend:off -chartTitle:"User Sessions"

The above code sample will generate a chart like the following:

There's a lot more that we could do with this, but eventually I have to get some sleep, so I think that's enough fun for the day.

Summary

In this blog post, I've shown you how to add your own custom input format to Log Parser by creating scriptlet as a COM plug-in. I hope that you take this information and create some great Log Parser plug-ins of your own.

;-]

Advanced Log Parser Charts Part 3 - Missing Office Web Components for Charting

25 May 2012 • by Bob • IIS, LogParser

In Part 3 of this series, I'll explain what to do when you're missing the Office Web Components that are required for creating the charts that I have been demonstrating in this series.

Here's a brief explanation of the symptoms: you try a simple query that will create a chart like the following example:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

And you get the following error message:

Error creating output format "CHART": This output format requires a licensed Microsoft Office Chart Web Component to be installed on the local machine

More often than not, this simply means that you have Office 2007 or Office 2010, which do not contain the Office Web Components that are used by Log Parser to create charts. Fortunately, you can download the missing components from the following URL on Microsoft's website:

Office 2003 Add-in: Office Web Components
http://www.microsoft.com/en-us/download/details.aspx?id=22276

When you run the installation, you will see the following license agreement:

When you check the box to accept the license agreement and click Install, you will eventually receive the following dialog box to let you know that the Office 2003 Web Components have been installed:

Once you have the Office 2003 Web Components installed, you can run the same query successfully:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

Statistics:
-----------
Elements processed: 123330
Elements output: 14
Execution time: 0.57 seconds

Note: The above query generates the following somewhat uninteresting chart:

That being said, the point of this blog was to let you know how to get charting back, not how to make pretty charts. I'll save pretty charts for a future blog. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Extensibility Updates in the FTP 8.0 Service

20 April 2012 • by Bob • IIS, FTP, Extensibility

A few years ago I wrote a blog that was titled "FTP 7.5 Service Extensibility References", in which I discussed the extensibility APIs that we added in FTP 7.5. Over the next couple of years I followed that initial blog with a series of walkthroughs on IIS.net and several related blog posts. Here are just a few examples:

In today's blog I'd like to discuss some of the extensibility features that we added in FTP 8.0, and show you how you can use those in your FTP providers.

Custom FTP Authorization

In FTP 7.5 we provided interfaces for IFtpAuthenticationProvider and IFtpRoleProvider, which respectively allowed developers to create FTP providers that performed user and role lookups. In FTP 8.0 we added a logical extension to that API set with IFtpAuthorizationProvider interface, which allows developers to create FTP providers that perform authorization tasks.

With that in mind, I wrote the following walkthrough on the IIS.net web site:

The title pretty much says it all: the provider that I describe in that walkthrough will walk you through the steps that are required to create an FTP provider that provides custom user authentication, verification of role memberships, and authorization lookups on a per-path basis.

Custom FTP Event Handling

In FTP 7.5 if you wanted your provider to respond to specific user activity, the best way to do so was to implement the IFtpLogProvider.Log() interface and use that to provide a form of pseudo-event handling. In FTP 8.0 we add two event handling interfaces, IFtpPreprocessProvider and IFtpPostprocessProvider, which respectively allow developers to write providers that implement functionality before or after events have occurred.

With that in mind, I wrote the following walkthrough on the IIS.net web site:

Once again, the title says it all: the provider that I describe in that walkthrough will walk you through the steps that are required to create an FTP provider that prevents FTP clients from downloading more files per-session than you have allowed in your configuration settings.

Happy coding!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

FTP Logon Restrictions in IIS 8

19 March 2012 • by Bob • FTP, IIS

One of the biggest asks from our customers over the years was to provide a way to prevent brute-force password attacks on the FTP service. On several of the FTP sites that I host, I used to see a large number of fraudulent logon requests from hackers that were trying to guess a username/password combination. My first step in trying to prevent these kinds of attacks, like most good administrators, was to implement strong password requirements and password lockout policies. This was a good first step, but there is an unfortunate downside to password lockout policies - once a hacker locks out a user account, that means that a valid user is locked out of their account. What's more, a hacker can continue your server.

The FTP service has had a feature to block IP addresses, but this required something of a manual process to discover malicious behavior. To accomplish this, you had to query your log files for excessive activity, and then added the IP addresses from potential hackers to your blacklist of banned IP addresses. Besides the manual nature of this process, another big drawback to this approach is the fact that it isn't real-time, so a malicious client could be attacking your system for some time before you discover their activity.

With that in mind, my next step was to go after the hackers and block their IP addresses from accessing my server. To that end, I created the custom authentication provider for the FTP 7.5 service that I documented in the following walkthrough:

How to Use Managed Code (C#) to Create an FTP Authentication Provider with Dynamic IP Restrictions

That was pretty effective, but it was really intended to be a stop-gap measure while we were working on a built-in feature for the FTP service that ships with IIS 8, which allows you to block malicious logon attempts.

Here's the way this feature works - at the server level, you configure the maximum number of failed logon attempts that you will allow within a given time period; if someone fails to logon within that time frame, the FTP service will drop the connection, and the client will be blocked from accessing your server until the time frame has passed.

Additional details are available in the walkthrough that I wrote at the following URL:

IIS 8.0 FTP Logon Attempt Restrictions

If you'd like to try out the new FTP Logon Restrictions feature, you can download the Windows Server 8 Beta from the following URL:

http://www.microsoft.com/en-us/server-cloud/windows-server/v8-default.aspx

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Blog Navigation

You are on page 3 of 10 pages.

1 2 3 4 5 6 7 8 9 10