www.geekybob.com

Just a short, simple blog for Bob to share his thoughts.

Some Thoughts About My 20th Anniversary At Microsoft

18 December 2015 • by Bob • Microsoft, Random Thoughts, Windows, IIS

At the risk of gratuitous self-aggrandizement - today is my actual 20th anniversary. My 20th anniversary crystal arrived the other day, so I added to the collection on a bookcase in my office. (Although to be honest, they didn't hand out anniversary crystals when I reached my 5th and 10th anniversaries, so I had to buy those after the fact.)

20thAnniversaryCrystal

Things I'm Most-Proud Of...

Disclaimer: Yeah, I realize that I'm definitely bragging here; hence why I labeled this section "Things I'm Most-Proud Of."

Embarrassing Microsoft Moments...

Here is a collection of amusing stories with which I have been involved over the years:

Weird Microsoft Traditions...

We have done some silly things over the years at Microsoft; some of these traditions are still in practice, while others are long-gone. For example:

A Few of the Hardest Things for Me at 20 Years...

A Few Manager Anecdotes...

I worked with some great first-level managers at Microsoft - too many to recall - and I worked with some great skip-level managers, too. That being said, here is a list with some of my all-time favorite first-level managers (many of whom are no longer with the company): NancyL, SibhonO, MicheleP, JayV, StaceyC, TerryM, SharonM, BillS, Mai-lanB, EricD, ThomasD, AndrewL, TomW, WonY, and of course - WadeP.

Here are a few small managerial anecdotes - I don't have stories about everyone, so anyone left off the list is not meant as a slight:

Windows Versions during my Tenure...

I thought it would be fun to take a look back at all the version of Windows that have existed during my time at Microsoft, with a few of my thoughts about each version. (Note: you should also see Windows turns 30: a visual history, which is a pretty amusing look back at the history of Windows.)

That about wraps it up for me. So if anyone is still reading by now, here is my statement that I have always used to summarize how I feel about working at Microsoft: "To me, working for Microsoft is like working for Santa Claus; we make all the coolest toys, and we get to play with them before anyone else."

With that in mind, who wouldn't want to spend 20 years working for Microsoft?

How to create an HTML Application to configure your IIS SMTP Username and Password settings

22 May 2015 • by Bob • IIS, Scripting, Configuration, HTMLA, Security, VBScript, SMTP

Like many IIS administrators, I usually install the local SMTP service on my IIS servers when I am setting up a new server from scratch. When I install the SMTP service, I configure it so that it only listens on the IP address of 127.0.0.1, so it can only send emails which originate on the server itself. What's more, I configure the SMTP service to relay all emails to a downstream SMTP service which can send emails out to the Internet. By configuring these options, I can write my ASP.NET, PHP, and Classic ASP applications so that they use the local SMTP service for all email-related functionality, which acts as a sort of message server for my applications. This system works great, and I have used this particular setup since the days of IIS 4.0. (Which was released in late 1997, as you may recall.)

That being said, in the interests of security, sometime ago I started using a downstream SMTP service which requires user credentials, (that way no one could use the downstream server anonymously). As an additional security step, I use an account which requires that the credentials are changed every 30 days or so. This is always a good security practice for myriad obvious reasons, but this meant that I needed to update the SMTP username/password settings in my IIS configuration settings every 30 days.

With that in mind, many years ago I wrote a simple VBScript application which I would use to update those credentials. At first I would simply enter the credentials directly into the script, then I would run it to update IIS, and then I would strip the credentials from the script. Needless to say, this was pretty low-tech - even considering that this was 17 or 18 years ago. I eventually updated the script so that it would use VBScript Input Boxes to prompt me for the credentials, so I no longer needed to store the credentials in the script itself. (Sometime after that I rewrote the script so that it would read the existing values from the IIS settings and pre-populate the input boxes.)

Jumping ahead a couple of years, I decided to rewrite the script as an HTML Application, which offered me considerably more options from a user interface perspective. That script has been serving me faithfully for some time now, so I thought that it would make a good blog subject.

Using the HTML Application

Using the application is pretty straight-forward; when you double click the HTA file, it will present you with the following user interface:

The script will read any existing credentials from your IIS settings and use those to pre-populate the interface. If no existing credentials are found, it will pre-populate the interface with the username of the currently-logged-on user.

Clicking Update will update your IIS settings, clicking Reset will reset the values back to the last saved version, and clicking Close will obviously close the application, but only after it has checked if you have any unsaved changes.

Creating the HTML Application

To create this HTML Application, save the following HTMLA code as "Reset SMTP Password.hta" to your computer, and then double-click its icon to run the application.

<html>
<head>
<title>Reset SMTP Password</title>
<HTA:APPLICATION
  APPLICATIONNAME="Reset SMTP Password"
  ID="ResetSmtpPassword"
  VERSION="1.0"
  BORDER="dialog"
  BORDERSTYLE="static"
  INNERBORDER="no"
  CAPTION="yes"
  SYSMENU="no"
  MAXIMIZEBUTTON="no"
  MINIMIZEBUTTON="no"
  SCROLL="no"
  SCROLLFLAT="yes"
  SINGLEINSTANCE="yes"
  CONTEXTMENU="no"
  SELECTION="no"/>
<style>
<!--
body,input
{
 font-family:calibri,arial;
 font-size:9pt;
 color: #000;
 background-color: #fff;
}
table,td,th
{
 border:none;
}
-->
</style>
</head>

<script language="VBScript">
Option Explicit

' Define the global variables.
Dim objWMIService, objIIsSmtpServer
Dim strRouteUserName, strRoutePassword
Dim blnCancelBubble, blnPendingUpdates

' ----------------------------------------
' 
' Initialization method for the application.
' 
' ----------------------------------------

Sub Window_OnLoad

    ' Define the local variables.
    Dim objNetwork

    ' Set up the UI dimensions.
    Const intDialogWidth = 280
    Const intDialogHeight = 220
    ' Specify the window position and size.
    Self.resizeTo intDialogWidth,intDialogHeight
    Self.moveTo (Screen.AvailWidth - intDialogWidth) / 2, _
        (Screen.AvailHeight - intDialogHeight) / 2
        
    ' Enable events.
    blnCancelBubble = False
    blnPendingUpdates = False

    ' Set up some base objects for the local computer and default SMTP instance.
    ' Note that these settings can be customized for a different computer or SMTP instance.
    Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")
    Set objIIsSmtpServer = GetObject("IIS://localhost/SmtpSvc/1")
    
    ' Retrieve the current username/password from the SMTP settings.
    strRouteUserName = objIIsSmtpServer.RouteUserName
    strRoutePassword = objIIsSmtpServer.RoutePassword
    
    ' Verify that a username was retrieved; otherwise, use the logged-on user.
    If Len(strRouteUserName)=0 Then
        Set objNetwork = CreateObject("WScript.Network")
        strRouteUserName = IIf(Len(objNetwork.UserDomain)>0, _
            objNetwork.UserDomain & "\","") & objNetwork.UserName
        Set objNetwork = Nothing
        blnPendingUpdates = True
    End If
    
    ' Store the username/password values in the UI.
    txtUsername.value = strRouteUserName
    txtPassword.value = strRoutePassword
    
End Sub

' ----------------------------------------
' 
' Implement the missing IIf() function.
' 
' ----------------------------------------

Function IIf(tx,ty,tz)
    If (tx) Then IIf = ty Else IIf = tz
End Function

' ----------------------------------------
' 
' Click handler for the Close button.
' 
' ----------------------------------------

Sub btnClose_OnClick()
    ' Test if we need to cancel bubbling of events.
    If blnCancelBubble = False Then
        ' Check if there are pending updates.
        If blnPendingUpdates = False Then
            ' If not, then close the application.
            Window.close
        ' Prompt the user to exit.
        ElseIf MsgBox("You have not saved your changes." & vbCrLf & _
                "Are you sure you wish to exit?", _
                vbYesNo+vbDefaultButton2+vbQuestion+vbSystemModal, _
                ResetSmtpPassword.applicationName)=vbYes Then
                ' Enable event bubbling.
                blnCancelBubble = True
                ' Close the application.
                Window.close
        End If
    End If
    ' Specify whether to bubble events.
       blnCancelBubble = IIf(blnCancelBubble=True,False,True)
End Sub

' ----------------------------------------
' 
' Change handler for text boxes.
' 
' ----------------------------------------

Sub Textbox_OnChange()
    ' Flag the application as having updates pending.
    blnPendingUpdates = True
End Sub

' ----------------------------------------
' 
' Focus handler for text boxes.
' 
' ----------------------------------------

Sub Textbox_OnFocus(objTextbox)
    ' Select the text in the textbox.
    objTextbox.Select
End Sub

' ----------------------------------------
' 
' Click handler for the Reset button.
' 
' ----------------------------------------

Sub btnReset_OnClick()
    ' Reset the username/password values in the UI.
    txtUsername.value = strRouteUserName
    txtPassword.value = strRoutePassword
    blnPendingUpdates = False
End Sub

' ----------------------------------------
' 
' Click handler for the Update button.
' 
' ----------------------------------------

Sub btnUpdate_OnClick()

    ' Catch bubbled events.
    If blnCancelBubble = True Then
        blnCancelBubble = False
        Exit Sub
    End If

    ' Verify valid data.
    If Len(txtUsername.value)=0 Or Len(txtPassword.value)=0 Then
        ' Inform the user that they made a mistake.
        MsgBox "An invalid username or password was entered.", _
            vbCritical + vbOKOnly, ResetSmtpPassword.applicationName
        ' Cancel event bubbling.
        blnCancelBubble = True
    Else
        ' Store the username/password values for the SMTP server.
        objIIsSmtpServer.RouteUserName = txtUsername.value
        objIIsSmtpServer.RoutePassword = txtPassword.value
        objIIsSmtpServer.SetInfo

        ' Save the username/password values.
        strRouteUserName = txtUsername.value
        strRoutePassword = txtPassword.value

        ' Flag the application as having no updates pending.
        blnPendingUpdates = False

        ' Cancel event bubbling.
        blnCancelBubble = True
    End If

End Sub
</script>

<body bgcolor="white" id="HtmlBody">
<div id="FormControls">
    <table>
        <tr><td>Please enter your SMTP credentials:</td></tr>
        <tr>
            <td align="left">
                <input type="text"
                style="width:250px;height:22px"
                name="txtUsername"
                id="txtUsername"
                onchange="Textbox_OnChange()"
                onfocus="Textbox_OnFocus(txtUsername)" />
            </td>
        </tr>
        <tr>
            <td align="left">
                <input type="password"
                style="width:250px;height:22px"
                name="txtPassword"
                id="txtPassword"
                onchange="Textbox_OnChange()"
                onfocus="Textbox_OnFocus(txtPassword)" />
            </td>
        </tr>
        <tr>
            <td align="right">
                <input type="button"
                style="width:125px;height:22px"
                name="btnUpdate"
                id="btnUpdate"
                value="Update"
                onclick="btnUpdate_OnClick()" />
            </td>
        </tr>
        <tr>
            <td align="right">
                <input type="button"
                style="width:125px;height:22px"
                name="btnReset"
                id="btnReset"
                value="Reset"
                onclick="btnReset_OnClick()" />
            </td>
        </tr>
        <tr>
            <td align="right">
                <input type="button"
                style="width:125px;height:22px"
                name="btnClose"
                id="btnClose"
                value="Close"
                onclick="btnClose_OnClick()" />
            </td>
        </tr>
    </table>
</div>
</body>
</html>

That's all that there is to it, although you might want to restart your SMTP service after you have made these changes.

Additional Notes

On a related topic, I get asked from time to time why I like to use HTML Applications (HTMLA) for some of my scripting examples, and the answer is quite simple: it is very easy to create powerful scripts in a very short amount of time which have sophisticated user interfaces and no compilation requirements.

I use Adersoft's HtaEdit as my IDE for HTMLA, which allows me to do normal development tasks like configuring project options, setting breakpoints, and stepping through my code.


Note: Click the image above to open it full-size in a separate window.

That being said, I have been experimenting with creating user interfaces in PowerShell, and it looks like it has some real promise for creating great UIs, too. But for now I only use PowerShell to create command line scripts, I use HTMLA to create cool UIs for administrative scripts, and I use C# to create "real" applications.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Some Useful and Obscure FTP Configuration Settings

30 June 2014 • by Bob • Configuration, FTP, IIS

I get a lot of question about various configuration settings for the IIS FTP service, and most of the settings that I discuss with people can be configured through the FTP features in the IIS Manager. That being said, there are some useful configuration settings for the FTP service which I periodical send to people that have no user interface for setting them. With that in mind, I thought I would write a quick blog to point out a few of these obscure settings that I personally use the most-often or I send to other people.

Using Domain Name Syntax as an FTP Virtual Hostname

I use this setting on all of my FTP servers because it seems a little more natural to me. Here's the scenario: the IIS FTP service supports two kinds of hostnames:

Real FTP hostnames are pretty straight-forward: an FTP client specifies the hostname with a HOST command when a user is connecting to the server. Once the IIS FTP service receives that command, the FTP service routes the FTP session to the correct FTP site.

That being said, the FTP HOST command is still rather new, so only a handful of FTP clients currently support it. Because of that, you can use FTP "virtual" hostnames with the IIS FTP service. By default that syntax uses the "vertical line" or "pipe" character to differentiate between the hostname and user name. For example:

When you are specifying your FTP credentials in your FTP client, you would enter your username like the preceding examples. While this syntax is valid for both the IIS FTP service and the underlying FTP protocol, it seems a little odd to most users (including me). With that in mind, we added a configuration setting for the FTP service that will allow you to use the more-familiar domain\username syntax like the following examples:

To enable this feature, use the following steps:

  1. Open a command prompt as an administrator.
  2. Type the following commands:
    cd /d "%SystemRoot%\System32\Inetsrv"
    appcmd.exe set config -section:system.ftpServer/serverRuntime /hostNameSupport.useDomainNameAsHostName:"True" /commit:apphost
    net.exe stop FTPSVC
    net.exe start FTPSVC
  3. Close the command prompt.

More information about this feature is available in the IIS configuration reference at the following URL:

FTP Credential Caching

The FTP service caches user credentials for successfully-authenticated user sessions in order to help improve login performance, and I wrote the following detailed blog about this a couple of years ago:

Credential Caching in FTP 7.0 and FTP 7.5

I don't want to re-post an old blog, but I have sent several people to that blog over the years, so I thought that it was worth mentioning here since it seems to be referenced quite often. The problem that people seem to run into the most is that their old password is still valid for FTP after they have changed it, and this is caused by the FTP service caching their user credentials.

This is especially annoying for me personally when I am working on a development computer where I am creating an authentication provider. Unless I disable credential caching on my development computer, I can never seem to get any work done. To resolve this issue, I disable credential caching for the FTP service by using the following steps:

  1. Open a command prompt as an administrator.
  2. Type the following commands:
    cd /d "%SystemRoot%\System32\Inetsrv"
    appcmd.exe set config -section:system.ftpServer/caching /credentialsCache.enabled:"False" /commit:apphost
    net.exe stop FTPSVC
    net.exe start FTPSVC
  3. Close the command prompt.

The blog which I mentioned earlier goes into more detail about setting a custom timeout interval for credential caching instead of disabling the feature entirely, and all of the settings for FTP credential caching are in the IIS configuration reference at the following URLs:

FTP Client Certificate Authentication

FTP Client Certificate Authentication is an often-overlooked feature of the IIS FTP service, and I think that this is due to two reasons:

  1. There is no user interface to configure the required settings
  2. Configuring the required settings is very difficult

My second reason cannot be understated; I usually have to set up FTP Client Certificate Authentication once or twice a year in order to test various scenarios, and each time I do so I am reminded of just how difficult it can be to get everything right, and equally how easy it is to get something wrong.

Fortunately I took the time a couple of years ago to write a blog which documents everything that it takes to configure the FTP service, and I have used my notes in that blog several times. In complement to my blog on the subject, Vivek Kumbhar wrote an excellent blog series with additional steps to configure your Active Directory for certificate authentication. With that in mind, here are all of the requisite blog posts that you would need to set up this feature:

As I have mentioned before, configuring this feature is not for the faint-of-heart, but it can be very beneficial from a security standpoint.

For more information about the settings that are required for FTP Client Certificate Authentication, see the following articles in the IIS configuration reference:

That wraps it up for today's post. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Case Study: Migrating Microsoft's .NET Community Websites to Microsoft Azure

30 May 2014 • by Bob • ASP.NET, Azure, IIS

Have you ever wondered how much work is involved when migrating a traditionally-hosted production website to Microsoft Azure? If so, the following case study might be of interest to you:

Microsoft Azure Migration: Microsoft’s .NET Community Websites
Migrating Microsoft’s ASP.NET and IIS.NET Community Websites to Microsoft Azure

Here's a little background information on this migration case study: last fall Microsoft worked with two of it's hosting partners, Neudesic and Orcsweb, to migrate the www.asp.net and www.iis.net websites from a traditional web hosting scenario (e.g. websites hosted on physical servers) to virtual machines that are hosted in the cloud on Microsoft Azure. Here's what the web farm looked like before the migration:

After the migration, Microsoft had reduced both the hosting costs and the number of servers required by almost 50%.  Here's what the web farm looked like when the migration had been completed:

There are a lot of people who helped make this migration a success - and there are far too many to name here - but I would like to say a special "thanks" to everyone at Neudesic and Orcsweb for making this migration process as painless as possible.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

FTP ETW Tracing and IIS 8 - Part 2

09 April 2014 • by Bob • Batch Files, ETW, IIS 8, Log Files, LogParser, Scripting, Troubleshooting, FTP, IIS, IIS 8, LogParser, FTP

Shortly after I published my FTP ETW Tracing and IIS 8 blog post, I was using the batch file from that blog to troubleshoot an issue that I was having with a custom FTP provider. One of the columns which I display in my results is Clock-Time, which is obviously a sequential timestamp that is used to indicate the time and order in which the events occurred.

(Click the following image to view it full-size.)

At first glance the Clock-Time values might appear to be a range of useless numbers, but I use Clock-Time values quite often when I import the data from my ETW traces into something like Excel and I need to sort the data by the various columns.

That being said, apart from keeping the trace events in order, Clock-Time isn't a very user-friendly value. However, LogParser has some great built-in functions for crunching date/time values, so I decided to update the script to take advantage of some LogParser coolness and reformat the Clock-Time value into a human-readable Date/Time value.

My first order of business was to figure out how to decode the Clock-Time value; since Clock-Time increases for each event, it is obviously an offset from some constant, and after a bit of searching I found that the Clock-Time value is the offset in 100-nanosecond intervals since midnight on January 1, 1601. (Windows uses that value in a lot of places, not just ETW.) Once I had that information, it was pretty easy to come up with a LogParser formula to convert the Clock-Time value into the local time for my system, which is much easier to read.

With that in mind, here is the modified batch file:

@echo off

rem ======================================================================

rem Clean up old log files
for %%a in (ETL CSV) do if exist "%~n0.%%a" del "%~n0.%%a"

echo Starting the ETW session for full FTP tracing...
LogMan.exe start "%~n0" -p "IIS: Ftp Server" 255 5 -ets
echo.
echo Now reproduce your problem.
echo.
echo After you have reproduced your issue, hit any key to close the FTP
echo tracing session. Your trace events will be displayed automatically.
echo.
pause>nul

rem ======================================================================

echo.
echo Closing the ETW session for full FTP tracing...
LogMan.exe stop "%~n0" -ets

rem ======================================================================

echo.
echo Parsing the results - this may take a long time depending on the size of the trace...
if exist "%~n0.etl" (
   TraceRpt.exe "%~n0.etl" -o "%~n0.csv" -of CSV
   LogParser.exe "SELECT [Clock-Time], TO_LOCALTIME(ADD(TO_TIMESTAMP('1601-01-01 00:00:00', 'yyyy-MM-dd hh:mm:ss'), TO_TIMESTAMP(DIV([Clock-Time],10000000)))) AS [Date/Time], [Event Name], Type, [User Data] FROM '%~n0.csv'" -i:csv -e 2 -o:DATAGRID -rtp 20
)

When you run this new batch file, it will display an additional "Date/Time" column with a more-informative value in local time for the sever where you captured the trace.

(Click the following image to view it full-size.)

The new Date/Time column is considerably more practical, so I'll probably keep it in the batch file that I use when I am troubleshooting. You will also notice that I kept the original Clock-Time column; I chose to do so because I will undoubtedly continue to use that column for sorting when I import the data into something else, but you can safely remove that column if you would prefer to use only the new Date/Time value.

That wraps it up for today's post. :-)

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

FTP ETW Tracing and IIS 8

08 April 2014 • by Bob • FTP, IIS, IIS 8, LogParser, IIS, IIS 8, LogParser, ETW, Troubleshooting, Scripting, Batch Files, Log Files

In the past I have written a couple of blogs about using the FTP service's Event Tracing for Windows (ETW) features to troubleshoot issues; see FTP and ETW Tracing and Troubleshooting Custom FTP Providers with ETW for details. Those blog posts contain batch files which use the built-in Windows LogMan utility to capture an ETW trace, and they use downloadable LogParser utility to parse the results into human-readable form. I use the batch files from those blogs quite often, and I tend to use them a lot when I am developing custom FTP providers which add new functionality to my FTP servers.

Unfortunately, sometime around the release of Windows 8 and Windows Server 2012 I discovered that the ETW format had changed, and the current version of LogParser (version 2.2) cannot read the new ETW files. When you try to use the batch files from my blog with IIS 8, you see the following errors:

Verifying that LogParser.exe is in the path...
Done.

Starting the ETW session for full FTP tracing...
The command completed successfully.

Now reproduce your problem.

After you have reproduced your issue, hit any key to close the FTP tracing session. Your trace events will be displayed automatically.

Closing the ETW session for full FTP tracing...
The command completed successfully.

Parsing the results - this may take a long time depending on the size of the trace...
Task aborted.
Cannot open <from-entity>: Trace file "C:\temp\ftp.etl" has been created on a OS version (6.3) that is not compatible with the current OS version

Statistics:
-----------
Elements processed: 0
Elements output: 0
Execution time: 0.06 seconds

I meant to research a workaround at the time, but one thing led to another and I simply forgot about doing so. But I needed to use ETW the other day when I was developing something, so that seemed like a good time to quit slacking and come up with an answer. :-)

With that in mind, I came up with a very easy workaround, which I will present here. Once again, this batch file has a requirement on LogParser being installed on your system, but for the sake of brevity I have removed the lines from this version of the batch file which check for LogParser. (You can copy those lines from my previous blog posts if you want that functionality restored.)

Here's the way that this workaround is implemented: instead of creating an ETW log and then parsing it directly with LogParser, this new batch file invokes the built-in Windows TraceRpt command to parse the ETW file and save the results as a CSV file, which is then read by LogParser to view the results in a datagrid like the batch files in my previous blogs:

@echo off

rem ======================================================================

rem Clean up old log files
for %%a in (ETL CSV) do if exist "%~n0.%%a" del "%~n0.%%a"

echo Starting the ETW session for full FTP tracing...
LogMan.exe start "%~n0" -p "IIS: Ftp Server" 255 5 -ets
echo.
echo Now reproduce your problem.
echo.
echo After you have reproduced your issue, hit any key to close the FTP
echo tracing session. Your trace events will be displayed automatically.
echo.
pause>nul

rem ======================================================================

echo.
echo Closing the ETW session for full FTP tracing...
LogMan.exe stop "%~n0" -ets

rem ======================================================================

echo.
echo Parsing the results - this may take a long time depending on the size of the trace...
if exist "%~n0.etl" (
    TraceRpt.exe "%~n0.etl" -o "%~n0.csv" -of CSV
    LogParser.exe "SELECT [Clock-Time], [Event Name], Type, [User Data] FROM '%~n0.csv'" -i:csv -e 2 -o:DATAGRID -rtp 20
)

Here's another great thing about this new batch file - it will also work down-level on Windows 7 and Windows Server 2008; so if you have been using my previous batch files with IIS 7 - you can simply replace your old batch file with this new version. You will see a few differences between the results from my old batch files and this new version, namely that I included a couple of extra columns that I like to use for troubleshooting.

(Click the following image to view it full-size.)

There is one last thing which I would like to mention in closing: I realize that it would be much easier on everyone if Microsoft simply released a new version of LogParser which works with the new ETW format, but unfortunately there are no plans at the moment to release a new version of LogParser. And trust me - I'm just as depressed about that fact as anyone else. :-(

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

FTP ETW Tracing and IIS 8

08 April 2014 • by Bob • FTP, IIS, IIS 8, LogParser, IIS, IIS 8, LogParser, ETW, Troubleshooting, Scripting, Batch Files, Log Files

In the past I have written a couple of blogs about using the FTP service's Event Tracing for Windows (ETW) features to troubleshoot issues; see FTP and ETW Tracing and Troubleshooting Custom FTP Providers with ETW for details. Those blog posts contain batch files which use the built-in Windows LogMan utility to capture an ETW trace, and they use downloadable LogParser utility to parse the results into human-readable form. I use the batch files from those blogs quite often, and I tend to use them a lot when I am developing custom FTP providers which add new functionality to my FTP servers.

Unfortunately, sometime around the release of Windows 8 and Windows Server 2012 I discovered that the ETW format had changed, and the current version of LogParser (version 2.2) cannot read the new ETW files. When you try to use the batch files from my blog with IIS 8, you see the following errors:

Verifying that LogParser.exe is in the path...
Done.

Starting the ETW session for full FTP tracing...
The command completed successfully.

Now reproduce your problem.

After you have reproduced your issue, hit any key to close the FTP tracing session. Your trace events will be displayed automatically.

Closing the ETW session for full FTP tracing...
The command completed successfully.

Parsing the results - this may take a long time depending on the size of the trace...
Task aborted.
Cannot open <from-entity>: Trace file "C:\temp\ftp.etl" has been created on a OS version (6.3) that is not compatible with the current OS version

Statistics:
-----------
Elements processed: 0
Elements output: 0
Execution time: 0.06 seconds

I meant to research a workaround at the time, but one thing led to another and I simply forgot about doing so. But I needed to use ETW the other day when I was developing something, so that seemed like a good time to quit slacking and come up with an answer. :-)

With that in mind, I came up with a very easy workaround, which I will present here. Once again, this batch file has a requirement on LogParser being installed on your system, but for the sake of brevity I have removed the lines from this version of the batch file which check for LogParser. (You can copy those lines from my previous blog posts if you want that functionality restored.)

Here's the way that this workaround is implemented: instead of creating an ETW log and then parsing it directly with LogParser, this new batch file invokes the built-in Windows TraceRpt command to parse the ETW file and save the results as a CSV file, which is then read by LogParser to view the results in a datagrid like the batch files in my previous blogs:

@echo off

rem ======================================================================

rem Clean up old log files
for %%a in (ETL CSV) do if exist "%~n0.%%a" del "%~n0.%%a"

echo Starting the ETW session for full FTP tracing...
LogMan.exe start "%~n0" -p "IIS: Ftp Server" 255 5 -ets
echo.
echo Now reproduce your problem.
echo.
echo After you have reproduced your issue, hit any key to close the FTP
echo tracing session. Your trace events will be displayed automatically.
echo.
pause>nul

rem ======================================================================

echo.
echo Closing the ETW session for full FTP tracing...
LogMan.exe stop "%~n0" -ets

rem ======================================================================

echo.
echo Parsing the results - this may take a long time depending on the size of the trace...
if exist "%~n0.etl" (
    TraceRpt.exe "%~n0.etl" -o "%~n0.csv" -of CSV
    LogParser.exe "SELECT [Clock-Time], [Event Name], Type, [User Data] FROM '%~n0.csv'" -i:csv -e 2 -o:DATAGRID -rtp 20
)

Here's another great thing about this new batch file - it will also work down-level on Windows 7 and Windows Server 2008; so if you have been using my previous batch files with IIS 7 - you can simply replace your old batch file with this new version. You will see a few differences between the results from my old batch files and this new version, namely that I included a couple of extra columns that I like to use for troubleshooting.

(Click the following image to view it full-size.)

There is one last thing which I would like to mention in closing: I realize that it would be much easier on everyone if Microsoft simply released a new version of LogParser which works with the new ETW format, but unfortunately there are no plans at the moment to release a new version of LogParser. And trust me - I'm just as depressed about that fact as anyone else. :-(

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

RFC 7151 - File Transfer Protocol HOST Command for Virtual Hosts

14 March 2014 • by Bob • IIS, FTP, SSL, HOST

I received an email yesterday from the RFC Editor that a new Request for Comments (RFC) document has just been published, RFC 7151, which adds support for a new "HOST" command to FTP. This new command allows hosting multiple FTP sites on a single IP address, much like what Host Headers provide for HTTP.

Here's the URL to the new RFC on the RFC Editor website:

File Transfer Protocol HOST Command for Virtual Hosts
http://www.ietf.org/rfc/rfc7151.txt

Or you can see the HTML-based version at the following URL:

http://tools.ietf.org/html/rfc7151

One minor point which I would like to clarify is that this adds a new command for FTP to specify which virtual host to connect to. I periodically hear people referring to this as "FTP Host Headers", but that is technically incorrect since FTP does not have request headers like HTTP. Here's a simple example of what the communications flow looks like when the HOST command is used:

CLIENT> HOST ftp.example.com
SERVER> 220 Host accepted
CLIENT> USER foo
SERVER> 331 Password required
CLIENT> PASS bar
SERVER> 230 User logged in

I need to make sure that I thank my co-author for this RFC, Paul Hethmon, who has authored other FTP-related RFCs over the years. For example, Paul wrote RFC 3659, and he co-wrote RFC 2389 with Robert Elz. As a result, the Internet community has Paul and Robert to thank for several great FTP command extensions in the past. (e.g. FEAT, OPTS, MDTM, SIZE, REST, MLST, MLSD, etc.) Paul and I co-wrote RFC 7151 over the past several years, and it was great working with him.

Support for the HOST command has been built-in to Microsoft's FTP service since IIS 7.0, but now that the RFC has been officially published I hope that this feature will be adopted by other FTP servers and clients. That being said, IIS is not the only implementation of the FTP HOST command; at the time of this blog post, these are the server and client implementations that I am aware of which already provide support for this new command. (Note: there may be more than I have listed here; these are just the implementations that I currently know about.)

In addition to the clients listed above, if you have been reading my series on FTP clients over the past few years, I have posted details on how to use the FTP HOST command with some other FTP clients which do not provide built-in support. For example, the Core FTP Client allows you to specific pre-login commands as part of an FTP site's connection properties, so you can manually type in the HOST command and save it along the site's settings.

A Little Bit of History

When I joined the feature team which was creating the FTP service for Windows Server 2008, one of the things that bothered us was that there was no way at the protocol level to host multiple FTP sites on the same IP address. There were several ways that FTP server implementations were approximating that sort of functionality, for example the User Isolation features that we ship with FTP for IIS, but each FTP server seemed to be implementing its own workaround and there was no standardization.

Because of this limitation, our team received a lot of requests to add "FTP Host Headers," although as I explained earlier FTP has no concept of request headers. To help address some of the questions which I was often seeing, I explained the lack of hostname support for FTP in detail in the comments section of my FTP User Isolation with Multiple User Accounts blog that I posted back in 2006, which was shortly before we began work on implementing the HOST command. I will paraphrase some of my comments here:

While I realize that the ability host multiple FTP sites on the same IP address and port like HTTP is a desired configuration, the simple answer is that FTP does not currently support this at the protocol level. To put things in perspective, RFC 959 is the governing document for FTP, and that was published in October of 1985. FTP was simply not designed for the Internet as we use and understand it today, even though it is a generally reliable protocol that many people will continue to use for some time. HTTP/1.1 was designed much later and resolved this problem, but only for HTTP requests.

There are three ways that you can create unique bindings for a web or HTTP site: IP address, port, or host header. FTP can create unique bindings by IP address or port, but the FTP protocol does not currently provide support for hostnames.

Here's why: HTTP packets consist of a set of request headers and possibly a block of data. Here's an example of a simple GET request:

GET /default.aspx HTTP/1.0 [CrLf]
Accept: */* [CrLf]
[CrLf]

When HTTP 1.1 was published in RFC 2068 and RFC 2616, it defined a header for specifying a "host" name in a separate name/value pair:

GET /default.aspx HTTP/1.1 [CrLf]
Host: example.com [CrLf]
Accept: */* [CrLf]
[CrLf]

The "Host" header allows multiple HTTP virtual servers ("hosts") on the same IP address and port that are differentiated by host name. While this works great for the HTTP protocol, FTP currently has no comparable functionality. As such, the FTP protocol would have to be updated to allow multiple hosts on the same IP address and port, then FTP servers and clients would need to be updated to accommodate the changes to FTP.

While my explanation may have clarified root cause of the FTP limitation for anyone who was asking about it, I personally thought the situation was unacceptable. This inspired me to research the addition of a new command for FTP which would allow FTP clients to specify hostnames. As I was researching how to propose a new RFC document to the IETF, I discovered that Paul Hethmon had been researching the same problem a few years earlier. I contacted Paul and offered to combine our work, and he agreed. After several years of work and a great deal of supportive assistance from dozens of great people whom I met through the IETF, RFC 7151 has finally been published.

There are a lot of people besides Paul whom I should thank, and we mention them in the acknowledgments section of our RFC, which you can read at the following URL:

http://tools.ietf.org/html/rfc7151#appendix-B

One final note - two of my coworkers, Jaroslav Dunajsky and Wade Hilmo, are mentioned in the acknowledgments section of the RFC. Jaroslav is the developer who implemented the FTP HOST command for IIS, and Wade is a senior developer on the IIS team who graciously allowed me to bounce ideas off him while I was doing my research over the past few years. (I probably I owe him a lunch or two.)

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Rapid PHP Deployment for IIS using a Batch File

30 January 2014 • by Bob • IIS, Scripting, IIS Express, PHP

Whenever I am delivering a presentation where I need to use PHP, I typically use a batch file that I wrote in order to rapidly deploy PHP on the system that I am using for my demos. The batch file usually takes less than a second to run, which always seems to amaze people in the audience. As a result, I usually have several people ask me for my batch file after each presentation, so I thought that it would make a good subject for today's blog.

I should mention that I have used this batch file in order to demonstrate PHP with IIS in a variety of scenarios, and one of my favorite demos is when I would borrow someone's laptop and plug in a flash drive where I had IIS Express pre-installed, and then I would run the batch file in this blog to deploy PHP. Next I would launch IIS Express, open a web browser on their system, and then browse to http://localhost/ in order to show that IIS Express was working correctly. Lastly I would write a simple PHP "Hello World" page to show that PHP was up-and-running on their system in a matter of seconds.

That being said, I have to point out that there is a very important prerequisite that you must have in order to follow the steps in the blog: you need to start with a known-good installation of PHP from one of your systems, and I'll explain what I mean by that.

My batch file expects to find a folder containing ready-to-run files for PHP in order to deploy PHP on a new system. I originally obtained my PHP files by using the Web Platform Installer (WebPI) to install PHP, and then I copied the files to my flash drive or some other repository. (Note that WebPI usually installs PHP in the "%ProgramFiles(x86)%\PHP" folder.) If you don't want to use WebPI, you can also download PHP from http://windows.php.net/, but you're on your own for configuration.

Once I have the files from a known-good installation of PHP, I create the following folder structure in the location where I will be storing the files that I use to deploy PHP on other systems:

One thing to note is that the PHP.INI file you use may contain paths which refer to specific directories on the system from which you are copying your PHP files, so you need to make sure that those paths will exist on the system where you deploy PHP.

Here is an example: when I used WebPI to install PHP 5.5 on a system with IIS, it installed PHP into my "%ProgramFiles(x86)%\PHP\v5.5" folder. During the installation process, WebPI updated the PHP file to reflect any paths that need to be defined. At the time that I put together my notes for this blog, those updates mainly applied to the path where PHP expects to find it's extensions:

extension_dir="C:\Program Files (x86)\PHP\v5.5\ext\"

What this means is - if you want to deploy PHP to some other path on subsequent systems, you will need to update at least that line in the PHP.INI file that you are using to deploy PHP. In my particular case, I prefer to deploy PHP to the "%SystemDrive%\PHP" path, but it can be anywhere as long as you update everything accordingly.

The following batch file will deploy the PHP files in the "%SystemDrive%\PHP" folder on your system, and then it will update IIS with the necessary settings for this PHP deployment to work:

@echo off

REM Change to the installation folder
pushd "%~dp0"

REM Cheap test to see if IIS is installed
if exist "%SystemRoot%\System32\inetsrv" (
  REM Check for the PHP installation files in a subfolder
  if exist "%~dp0PHP" (
    REM Check for an existing installation of PHP
    if not exist "%SystemDrive%\PHP" (
      REM Create the folder for PHP
      md "%SystemDrive%\PHP"
      REM Deploy the PHP files
      xcopy /erhky "%~dp0PHP\*" "%SystemDrive%\PHP"
    )
    pushd "%SystemRoot%\System32\inetsrv"
    REM Configure the IIS settings for PHP
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000']" /commit:apphost
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000'].environmentVariables.[name='PHP_FCGI_MAX_REQUESTS',value='10000']" /commit:apphost
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000'].environmentVariables.[name='PHPRC',value='%SystemDrive%\PHP']" /commit:apphost
    appcmd.exe set config -section:system.webServer/handlers /+"[name='PHP_via_FastCGI',path='*.php',verb='GET,HEAD,POST',modules='FastCgiModule',scriptProcessor='%SystemDrive%\PHP\php-cgi.exe',resourceType='Either']" /commit:apphost
    popd
  )
)
popd

Once you have all of that in place, it usually takes less than a second to deploy PHP, which is why so many people seem interested during my presentations.

Note that you can deploy PHP for IIS Express just as easily by updating the "%SystemRoot%\System32\inetsrv" paths in the batch file to "%ProgramFiles%\IIS Express" or "%ProgramFiles(x86)%\IIS Express" paths. You can also use this batch file as part of a deployment process for PHP within a web farm; in which case, you will need to pay attention to the paths inside your PHP.INI file which I mentioned earlier.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Revisiting My Classic ASP and URL Rewrite for Dynamic SEO Functionality Examples

20 November 2013 • by Bob • Classic ASP, IIS, SEO, URL Rewrite

Last year I wrote a blog titled Using Classic ASP and URL Rewrite for Dynamic SEO Functionality, in which I described how you could combine Classic ASP and the URL Rewrite module for IIS to dynamically create Robots.txt and Sitemap.xml files for your website, thereby helping with your Search Engine Optimization (SEO) results. A few weeks ago I had a follow-up question which I thought was worth answering in a blog post.

Overview

Here is the question that I was asked:

"What if I don't want to include all dynamic pages in sitemap.xml but only a select few or some in certain directories because I don't want bots to crawl all of them. What can I do?"

That's a great question, and it wasn't tremendously difficult for me to update my original code samples to address this request. First of all, the majority of the code from my last blog will remain unchanged - here's the file by file breakdown for the changes that need made:

FilenameChanges
Robots.asp None
Sitemap.asp See the sample later in this blog
Web.config None

So if you are already using the files from my original blog, no changes need to be made to your Robot.asp file or the URL Rewrite rules in your Web.config file because the question only concerns the files that are returned in the the output for Sitemap.xml.

Updating the Necessary Files

The good news it, I wrote most of the heavy duty code in my last blog - there were only a few changes that needed to made in order to accommodate the requested functionality. The main difference is that the original Sitemap.asp file used to have a section that recursively parsed the entire website and listed all of the files in the website, whereas this new version moves that section of code into a separate function to which you pass the unique folder name to parse recursively. This allows you to specify only those folders within your website that you want in the resultant sitemap output.

With that being said, here's the new code for the Sitemap.asp file:

<%
    Option Explicit
    On Error Resume Next
    
    Response.Clear
    Response.Buffer = True
    Response.AddHeader "Connection", "Keep-Alive"
    Response.CacheControl = "public"
    
    Dim strUrlRoot, strPhysicalRoot, strFormat
    Dim objFSO, objFolder, objFile

    strPhysicalRoot = Server.MapPath("/")
    Set objFSO = Server.CreateObject("Scripting.Filesystemobject")
    
    strUrlRoot = "http://" & Request.ServerVariables("HTTP_HOST")
    
    ' Check for XML or TXT format.
    If UCase(Trim(Request("format")))="XML" Then
        strFormat = "XML"
        Response.ContentType = "text/xml"
    Else
        strFormat = "TXT"
        Response.ContentType = "text/plain"
    End If

    ' Add the UTF-8 Byte Order Mark.
    Response.Write Chr(CByte("&hEF"))
    Response.Write Chr(CByte("&hBB"))
    Response.Write Chr(CByte("&hBF"))
    
    If strFormat = "XML" Then
        Response.Write "<?xml version=""1.0"" encoding=""UTF-8""?>" & vbCrLf
        Response.Write "<urlset xmlns=""http://www.sitemaps.org/schemas/sitemap/0.9"">" & vbCrLf
    End if
    
    ' Always output the root of the website.
    Call WriteUrl(strUrlRoot,Now,"weekly",strFormat)

    ' Output only specific folders.
    Call ParseFolder("/marketing")
    Call ParseFolder("/sales")
    Call ParseFolder("/hr/jobs")

    ' --------------------------------------------------
    ' End of file system loop.
    ' -------------------------------------------------- 
    If strFormat = "XML" Then
        Response.Write "</urlset>"
    End If
    
    Response.End

    ' ======================================================================
    '
    ' Recursively walks a folder path and return URLs based on the
    ' static *.html files that it locates.
    ' 
    ' strRootFolder = The base path for recursion
    '
    ' ======================================================================

    Sub ParseFolder(strParentFolder)
        On Error Resume Next

        Dim strChildFolders, lngChildFolders
        Dim strUrlRelative, strExt

        ' Get the list of child folders under a parent folder.
        strChildFolders = GetFolderTree(Server.MapPath(strParentFolder))

        ' Loop through the collection of folders.
        For lngChildFolders = 1 to UBound(strChildFolders)
            strUrlRelative = Replace(Mid(strChildFolders(lngChildFolders),Len(strPhysicalRoot)+1),"\","/")
            Set objFolder = objFSO.GetFolder(Server.MapPath("." & strUrlRelative))
            ' Loop through the collection of files.
            For Each objFile in objFolder.Files
                strExt = objFSO.GetExtensionName(objFile.Name)
                If StrComp(strExt,"html",vbTextCompare)=0 Then
                    If StrComp(Left(objFile.Name,6),"google",vbTextCompare)<>0 Then
                        Call WriteUrl(strUrlRoot & strUrlRelative & "/" & objFile.Name, objFile.DateLastModified, "weekly", strFormat)
                    End If
                End If
            Next
        Next

    End Sub

    ' ======================================================================
    '
    ' Outputs a sitemap URL to the client in XML or TXT format.
    ' 
    ' tmpStrFreq = always|hourly|daily|weekly|monthly|yearly|never 
    ' tmpStrFormat = TXT|XML
    '
    ' ======================================================================

    Sub WriteUrl(tmpStrUrl,tmpLastModified,tmpStrFreq,tmpStrFormat)
        On Error Resume Next
        Dim tmpDate : tmpDate = CDate(tmpLastModified)
        ' Check if the request is for XML or TXT and return the appropriate syntax.
        If tmpStrFormat = "XML" Then
            Response.Write " <url>" & vbCrLf
            Response.Write " <loc>" & Server.HtmlEncode(tmpStrUrl) & "</loc>" & vbCrLf
            Response.Write " <lastmod>" & Year(tmpLastModified) & "-" & Right("0" & Month(tmpLastModified),2) & "-" & Right("0" & Day(tmpLastModified),2) & "</lastmod>" & vbCrLf
            Response.Write " <changefreq>" & tmpStrFreq & "</changefreq>" & vbCrLf
            Response.Write " </url>" & vbCrLf
        Else
            Response.Write tmpStrUrl & vbCrLf
        End If
    End Sub

    ' ======================================================================
    '
    ' Returns a string array of folders under a root path
    '
    ' ======================================================================

    Function GetFolderTree(strBaseFolder)
        Dim tmpFolderCount,tmpBaseCount
        Dim tmpFolders()
        Dim tmpFSO,tmpFolder,tmpSubFolder
        ' Define the initial values for the folder counters.
        tmpFolderCount = 1
        tmpBaseCount = 0
        ' Dimension an array to hold the folder names.
        ReDim tmpFolders(1)
        ' Store the root folder in the array.
        tmpFolders(tmpFolderCount) = strBaseFolder
        ' Create file system object.
        Set tmpFSO = Server.CreateObject("Scripting.Filesystemobject")
        ' Loop while we still have folders to process.
        While tmpFolderCount <> tmpBaseCount
            ' Set up a folder object to a base folder.
            Set tmpFolder = tmpFSO.GetFolder(tmpFolders(tmpBaseCount+1))
              ' Loop through the collection of subfolders for the base folder.
            For Each tmpSubFolder In tmpFolder.SubFolders
                ' Increment the folder count.
                tmpFolderCount = tmpFolderCount + 1
                ' Increase the array size
                ReDim Preserve tmpFolders(tmpFolderCount)
                ' Store the folder name in the array.
                tmpFolders(tmpFolderCount) = tmpSubFolder.Path
            Next
            ' Increment the base folder counter.
            tmpBaseCount = tmpBaseCount + 1
        Wend
        GetFolderTree = tmpFolders
    End Function
%>

It should be easily seen that the code is largely unchanged from my previous blog.

In Closing...

One last thing to consider, I didn't make any changes to the Robots.asp file in this blog. But that being said, when you do not want specific paths crawled, you should add rules to your Robots.txt file to disallow those paths. For example, here is a simple Robots.txt file which allows your entire website:

# Robots.txt
# For more information on this file see:
# http://www.robotstxt.org/

# Define the sitemap path
Sitemap: http://localhost:53644/sitemap.xml

# Make changes for all web spiders
User-agent: *
Allow: /
Disallow:

If you were going to deny crawling on certain paths, you would need to add the specific paths that you do not want crawled to your Robots.txt file like the following example:

# Robots.txt
# For more information on this file see:
# http://www.robotstxt.org/

# Define the sitemap path
Sitemap: http://localhost:53644/sitemap.xml

# Make changes for all web spiders
User-agent: *
Disallow: /foo
Disallow: /bar

With that being said, if you are using my Robots.asp file from my last blog, you would need to update the section of code that defines the paths like my previous example:

Response.Write "# Make changes for all web spiders" & vbCrLf
Response.Write "User-agent: *" & vbCrLf
Response.Write "Disallow: /foo" & vbCrLf
Response.Write "Disallow: /bar" & vbCrLf

I hope this helps. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Blog Navigation

You are on page 1 of 10 pages.

1 2 3 4 5 6 7 8 9 10