www.geekybob.com

Just a short, simple blog for Bob to share his thoughts.

Some Thoughts About My 20th Anniversary At Microsoft

18 December 2015 • by Bob • Microsoft, Random Thoughts, Windows, IIS

At the risk of gratuitous self-aggrandizement - today is my actual 20th anniversary. My 20th anniversary crystal arrived the other day, so I added to the collection on a bookcase in my office. (Although to be honest, they didn't hand out anniversary crystals when I reached my 5th and 10th anniversaries, so I had to buy those after the fact.)

20thAnniversaryCrystal

Things I'm Most-Proud Of...

Disclaimer: Yeah, I realize that I'm definitely bragging here; hence why I labeled this section "Things I'm Most-Proud Of."

Embarrassing Microsoft Moments...

Here is a collection of amusing stories with which I have been involved over the years:

Weird Microsoft Traditions...

We have done some silly things over the years at Microsoft; some of these traditions are still in practice, while others are long-gone. For example:

A Few of the Hardest Things for Me at 20 Years...

A Few Manager Anecdotes...

I worked with some great first-level managers at Microsoft - too many to recall - and I worked with some great skip-level managers, too. That being said, here is a list with some of my all-time favorite first-level managers (many of whom are no longer with the company): NancyL, SibhonO, MicheleP, JayV, StaceyC, TerryM, SharonM, BillS, Mai-lanB, EricD, ThomasD, AndrewL, TomW, WonY, and of course - WadeP.

Here are a few small managerial anecdotes - I don't have stories about everyone, so anyone left off the list is not meant as a slight:

Windows Versions during my Tenure...

I thought it would be fun to take a look back at all the version of Windows that have existed during my time at Microsoft, with a few of my thoughts about each version. (Note: you should also see Windows turns 30: a visual history, which is a pretty amusing look back at the history of Windows.)

That about wraps it up for me. So if anyone is still reading by now, here is my statement that I have always used to summarize how I feel about working at Microsoft: "To me, working for Microsoft is like working for Santa Claus; we make all the coolest toys, and we get to play with them before anyone else."

With that in mind, who wouldn't want to spend 20 years working for Microsoft?

What I Do For A Living

29 April 2014 • by Bob • Random Thoughts

It seems that I have always had a difficult time explaining to people what I do at Microsoft. It's not that I'm unsure about what I do - the details of my job have always been crystal-clear to me and I love what I am doing. It's just that I can't find a way to explain things in a way that doesn't result in blank stares from anyone who isn't a geek. (This problem isn't limited to me, though; my non-technical wife simply responds "I have no idea what he does" when someone asks her what I do for a living.)

Here's a perfect example: when I was a Program Manager on the Internet Information Services (IIS) team, people would often ask me what I did for Microsoft, and I would reply with something like, "I help design and implement the web publishing protocols for Microsoft's web server."

Other Person: [Blank Stare]

I would attempt to remedy the situation by adding, "You know, I design Microsoft's implementation of Internet technologies like the File Transfer Protocol, WebDAV, and the FrontPage Server Extensions."

Other Person: [Blank Stare]

In a sometimes-futile effort to salvage the conversation from complete disaster, I would interject, "You like to use the Internet, right? Well, your computer is on one side of the Internet, and my team helps build the other side of the Internet. That's kind of what I do."

That comment would usually be met with a slight spark of recognition, which was sometimes followed by a half-muttered, "That's nice."

At one time or other during my tenure as a Program Manager on the IIS team I was responsible for a smattering of disparate technologies; things like FTP, WebDAV, FPSE, FastCGI, PHP, URL Rewrite, IIS Express, Log Parser, etc. Most of those technologies garnered little to no interest for the average person, and many of my coworkers found them pretty boring as well. Just the same, I personally found every one of those technologies completely fascinating. (Why else would I spend eight years trying to get just one new command added to FTP?)

A couple of years ago I left the IIS program management team and I joined the writing team which is responsible for documenting Microsoft's ASP.NET framework; and if you have to ask what that means, then you are probably not interested in the answer.

Still, people would ask me what I do for Microsoft, and I would try to explain my job with statements like, "I document the Application Programming Interfaces (or APIs) for Microsoft's ASP.NET."

Other Person: [Blank Stare]

I would try to nudge the conversation along by saying things like, "I help people write web code."

Other Person: [Blank Stare]

Skipping ahead in the conversation, I would usually make a last-ditch attempt by stating, "Let's say you wanted to create a website; if so, you might read something that I wrote in order to help you get started."

Sometimes this remark would illicit a hint of acknowledgment, but usually I just got another blank stare.

This leads me to a few days ago. My wife and I were at dinner, and a waiter asked me what I did for a living. In the back of my mind I started to say something like, "Well, these days I'm documenting a set of APIs that Java programmers will use with Microsoft Azure technologies [blah blah blah]..."

But what actually came out of my mouth was, "I could explain it to you, but I'm pretty sure you wouldn't want me to. Trust me."

I like that answer. I think I'll stick with it in the future. :-)

FTP ETW Tracing and IIS 8 - Part 2

09 April 2014 • by Bob • Batch Files, ETW, IIS 8, Log Files, LogParser, Scripting, Troubleshooting, FTP, IIS, IIS 8, LogParser, FTP

Shortly after I published my FTP ETW Tracing and IIS 8 blog post, I was using the batch file from that blog to troubleshoot an issue that I was having with a custom FTP provider. One of the columns which I display in my results is Clock-Time, which is obviously a sequential timestamp that is used to indicate the time and order in which the events occurred.

(Click the following image to view it full-size.)

At first glance the Clock-Time values might appear to be a range of useless numbers, but I use Clock-Time values quite often when I import the data from my ETW traces into something like Excel and I need to sort the data by the various columns.

That being said, apart from keeping the trace events in order, Clock-Time isn't a very user-friendly value. However, LogParser has some great built-in functions for crunching date/time values, so I decided to update the script to take advantage of some LogParser coolness and reformat the Clock-Time value into a human-readable Date/Time value.

My first order of business was to figure out how to decode the Clock-Time value; since Clock-Time increases for each event, it is obviously an offset from some constant, and after a bit of searching I found that the Clock-Time value is the offset in 100-nanosecond intervals since midnight on January 1, 1601. (Windows uses that value in a lot of places, not just ETW.) Once I had that information, it was pretty easy to come up with a LogParser formula to convert the Clock-Time value into the local time for my system, which is much easier to read.

With that in mind, here is the modified batch file:

@echo off

rem ======================================================================

rem Clean up old log files
for %%a in (ETL CSV) do if exist "%~n0.%%a" del "%~n0.%%a"

echo Starting the ETW session for full FTP tracing...
LogMan.exe start "%~n0" -p "IIS: Ftp Server" 255 5 -ets
echo.
echo Now reproduce your problem.
echo.
echo After you have reproduced your issue, hit any key to close the FTP
echo tracing session. Your trace events will be displayed automatically.
echo.
pause>nul

rem ======================================================================

echo.
echo Closing the ETW session for full FTP tracing...
LogMan.exe stop "%~n0" -ets

rem ======================================================================

echo.
echo Parsing the results - this may take a long time depending on the size of the trace...
if exist "%~n0.etl" (
   TraceRpt.exe "%~n0.etl" -o "%~n0.csv" -of CSV
   LogParser.exe "SELECT [Clock-Time], TO_LOCALTIME(ADD(TO_TIMESTAMP('1601-01-01 00:00:00', 'yyyy-MM-dd hh:mm:ss'), TO_TIMESTAMP(DIV([Clock-Time],10000000)))) AS [Date/Time], [Event Name], Type, [User Data] FROM '%~n0.csv'" -i:csv -e 2 -o:DATAGRID -rtp 20
)

When you run this new batch file, it will display an additional "Date/Time" column with a more-informative value in local time for the sever where you captured the trace.

(Click the following image to view it full-size.)

The new Date/Time column is considerably more practical, so I'll probably keep it in the batch file that I use when I am troubleshooting. You will also notice that I kept the original Clock-Time column; I chose to do so because I will undoubtedly continue to use that column for sorting when I import the data into something else, but you can safely remove that column if you would prefer to use only the new Date/Time value.

That wraps it up for today's post. :-)

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

FTP ETW Tracing and IIS 8

08 April 2014 • by Bob • FTP, IIS, IIS 8, LogParser, IIS, IIS 8, LogParser, ETW, Troubleshooting, Scripting, Batch Files, Log Files

In the past I have written a couple of blogs about using the FTP service's Event Tracing for Windows (ETW) features to troubleshoot issues; see FTP and ETW Tracing and Troubleshooting Custom FTP Providers with ETW for details. Those blog posts contain batch files which use the built-in Windows LogMan utility to capture an ETW trace, and they use downloadable LogParser utility to parse the results into human-readable form. I use the batch files from those blogs quite often, and I tend to use them a lot when I am developing custom FTP providers which add new functionality to my FTP servers.

Unfortunately, sometime around the release of Windows 8 and Windows Server 2012 I discovered that the ETW format had changed, and the current version of LogParser (version 2.2) cannot read the new ETW files. When you try to use the batch files from my blog with IIS 8, you see the following errors:

Verifying that LogParser.exe is in the path...
Done.

Starting the ETW session for full FTP tracing...
The command completed successfully.

Now reproduce your problem.

After you have reproduced your issue, hit any key to close the FTP tracing session. Your trace events will be displayed automatically.

Closing the ETW session for full FTP tracing...
The command completed successfully.

Parsing the results - this may take a long time depending on the size of the trace...
Task aborted.
Cannot open <from-entity>: Trace file "C:\temp\ftp.etl" has been created on a OS version (6.3) that is not compatible with the current OS version

Statistics:
-----------
Elements processed: 0
Elements output: 0
Execution time: 0.06 seconds

I meant to research a workaround at the time, but one thing led to another and I simply forgot about doing so. But I needed to use ETW the other day when I was developing something, so that seemed like a good time to quit slacking and come up with an answer. :-)

With that in mind, I came up with a very easy workaround, which I will present here. Once again, this batch file has a requirement on LogParser being installed on your system, but for the sake of brevity I have removed the lines from this version of the batch file which check for LogParser. (You can copy those lines from my previous blog posts if you want that functionality restored.)

Here's the way that this workaround is implemented: instead of creating an ETW log and then parsing it directly with LogParser, this new batch file invokes the built-in Windows TraceRpt command to parse the ETW file and save the results as a CSV file, which is then read by LogParser to view the results in a datagrid like the batch files in my previous blogs:

@echo off

rem ======================================================================

rem Clean up old log files
for %%a in (ETL CSV) do if exist "%~n0.%%a" del "%~n0.%%a"

echo Starting the ETW session for full FTP tracing...
LogMan.exe start "%~n0" -p "IIS: Ftp Server" 255 5 -ets
echo.
echo Now reproduce your problem.
echo.
echo After you have reproduced your issue, hit any key to close the FTP
echo tracing session. Your trace events will be displayed automatically.
echo.
pause>nul

rem ======================================================================

echo.
echo Closing the ETW session for full FTP tracing...
LogMan.exe stop "%~n0" -ets

rem ======================================================================

echo.
echo Parsing the results - this may take a long time depending on the size of the trace...
if exist "%~n0.etl" (
    TraceRpt.exe "%~n0.etl" -o "%~n0.csv" -of CSV
    LogParser.exe "SELECT [Clock-Time], [Event Name], Type, [User Data] FROM '%~n0.csv'" -i:csv -e 2 -o:DATAGRID -rtp 20
)

Here's another great thing about this new batch file - it will also work down-level on Windows 7 and Windows Server 2008; so if you have been using my previous batch files with IIS 7 - you can simply replace your old batch file with this new version. You will see a few differences between the results from my old batch files and this new version, namely that I included a couple of extra columns that I like to use for troubleshooting.

(Click the following image to view it full-size.)

There is one last thing which I would like to mention in closing: I realize that it would be much easier on everyone if Microsoft simply released a new version of LogParser which works with the new ETW format, but unfortunately there are no plans at the moment to release a new version of LogParser. And trust me - I'm just as depressed about that fact as anyone else. :-(

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Adding Additional Content Types to my Classic ASP and URL Rewrite Samples for Dynamic SEO Functionality

28 February 2014 • by Bob • URL Rewrite, SEO, Classic ASP

In December of 2012 I wrote a blog titled "Using Classic ASP and URL Rewrite for Dynamic SEO Functionality", in which I described how to use Classic ASP and URL Rewrite to dynamically-generate Robots.txt and Sitemap.xml or Sitemap.txt files. I received a bunch of requests for additional information, so I wrote a follow-up blog this past November titled "Revisiting My Classic ASP and URL Rewrite for Dynamic SEO Functionality Examples" which illustrated how to limit the output for the Robots.asp and Sitemap.asp files to specific sections of your website.

That being said, I continue to receive requests for additional ways to stretch those samples, so I thought that I would write at least a couple of blogs on the subject. With that in mind, for today I wanted to show how you can add additional content types to the samples.

Overview

Here is a common question that I have been asked by several people:

"The example only works with *.html files; how do I include my other files?"

That's a great question, and additional content types are really easy to implement, and the majority of the code from my original blog will remain unchanged. Here's the file by file breakdown for the changes that need made:

FilenameChanges
Robots.asp None
Sitemap.asp See the sample later in this blog
Web.config None

If you are already using the files from my original blog, no changes need to be made to your Robots.asp file or the URL Rewrite rules in your Web.config file because the updates in this blog will only impact the output from the Sitemap.asp file.

Updating the Sitemap.asp File

My original sample contained a line of code which read "If StrComp(strExt,"html",vbTextCompare)=0 Then" and this line was used to restrict the sitemap output to static *.html files. For this new sample I need to make two changes:

  1. I am adding a string constant which contains a list of file extensions from several content types to use for the output.
  2. I replace the line of code which performs the comparison.

Note: I define the constant near the beginning of the file so it's easier for other people to find; I would normally define that constant elsewhere in the code.

<%
    Option Explicit
    On Error Resume Next
    
    Const strContentTypes = "htm|html|asp|aspx|txt"
    
    Response.Clear
    Response.Buffer = True
    Response.AddHeader "Connection", "Keep-Alive"
    Response.CacheControl = "public"
    
    Dim strFolderArray, lngFolderArray
    Dim strUrlRoot, strPhysicalRoot, strFormat
    Dim strUrlRelative, strExt

    Dim objFSO, objFolder, objFile

    strPhysicalRoot = Server.MapPath("/")
    Set objFSO = Server.CreateObject("Scripting.Filesystemobject")
    
    strUrlRoot = "http://" & Request.ServerVariables("HTTP_HOST")
    
    ' Check for XML or TXT format.
    If UCase(Trim(Request("format")))="XML" Then
        strFormat = "XML"
        Response.ContentType = "text/xml"
    Else
        strFormat = "TXT"
        Response.ContentType = "text/plain"
    End If

    ' Add the UTF-8 Byte Order Mark.
    Response.Write Chr(CByte("&hEF"))
    Response.Write Chr(CByte("&hBB"))
    Response.Write Chr(CByte("&hBF"))
    
    If strFormat = "XML" Then
        Response.Write "<?xml version=""1.0"" encoding=""UTF-8""?>" & vbCrLf
        Response.Write "<urlset xmlns=""http://www.sitemaps.org/schemas/sitemap/0.9"">" & vbCrLf
    End if
    
    ' Always output the root of the website.
    Call WriteUrl(strUrlRoot,Now,"weekly",strFormat)

    ' --------------------------------------------------
    ' This following section contains the logic to parse
    ' the directory tree and return URLs based on the
    ' files that it locates.
    ' -------------------------------------------------- 
    strFolderArray = GetFolderTree(strPhysicalRoot)

    For lngFolderArray = 1 to UBound(strFolderArray)
        strUrlRelative = Replace(Mid(strFolderArray(lngFolderArray),Len(strPhysicalRoot)+1),"\","/")
        Set objFolder = objFSO.GetFolder(Server.MapPath("." & strUrlRelative))
        For Each objFile in objFolder.Files
            strExt = objFSO.GetExtensionName(objFile.Name)
            If InStr(1,strContentTypes,strExt,vbTextCompare) Then
                If StrComp(Left(objFile.Name,6),"google",vbTextCompare)<>0 Then
                    Call WriteUrl(strUrlRoot & strUrlRelative & "/" & objFile.Name, objFile.DateLastModified, "weekly", strFormat)
                End If
            End If
        Next
    Next

    ' --------------------------------------------------
    ' End of file system loop.
    ' -------------------------------------------------- 
    If strFormat = "XML" Then
        Response.Write "</urlset>"
    End If
    
    Response.End

    ' ======================================================================
    '
    ' Outputs a sitemap URL to the client in XML or TXT format.
    ' 
    ' tmpStrFreq = always|hourly|daily|weekly|monthly|yearly|never 
    ' tmpStrFormat = TXT|XML
    '
    ' ======================================================================

    Sub WriteUrl(tmpStrUrl,tmpLastModified,tmpStrFreq,tmpStrFormat)
        On Error Resume Next
        Dim tmpDate : tmpDate = CDate(tmpLastModified)
        ' Check if the request is for XML or TXT and return the appropriate syntax.
        If tmpStrFormat = "XML" Then
            Response.Write " <url>" & vbCrLf
            Response.Write " <loc>" & Server.HtmlEncode(tmpStrUrl) & "</loc>" & vbCrLf
            Response.Write " <lastmod>" & Year(tmpLastModified) & "-" & Right("0" & Month(tmpLastModified),2) & "-" & Right("0" & Day(tmpLastModified),2) & "</lastmod>" & vbCrLf
            Response.Write " <changefreq>" & tmpStrFreq & "</changefreq>" & vbCrLf
            Response.Write " </url>" & vbCrLf
        Else
            Response.Write tmpStrUrl & vbCrLf
        End If
    End Sub

    ' ======================================================================
    '
    ' Returns a string array of folders under a root path
    '
    ' ======================================================================

    Function GetFolderTree(strBaseFolder)
        Dim tmpFolderCount,tmpBaseCount
        Dim tmpFolders()
        Dim tmpFSO,tmpFolder,tmpSubFolder
        ' Define the initial values for the folder counters.
        tmpFolderCount = 1
        tmpBaseCount = 0
        ' Dimension an array to hold the folder names.
        ReDim tmpFolders(1)
        ' Store the root folder in the array.
        tmpFolders(tmpFolderCount) = strBaseFolder
        ' Create file system object.
        Set tmpFSO = Server.CreateObject("Scripting.Filesystemobject")
        ' Loop while we still have folders to process.
        While tmpFolderCount <> tmpBaseCount
            ' Set up a folder object to a base folder.
            Set tmpFolder = tmpFSO.GetFolder(tmpFolders(tmpBaseCount+1))
              ' Loop through the collection of subfolders for the base folder.
            For Each tmpSubFolder In tmpFolder.SubFolders
                ' Increment the folder count.
                tmpFolderCount = tmpFolderCount + 1
                ' Increase the array size
                ReDim Preserve tmpFolders(tmpFolderCount)
                ' Store the folder name in the array.
                tmpFolders(tmpFolderCount) = tmpSubFolder.Path
            Next
            ' Increment the base folder counter.
            tmpBaseCount = tmpBaseCount + 1
        Wend
        GetFolderTree = tmpFolders
    End Function
%>

That's it. Pretty easy, eh?

I have also received several requests about creating a sitemap which contains URLs with query strings, but I'll cover that scenario in a later blog.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Rapid PHP Deployment for IIS using a Batch File

30 January 2014 • by Bob • IIS, Scripting, IIS Express, PHP

Whenever I am delivering a presentation where I need to use PHP, I typically use a batch file that I wrote in order to rapidly deploy PHP on the system that I am using for my demos. The batch file usually takes less than a second to run, which always seems to amaze people in the audience. As a result, I usually have several people ask me for my batch file after each presentation, so I thought that it would make a good subject for today's blog.

I should mention that I have used this batch file in order to demonstrate PHP with IIS in a variety of scenarios, and one of my favorite demos is when I would borrow someone's laptop and plug in a flash drive where I had IIS Express pre-installed, and then I would run the batch file in this blog to deploy PHP. Next I would launch IIS Express, open a web browser on their system, and then browse to http://localhost/ in order to show that IIS Express was working correctly. Lastly I would write a simple PHP "Hello World" page to show that PHP was up-and-running on their system in a matter of seconds.

That being said, I have to point out that there is a very important prerequisite that you must have in order to follow the steps in the blog: you need to start with a known-good installation of PHP from one of your systems, and I'll explain what I mean by that.

My batch file expects to find a folder containing ready-to-run files for PHP in order to deploy PHP on a new system. I originally obtained my PHP files by using the Web Platform Installer (WebPI) to install PHP, and then I copied the files to my flash drive or some other repository. (Note that WebPI usually installs PHP in the "%ProgramFiles(x86)%\PHP" folder.) If you don't want to use WebPI, you can also download PHP from http://windows.php.net/, but you're on your own for configuration.

Once I have the files from a known-good installation of PHP, I create the following folder structure in the location where I will be storing the files that I use to deploy PHP on other systems:

One thing to note is that the PHP.INI file you use may contain paths which refer to specific directories on the system from which you are copying your PHP files, so you need to make sure that those paths will exist on the system where you deploy PHP.

Here is an example: when I used WebPI to install PHP 5.5 on a system with IIS, it installed PHP into my "%ProgramFiles(x86)%\PHP\v5.5" folder. During the installation process, WebPI updated the PHP file to reflect any paths that need to be defined. At the time that I put together my notes for this blog, those updates mainly applied to the path where PHP expects to find it's extensions:

extension_dir="C:\Program Files (x86)\PHP\v5.5\ext\"

What this means is - if you want to deploy PHP to some other path on subsequent systems, you will need to update at least that line in the PHP.INI file that you are using to deploy PHP. In my particular case, I prefer to deploy PHP to the "%SystemDrive%\PHP" path, but it can be anywhere as long as you update everything accordingly.

The following batch file will deploy the PHP files in the "%SystemDrive%\PHP" folder on your system, and then it will update IIS with the necessary settings for this PHP deployment to work:

@echo off

REM Change to the installation folder
pushd "%~dp0"

REM Cheap test to see if IIS is installed
if exist "%SystemRoot%\System32\inetsrv" (
  REM Check for the PHP installation files in a subfolder
  if exist "%~dp0PHP" (
    REM Check for an existing installation of PHP
    if not exist "%SystemDrive%\PHP" (
      REM Create the folder for PHP
      md "%SystemDrive%\PHP"
      REM Deploy the PHP files
      xcopy /erhky "%~dp0PHP\*" "%SystemDrive%\PHP"
    )
    pushd "%SystemRoot%\System32\inetsrv"
    REM Configure the IIS settings for PHP
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000']" /commit:apphost
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000'].environmentVariables.[name='PHP_FCGI_MAX_REQUESTS',value='10000']" /commit:apphost
    appcmd.exe set config -section:system.webServer/fastCgi /+"[fullPath='%SystemDrive%\PHP\php-cgi.exe',monitorChangesTo='php.ini',activityTimeout='600',requestTimeout='600',instanceMaxRequests='10000'].environmentVariables.[name='PHPRC',value='%SystemDrive%\PHP']" /commit:apphost
    appcmd.exe set config -section:system.webServer/handlers /+"[name='PHP_via_FastCGI',path='*.php',verb='GET,HEAD,POST',modules='FastCgiModule',scriptProcessor='%SystemDrive%\PHP\php-cgi.exe',resourceType='Either']" /commit:apphost
    popd
  )
)
popd

Once you have all of that in place, it usually takes less than a second to deploy PHP, which is why so many people seem interested during my presentations.

Note that you can deploy PHP for IIS Express just as easily by updating the "%SystemRoot%\System32\inetsrv" paths in the batch file to "%ProgramFiles%\IIS Express" or "%ProgramFiles(x86)%\IIS Express" paths. You can also use this batch file as part of a deployment process for PHP within a web farm; in which case, you will need to pay attention to the paths inside your PHP.INI file which I mentioned earlier.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Revisiting My Classic ASP and URL Rewrite for Dynamic SEO Functionality Examples

20 November 2013 • by Bob • Classic ASP, IIS, SEO, URL Rewrite

Last year I wrote a blog titled Using Classic ASP and URL Rewrite for Dynamic SEO Functionality, in which I described how you could combine Classic ASP and the URL Rewrite module for IIS to dynamically create Robots.txt and Sitemap.xml files for your website, thereby helping with your Search Engine Optimization (SEO) results. A few weeks ago I had a follow-up question which I thought was worth answering in a blog post.

Overview

Here is the question that I was asked:

"What if I don't want to include all dynamic pages in sitemap.xml but only a select few or some in certain directories because I don't want bots to crawl all of them. What can I do?"

That's a great question, and it wasn't tremendously difficult for me to update my original code samples to address this request. First of all, the majority of the code from my last blog will remain unchanged - here's the file by file breakdown for the changes that need made:

FilenameChanges
Robots.asp None
Sitemap.asp See the sample later in this blog
Web.config None

So if you are already using the files from my original blog, no changes need to be made to your Robot.asp file or the URL Rewrite rules in your Web.config file because the question only concerns the files that are returned in the the output for Sitemap.xml.

Updating the Necessary Files

The good news it, I wrote most of the heavy duty code in my last blog - there were only a few changes that needed to made in order to accommodate the requested functionality. The main difference is that the original Sitemap.asp file used to have a section that recursively parsed the entire website and listed all of the files in the website, whereas this new version moves that section of code into a separate function to which you pass the unique folder name to parse recursively. This allows you to specify only those folders within your website that you want in the resultant sitemap output.

With that being said, here's the new code for the Sitemap.asp file:

<%
    Option Explicit
    On Error Resume Next
    
    Response.Clear
    Response.Buffer = True
    Response.AddHeader "Connection", "Keep-Alive"
    Response.CacheControl = "public"
    
    Dim strUrlRoot, strPhysicalRoot, strFormat
    Dim objFSO, objFolder, objFile

    strPhysicalRoot = Server.MapPath("/")
    Set objFSO = Server.CreateObject("Scripting.Filesystemobject")
    
    strUrlRoot = "http://" & Request.ServerVariables("HTTP_HOST")
    
    ' Check for XML or TXT format.
    If UCase(Trim(Request("format")))="XML" Then
        strFormat = "XML"
        Response.ContentType = "text/xml"
    Else
        strFormat = "TXT"
        Response.ContentType = "text/plain"
    End If

    ' Add the UTF-8 Byte Order Mark.
    Response.Write Chr(CByte("&hEF"))
    Response.Write Chr(CByte("&hBB"))
    Response.Write Chr(CByte("&hBF"))
    
    If strFormat = "XML" Then
        Response.Write "<?xml version=""1.0"" encoding=""UTF-8""?>" & vbCrLf
        Response.Write "<urlset xmlns=""http://www.sitemaps.org/schemas/sitemap/0.9"">" & vbCrLf
    End if
    
    ' Always output the root of the website.
    Call WriteUrl(strUrlRoot,Now,"weekly",strFormat)

    ' Output only specific folders.
    Call ParseFolder("/marketing")
    Call ParseFolder("/sales")
    Call ParseFolder("/hr/jobs")

    ' --------------------------------------------------
    ' End of file system loop.
    ' -------------------------------------------------- 
    If strFormat = "XML" Then
        Response.Write "</urlset>"
    End If
    
    Response.End

    ' ======================================================================
    '
    ' Recursively walks a folder path and return URLs based on the
    ' static *.html files that it locates.
    ' 
    ' strRootFolder = The base path for recursion
    '
    ' ======================================================================

    Sub ParseFolder(strParentFolder)
        On Error Resume Next

        Dim strChildFolders, lngChildFolders
        Dim strUrlRelative, strExt

        ' Get the list of child folders under a parent folder.
        strChildFolders = GetFolderTree(Server.MapPath(strParentFolder))

        ' Loop through the collection of folders.
        For lngChildFolders = 1 to UBound(strChildFolders)
            strUrlRelative = Replace(Mid(strChildFolders(lngChildFolders),Len(strPhysicalRoot)+1),"\","/")
            Set objFolder = objFSO.GetFolder(Server.MapPath("." & strUrlRelative))
            ' Loop through the collection of files.
            For Each objFile in objFolder.Files
                strExt = objFSO.GetExtensionName(objFile.Name)
                If StrComp(strExt,"html",vbTextCompare)=0 Then
                    If StrComp(Left(objFile.Name,6),"google",vbTextCompare)<>0 Then
                        Call WriteUrl(strUrlRoot & strUrlRelative & "/" & objFile.Name, objFile.DateLastModified, "weekly", strFormat)
                    End If
                End If
            Next
        Next

    End Sub

    ' ======================================================================
    '
    ' Outputs a sitemap URL to the client in XML or TXT format.
    ' 
    ' tmpStrFreq = always|hourly|daily|weekly|monthly|yearly|never 
    ' tmpStrFormat = TXT|XML
    '
    ' ======================================================================

    Sub WriteUrl(tmpStrUrl,tmpLastModified,tmpStrFreq,tmpStrFormat)
        On Error Resume Next
        Dim tmpDate : tmpDate = CDate(tmpLastModified)
        ' Check if the request is for XML or TXT and return the appropriate syntax.
        If tmpStrFormat = "XML" Then
            Response.Write " <url>" & vbCrLf
            Response.Write " <loc>" & Server.HtmlEncode(tmpStrUrl) & "</loc>" & vbCrLf
            Response.Write " <lastmod>" & Year(tmpLastModified) & "-" & Right("0" & Month(tmpLastModified),2) & "-" & Right("0" & Day(tmpLastModified),2) & "</lastmod>" & vbCrLf
            Response.Write " <changefreq>" & tmpStrFreq & "</changefreq>" & vbCrLf
            Response.Write " </url>" & vbCrLf
        Else
            Response.Write tmpStrUrl & vbCrLf
        End If
    End Sub

    ' ======================================================================
    '
    ' Returns a string array of folders under a root path
    '
    ' ======================================================================

    Function GetFolderTree(strBaseFolder)
        Dim tmpFolderCount,tmpBaseCount
        Dim tmpFolders()
        Dim tmpFSO,tmpFolder,tmpSubFolder
        ' Define the initial values for the folder counters.
        tmpFolderCount = 1
        tmpBaseCount = 0
        ' Dimension an array to hold the folder names.
        ReDim tmpFolders(1)
        ' Store the root folder in the array.
        tmpFolders(tmpFolderCount) = strBaseFolder
        ' Create file system object.
        Set tmpFSO = Server.CreateObject("Scripting.Filesystemobject")
        ' Loop while we still have folders to process.
        While tmpFolderCount <> tmpBaseCount
            ' Set up a folder object to a base folder.
            Set tmpFolder = tmpFSO.GetFolder(tmpFolders(tmpBaseCount+1))
              ' Loop through the collection of subfolders for the base folder.
            For Each tmpSubFolder In tmpFolder.SubFolders
                ' Increment the folder count.
                tmpFolderCount = tmpFolderCount + 1
                ' Increase the array size
                ReDim Preserve tmpFolders(tmpFolderCount)
                ' Store the folder name in the array.
                tmpFolders(tmpFolderCount) = tmpSubFolder.Path
            Next
            ' Increment the base folder counter.
            tmpBaseCount = tmpBaseCount + 1
        Wend
        GetFolderTree = tmpFolders
    End Function
%>

It should be easily seen that the code is largely unchanged from my previous blog.

In Closing...

One last thing to consider, I didn't make any changes to the Robots.asp file in this blog. But that being said, when you do not want specific paths crawled, you should add rules to your Robots.txt file to disallow those paths. For example, here is a simple Robots.txt file which allows your entire website:

# Robots.txt
# For more information on this file see:
# http://www.robotstxt.org/

# Define the sitemap path
Sitemap: http://localhost:53644/sitemap.xml

# Make changes for all web spiders
User-agent: *
Allow: /
Disallow:

If you were going to deny crawling on certain paths, you would need to add the specific paths that you do not want crawled to your Robots.txt file like the following example:

# Robots.txt
# For more information on this file see:
# http://www.robotstxt.org/

# Define the sitemap path
Sitemap: http://localhost:53644/sitemap.xml

# Make changes for all web spiders
User-agent: *
Disallow: /foo
Disallow: /bar

With that being said, if you are using my Robots.asp file from my last blog, you would need to update the section of code that defines the paths like my previous example:

Response.Write "# Make changes for all web spiders" & vbCrLf
Response.Write "User-agent: *" & vbCrLf
Response.Write "Disallow: /foo" & vbCrLf
Response.Write "Disallow: /bar" & vbCrLf

I hope this helps. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Updating my HTML Application for Configuring your WebDAV Redirector Settings

20 September 2013 • by Bob • IIS, Scripting, WebDAV

A couple of years ago I wrote a blog that was titled "How to create an HTML Application to configure your WebDAV Redirector settings", where I showed how to use HTMLA to create a simple editor for most of the WebDAV Redirector settings. These settings have no other user interface, so prior to my blog post users had to manually edit the registry in order to modify their WebDAV Redirector settings.

Click image to expand

In the past two years since I wrote that blog, I have found myself using that simple application so often that I now keep it in my personal utilities folder on my SkyDrive so I can have it with me no matter where I am travelling. But that being said, I ran into an interesting situation the other day that made me want to update the application, so I thought that it was time to write a new blog with the updated changes.

Here's what happened - I had opened my application for modifying my WebDAV Redirector settings, but then something happened which distracted me, and then I headed off to lunch before I committed my changes to the registry. When I came back to my office, I noticed that my WebDAV Redirector settings application was still open and I clicked the Exit Application button. The application popped up a dialog which informed me that I had changes that hadn't been saved to the registry, but I forgot what they were. This put me in a quandary - I could simply click Yes and hope for the best, or I could click No and lose whatever changes that I had made and re-open the application to start over.

It was at that time that I thought to myself, "If only I had a Reset Values button..."

By now you can probably see where this blog is going, and here's what the new application looks like - it's pretty much the same as the last application, with the additional button that allows you to reset your values without exiting the application. (Note - the application will prompt you for confirmation if you attempt to reset the values and you have unsaved changes.)

Click image to expand

Creating the Updated HTML Application

To create this HTML Application, you need to use the same steps as my last blog: save the following HTMLA code as "WebDAV Redirector Settings.hta" to your computer, and then double-click its icon to run the application.

<html>

<head>
<title>WebDAV Redirector Settings</title>
<HTA:APPLICATION
  APPLICATIONNAME="WebDAV Redirector Settings"
  ID="WebDAV Redirector Settings"
  VERSION="1.0"
  BORDER="dialog"
  BORDERSTYLE="static"
  INNERBORDER="no"
  SYSMENU="no"
  MAXIMIZEBUTTON="no"
  MINIMIZEBUTTON="no"
  SCROLL="no"
  SCROLLFLAT="yes"
  SINGLEINSTANCE="yes"
  CONTEXTMENU="no"
  SELECTION="no"/>

<script language="vbscript">

' ----------------------------------------
' Start of main code section.
' ----------------------------------------

Option Explicit

Const intDialogWidth = 700
Const intDialogHeight = 620
Const HKEY_LOCAL_MACHINE = &H80000002
Const strWebClientKeyPath = "SYSTEM\CurrentControlSet\Services\WebClient\Parameters"
Const strLuaKeyPath = "Software\Microsoft\Windows\CurrentVersion\Policies\System"
Dim objRegistry
Dim blnHasChanges

' ----------------------------------------
' Start the application.
' ----------------------------------------

Sub Window_OnLoad
  On Error Resume Next
  ' Set up the UI dimensions.
  Self.resizeTo intDialogWidth,intDialogHeight
  Self.moveTo (Screen.AvailWidth - intDialogWidth) / 2, _
    (Screen.AvailHeight - intDialogHeight) / 2
  ' Retrieve the current settings.
  Document.all.TheBody.ClassName = "hide"
  Set objRegistry = GetObject( _
    "winmgmts:{impersonationLevel=impersonate}!\\.\root\default:StdRegProv")
  Call CheckForLUA()
  Call GetValues()
  Document.All.TheBody.ClassName = "show"
End Sub

' ----------------------------------------
' Check for User Access Control
' ----------------------------------------

Sub CheckForLUA()
  If GetRegistryDWORD(strLuaKeyPath,"EnableLUA",1)<> 0 Then
    MsgBox "User Access Control (UAC) is enabled on this computer." & _
      vbCrLf & vbCrLf & "UAC must be disabled in order to edit " & _
      "the registry and restart the service for the WebDAV Redirector. " & _
      "Please disable UAC before running this application again. " & _
      "This application will now exit.", _
      vbCritical, "User Access Control"
    Self.close
  End If 
End Sub

' ----------------------------------------
' Exit the application.
' ----------------------------------------

Sub ExitApplication()
  If blnHasChanges = False Then
    If MsgBox("Are you sure you want to exit?", _
      vbQuestion Or vbYesNo Or vbDefaultButton2, _
      "Exit Application") = vbNo Then
      Exit Sub
    End If
  Else
    Dim intRetVal
    intRetVal = MsgBox("You have unsaved changes. " & _
      "Do you want to save them before you exit?", _
      vbQuestion Or vbYesNoCancel Or vbDefaultButton1, _
      "Reset Application")
    If intRetVal = vbYes Then
      Call SetValues()
    ElseIf intRetVal = vbCancel Then
      Exit Sub
    End If
  End If
  Self.close
End Sub

' ----------------------------------------
' Reset the application.
' ----------------------------------------

Sub ResetApplication()
  If blnHasChanges = True Then
    Dim intRetVal
    intRetVal = MsgBox("You have unsaved changes. " & _
      "Do you want to save them before you reset the values?", _
      vbQuestion Or vbYesNoCancel Or vbDefaultButton1, _
      "Reset Application")
    If intRetVal = vbYes Then
      Call SetValues()
    ElseIf intRetVal = vbCancel Then
      Exit Sub
    End If
  End If
  Call GetValues()
End Sub

' ----------------------------------------
' Flag the application as having changes.
' ----------------------------------------

Sub FlagChanges()
  blnHasChanges = True
End Sub

' ----------------------------------------
' Retrieve the settings from the registry.
' ----------------------------------------

Sub GetValues()
  On Error Resume Next
  Dim tmpCount,tmpArray,tmpString
  ' Get the radio button values
  Call SetRadioValue(Document.all.BasicAuthLevel, _
    GetRegistryDWORD(strWebClientKeyPath, _
    "BasicAuthLevel",1))
  Call SetRadioValue(Document.all.SupportLocking, _
    GetRegistryDWORD(strWebClientKeyPath, _
    "SupportLocking",1))
  ' Get the text box values
  Document.all.InternetServerTimeoutInSec.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "InternetServerTimeoutInSec",30)
  Document.all.FileAttributesLimitInBytes.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "FileAttributesLimitInBytes",1000000)
  Document.all.FileSizeLimitInBytes.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "FileSizeLimitInBytes",50000000)
  Document.all.LocalServerTimeoutInSec.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "LocalServerTimeoutInSec",15)
  Document.all.SendReceiveTimeoutInSec.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "SendReceiveTimeoutInSec",60)
  Document.all.ServerNotFoundCacheLifeTimeInSec.Value = _
    GetRegistryDWORD(strWebClientKeyPath, _
    "ServerNotFoundCacheLifeTimeInSec",60)
  ' Get the text area values
  tmpArray = GetRegistryMULTISZ( _
    strWebClientKeyPath,"AuthForwardServerList")
  For tmpCount = 0 To UBound(tmpArray)
    tmpString = tmpString & tmpArray(tmpCount) & vbTab
  Next
  If Len(tmpString)>0 Then
    Document.all.AuthForwardServerList.Value = _
      Replace(Left(tmpString,Len(tmpString)-1),vbTab,vbCrLf)
  End If
  blnHasChanges = False
End Sub

' ----------------------------------------
' Save the settings in the registry.
' ----------------------------------------

Sub SetValues()
  On Error Resume Next
  ' Set the radio button values
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "BasicAuthLevel", _
    GetRadioValue(Document.all.BasicAuthLevel))
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "SupportLocking", _
    GetRadioValue(Document.all.SupportLocking))
  ' Set the text box values
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "InternetServerTimeoutInSec", _
    Document.all.InternetServerTimeoutInSec.Value)
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "FileAttributesLimitInBytes", _
    Document.all.FileAttributesLimitInBytes.Value)
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "FileSizeLimitInBytes", _
    Document.all.FileSizeLimitInBytes.Value)
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "LocalServerTimeoutInSec", _
    Document.all.LocalServerTimeoutInSec.Value)
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "SendReceiveTimeoutInSec", _
    Document.all.SendReceiveTimeoutInSec.Value)
  Call SetRegistryDWORD( _
    strWebClientKeyPath, _
    "ServerNotFoundCacheLifeTimeInSec", _
    Document.all.ServerNotFoundCacheLifeTimeInSec.Value)
  ' Set the text area values
  Call SetRegistryMULTISZ( _
    strWebClientKeyPath, _
    "AuthForwardServerList", _
    Split(Document.all.AuthForwardServerList.Value,vbCrLf))
  ' Prompt to restart the WebClient service
  If MsgBox("Do you want to restart the WebDAV Redirector " & _
    "service so your settings will take effect?", _
    vbQuestion Or vbYesNo Or vbDefaultButton2, _
    "Restart WebDAV Redirector") = vbYes Then
    ' Restart the WebClient service.
    Call RestartWebClient()
  End If
  Call GetValues()
End Sub

' ----------------------------------------
' Start the WebClient service.
' ----------------------------------------

Sub RestartWebClient()
  On Error Resume Next
  Dim objWMIService,colServices,objService
  Document.All.TheBody.ClassName = "hide"
  Set objWMIService = GetObject( _
    "winmgmts:{impersonationLevel=impersonate}!\\.\root\cimv2")
  Set colServices = objWMIService.ExecQuery( _
    "Select * from Win32_Service Where Name='WebClient'")
  For Each objService in colServices
    objService.StopService()
    objService.StartService()
  Next
  Document.All.TheBody.ClassName = "show"
End Sub

' ----------------------------------------
' Retrieve a DWORD value from the registry.
' ----------------------------------------

Function GetRegistryDWORD( _
    ByVal tmpKeyPath, _
    ByVal tmpValueName, _
    ByVal tmpDefaultValue)
  On Error Resume Next
  Dim tmpDwordValue
  If objRegistry.GetDWORDValue( _
      HKEY_LOCAL_MACHINE, _
      tmpKeyPath, _
      tmpValueName, _
      tmpDwordValue)=0 Then
    GetRegistryDWORD = CLng(tmpDwordValue)
  Else
    GetRegistryDWORD = CLng(tmpDefaultValue)
  End If
End Function

' ----------------------------------------
' Set a DWORD value in the registry.
' ----------------------------------------

Sub SetRegistryDWORD( _
    ByVal tmpKeyPath, _
    ByVal tmpValueName, _
    ByVal tmpDwordValue)
  On Error Resume Next
  Call objRegistry.SetDWORDValue( _
    HKEY_LOCAL_MACHINE, _
    tmpKeyPath, _
    tmpValueName, _
    CLng(tmpDwordValue))
End Sub

' ----------------------------------------
' Retrieve a MULTISZ value from the registry.
' ----------------------------------------

Function GetRegistryMULTISZ( _
    ByVal tmpKeyPath, _
    ByVal tmpValueName)
  On Error Resume Next
  Dim tmpMultiSzValue
  If objRegistry.GetMultiStringValue( _
      HKEY_LOCAL_MACHINE, _
      tmpKeyPath, _
      tmpValueName, _
      tmpMultiSzValue)=0 Then
    GetRegistryMULTISZ = tmpMultiSzValue
  Else
    GetRegistryMULTISZ = Array()
  End If
End Function

' ----------------------------------------
' Set a MULTISZ value in the registry.
' ----------------------------------------

Sub SetRegistryMULTISZ( _
    ByVal tmpKeyPath, _
    ByVal tmpValueName, _
    ByVal tmpMultiSzValue)
  On Error Resume Next
  Call objRegistry.SetMultiStringValue( _
    HKEY_LOCAL_MACHINE, _
    tmpKeyPath, _
    tmpValueName, _
    tmpMultiSzValue)
End Sub

' ----------------------------------------
' Retrieve the value of a radio button group.
' ----------------------------------------

Function GetRadioValue(ByVal tmpRadio)
  On Error Resume Next
  Dim tmpCount
  For tmpCount = 0 To (tmpRadio.Length-1)
    If tmpRadio(tmpCount).Checked Then
      GetRadioValue = CLng(tmpRadio(tmpCount).Value)
      Exit For
    End If
  Next
End Function

' ----------------------------------------
' Set the value for a radio button group.
' ----------------------------------------

Sub SetRadioValue(ByVal tmpRadio, ByVal tmpValue)
  On Error Resume Next
  Dim tmpCount
  For tmpCount = 0 To (tmpRadio.Length-1)
    If CLng(tmpRadio(tmpCount).Value) = CLng(tmpValue) Then
      tmpRadio(tmpCount).Checked = True
      Exit For
    End If
  Next
End Sub

' ----------------------------------------
'
' ----------------------------------------

Sub Validate(tmpField)
  Dim tmpRegEx, tmpMatches
  Set tmpRegEx = New RegExp
  tmpRegEx.Pattern = "[0-9]"
  tmpRegEx.IgnoreCase = True
  tmpRegEx.Global = True
  Set tmpMatches = tmpRegEx.Execute(tmpField.Value)
  If tmpMatches.Count = Len(CStr(tmpField.Value)) Then
    If CDbl(tmpField.Value) => 0 And _
      CDbl(tmpField.Value) =< 4294967295 Then
      Exit Sub
    End If
  End If
  MsgBox "Please enter a whole number between 0 and 4294967295.", _
    vbCritical, "Validation Error"
  tmpField.Focus
End Sub

' ----------------------------------------
'
' ----------------------------------------

Sub BasicAuthWarning()
  MsgBox "WARNING:" & vbCrLf  & vbCrLf & _
    "Using Basic Authentication over non-SSL connections can cause " & _
    "serious security issues. Usernames and passwords are transmitted " & _
    "in clear text, therefore the use of Basic Authentication with " & _
    "WebDAV is disabled by default for non-SSL connections. That " & _
    "being said, this setting can override the default behavior for " & _
    "Basic Authentication, but it is strongly discouraged.", _
    vbCritical, "Basic Authentication Warning"
End Sub

' ----------------------------------------
' End of main code section.
' ----------------------------------------

</script>
<style>
body { color:#000000; background-color:#cccccc;
  font-family:'Segoe UI',Tahoma,Verdana,Arial; font-size:9pt; }
fieldset { padding:10px; width:640px; }
.button { width:150px; }
.textbox { width:200px; height:22px; text-align:right; }
.textarea { width:300px; height:50px; text-align:left; }
.radio { margin-left:-5px; margin-top: -2px; }
.hide { display:none; }
.show { display:block; }
select { width:300px; text-align:left; }
table { border-collapse:collapse; empty-cells:hide; }
h1 { font-size:14pt; }
th { font-size:9pt; text-align:left; vertical-align:top; padding:2px; }
td { font-size:9pt; text-align:left; vertical-align:top; padding:2px; }
big { font-size:11pt; }
small { font-size:8pt; }
</style>
</head>

<body id="TheBody" class="hide">

<h1 align="center" id="TheTitle" style="margin-bottom:-20px;">WebDAV Redirector Settings</h1>
<div align="center">
<p style="margin-bottom:-20px;"><i><small><b>Note</b>: See <a target="_blank" href="http://go.microsoft.com/fwlink/?LinkId=324291">Using the WebDAV Redirector</a> for additional details.</small></i></p>
  <form>
    <center>
    <table border="0" cellpadding="2" cellspacing="2" style="width:600px;">
      <tr>
        <td style="width:600px;text-align:left"><fieldset title="Security Settings">
        <legend>&nbsp;<b>Security Settings</b>&nbsp;</legend>
        These values affect the security behavior for the WebDAV Redirector.<br>
        <table style="width:600px;">
          <tr title="Specifies whether the WebDAV Redirector can use Basic Authentication to communicate with a server.">
            <td style="width:300px">
            <table border="0">
              <tr>
                <td style="width:300px"><b>Basic Authentication Level</b></td>
              </tr>
              <tr>
                <td style="width:300px;"><span style="width:280px;padding-left:20px;"><small><i><b>Note</b>: Using basic authentication can cause <u>serious security issues</u> as the username and password are transmitted in clear text, therefore the use of basic authentication over WebDAV is disabled by default unless the connection is using SSL.</i></small></span></td>
              </tr>
            </table>
            </td>
            <td style="width:300px">
            <table style="width:300px">
              <tr>
                <td style="width:020px"><input class="radio" type="radio" value="0" name="BasicAuthLevel" onchange="VBScript:FlagChanges()" id="BasicAuthLevel0"></td>
                <td style="width:280px"><label for="BasicAuthLevel0">Basic Authentication is disabled</label></td>
              </tr>
              <tr>
                <td style="width:020px"><input class="radio" type="radio" value="1" checked name="BasicAuthLevel" onchange="VBScript:FlagChanges()" id="BasicAuthLevel1"></td>
                <td style="width:280px"><label for="BasicAuthLevel1">Basic Authentication is enabled for SSL web sites only</label></td>
              </tr>
              <tr>
                <td style="width:020px"><input class="radio" type="radio" value="2" name="BasicAuthLevel" onchange="VBScript:FlagChanges()" id="BasicAuthLevel2" onClick="VBScript:BasicAuthWarning()"></td>
                <td style="width:280px"><label for="BasicAuthLevel2">Basic Authentication is enabled for SSL and non-SSL web sites</label></td>
              </tr>
            </table>
            </td>
          </tr>
          <tr title="Specifies a list of local URLs for forwarding credentials that bypasses any proxy settings. (Note: This requires Windows Vista SP1 or later.)">
            <td style="width:300px">
            <table border="0">
              <tr>
                <td style="width:300px"><b>Authentication Forwarding Server List</b></td>
              </tr>
              <tr>
                <td style="width:300px;"><span style="width:280px;padding-left:20px;"><small><i><b>Note</b>: Include one server name per line.</i></small></span></td>
              </tr>
            </table>
            </td>
            <td style="width:300px"><textarea class="textarea" name="AuthForwardServerList" onchange="VBScript:FlagChanges()"></textarea></td>
          </tr>
          <tr title="Specifies whether the WebDAV Redirector supports locking.">
            <td style="width:300px"><b>Support for WebDAV Locking</b></td>
            <td style="width:300px">
            <table style="width:300px">
              <tr>
                <td style="width:020px"><input class="radio" type="radio" value="1" checked name="SupportLocking" onchange="VBScript:FlagChanges()" id="SupportLocking1"></td>
                <td style="width:280px"><label for="SupportLocking1">Enable Lock Support</label></td>
              </tr>
              <tr>
                <td style="width:020px"><input class="radio" type="radio" value="0" name="SupportLocking" onchange="VBScript:FlagChanges()" id="SupportLocking0"></td>
                <td style="width:280px"><label for="SupportLocking0">Disable Lock Support</label></td>
              </tr>
            </table>
            </td>
          </tr>
        </table>
        </fieldset> </td>
      </tr>
      <tr>
        <td style="width:600px;text-align:left"><fieldset title="Time-outs">
        <legend>&nbsp;<b>Time-outs and Maximum Sizes</b>&nbsp;</legend>
        These values affect the behavior for WebDAV Client/Server operations.<br>
        <table border="0" style="width:600px;">
          <tr title="Specifies the connection time-out for the WebDAV Redirector uses when communicating with non-local WebDAV servers.">
            <td style="width:300px"><b>Internet Server Time-out</b> <small>(In Seconds)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="InternetServerTimeoutInSec" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="30"></td>
          </tr>
          <tr title="Specifies the connection time-out for the WebDAV Redirector uses when communicating with a local WebDAV server.">
            <td style="width:300px"><b>Local Server Time-out</b> <small>(In Seconds)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="LocalServerTimeoutInSec" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="15"></td>
          </tr>
          <tr title="Specifies the time-out in seconds that the WebDAV Redirector uses after issuing a request.">
            <td style="width:300px"><b>Send/Receive Time-out</b> <small>(In Seconds)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="SendReceiveTimeoutInSec" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="60"></td>
          </tr>
          <tr title="Specifies the period of time that a server is cached as non-WebDAV by the WebDAV Redirector. If a server is found in this list, a fail is returned immediately without attempting to contact the server.">
            <td style="width:300px"><b>Server Not Found Cache Time-out</b> <small>(In Seconds)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="ServerNotFoundCacheLifeTimeInSec" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="60"></td>
          </tr>
          <tr title="Specifies the maximum size in bytes that the WebDAV Redirector allows for file transfers.">
            <td style="width:300px"><b>Maximum File Size</b> <small>(In Bytes)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="FileSizeLimitInBytes" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="50000000"></td>
          </tr>
          <tr title="Specifies the maximum size that is allowed by the WebDAV Redirector for all properties on a specific collection.">
            <td style="width:300px"><b>Maximum Attributes Size</b> <small>(In Bytes)</small></td>
            <td style="width:300px"><input class="textbox" type="text" name="FileAttributesLimitInBytes" onchange="VBScript:FlagChanges()" onblur="VBScript:Validate(Me)" value="1000000"></td>
          </tr>
        </table>
        </fieldset> </td>
      </tr>
      <tr>
        <td style="text-align:center">
        <table border="0">
          <tr>
            <td style="text-align:center"><input class="button" type="button" value="Apply Settings" onclick="VBScript:SetValues()">
            <td style="text-align:center"><input class="button" type="button" value="Reset Values" onclick="VBScript:ResetApplication()">
            <td style="text-align:center"><input class="button" type="button" value="Exit Application" onclick="VBScript:ExitApplication()">
          </tr>
        </table>
        </td>
      </tr>
    </table>
    </center>
  </form>
</div>

</body>

</html>
Additional Notes

As with the last version of this HTML Application, you will need to run this application as an administrator in order to save the settings to the registry and restart the WebDAV Redirector service.

Have fun! ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

How to Create a Blind Drop WebDAV Share

17 September 2013 • by Bob • IIS, WebDAV, URL Rewrite

I had an interesting WebDAV question earlier today that I had not considered before: how can someone create a "Blind Drop Share" using WebDAV? By way of explanation, a Blind Drop Share is a path where users can copy files, but never see the files that are in the directory. You can setup something like this by using NTFS permissions, but that environment can be a little difficult to set up and maintain.

With that in mind, I decided to research a WebDAV-specific solution that didn't require mangling my NTFS permissions. In the end it was pretty easy to achieve, so I thought that it would make a good blog for anyone who wants to do this.

A Little Bit About WebDAV

NTFS permissions contain access controls that configure the directory-listing behavior for files and folders; if you modify those settings, you can control who can see files and folders when they connect to your shared resources. However, there are no built-in features for the WebDAV module which ships with IIS that will approximate the NTFS behavior. But that being said, there is an interesting WebDAV quirk that you can use that will allow you to restrict directory listings, and I will explain how that works.

WebDAV uses the PROPFIND command to retrieve the properties for files and folders, and the WebDAV Redirector will use the response from a PROPFIND command to display a directory listing. (Note: Official WebDAV terminology has no concept of files and folders, those physical objects are respectively called Resources and Collections in WebDAV jargon. But that being said, I will use files and folders throughout this blog post for ease of understanding.)

In any event, one of the HTTP headers that WebDAV uses with the PROPFIND command is the Depth header, which is used to specify how deep the folder/collection traversal should go:

By analyzing the above information, it should be obvious that what you need to do is to restrict users to using a Depth:0 header/value. But that's where this scenario gets interesting: if your end-users are using the Windows WebDAV Redirector or other similar technology to map a drive to your HTTP website, you have no control over the value of the Depth header. So how can you restrict that?

In the past I would have written custom native-code HTTP module or ISAPI filter to modify the value of the Depth header; but once you understand how WebDAV works, you can use the URL Rewrite module to modify the headers of incoming HTTP requests to accomplish some pretty cool things - like modifying the values WebDAV-related HTTP headers.

Adding URL Rewrite Rules to Modify the WebDAV Depth Header

Here's how I configured URL Rewrite to set the value of the Depth header to 0, which allowed me to create a "Blind Drop" WebDAV site:

  1. Open the URL Rewrite feature in IIS Manager for your website.
    Click image to expand
  2. Click the Add Rules link in the Actionspane.
    Click image to expand
  3. When the Add Rules dialog box appears, highlight Blank rule and click OK.
    Click image to expand
  4. When the Edit Inbound Rulepage appears, configure the following settings:
    1. Name the rule "Modify Depth Header".
      Click image to expand
    2. In the Match URLsection:
      1. Choose Matches the Pattern in the Requested URL drop-down menu.
      2. Choose Wildcards in the Using drop-down menu.
      3. Type a single asterisk "*" in the Pattern text box.
      Click image to expand
    3. Expand the Server Variables collection and click Add.
      Click image to expand
    4. When the Set Server Variabledialog box appears:
      1. Type "HTTP_DEPTH" in the Server variable name text box.
      2. Type "0" in the Value text box.
      3. Make sure that Replace the existing value checkbox is checked.
      4. Click OK.
    5. In the Action group, choose None in the Action typedrop-down menu.
      Click image to expand
    6. Click Apply in the Actions pane, and then click Back to Rules.
      Click image to expand
  5. Click View Server Variables in the Actionspane.
    Click image to expand
  6. When the Allowed Server Variablespage appears, configure the following settings:
    1. Click Add in the Actionspane.
      Click image to expand
    2. When the Add Server Variabledialog box appears:
      1. Type "HTTP_DEPTH" in the Server variable name text box.
      2. Click OK.
    3. Click Back to Rules in the Actionspane.
      Click image to expand

If all of these changes were saved to your applicationHost.config file, the resulting XML might resemble the following example - with XML comments added by me to highlight some of the major sections:

<location path="Default Web Site">
    <system.webServer>
    
        <-- Start of Security Settings -->
        <security>
            <authentication>
                <anonymousAuthentication enabled="false" />
                <basicAuthentication enabled="true" />
            </authentication>
            <requestFiltering>
                <fileExtensions applyToWebDAV="false" />
                <verbs applyToWebDAV="false" />
                <hiddenSegments applyToWebDAV="false" />
            </requestFiltering>
        </security>
        
        <-- Start of WebDAV Settings -->
        <webdav>
            <authoringRules>
                <add roles="administrators" path="*" access="Read, Write, Source" />
            </authoringRules>
            <authoring enabled="true">
                <properties allowAnonymousPropfind="false" allowInfinitePropfindDepth="true">
                    <clear />
                    <add xmlNamespace="*" propertyStore="webdav_simple_prop" />
                </properties>
            </authoring>
        </webdav>
        
        <-- Start of URL Rewrite Settings -->
        <rewrite>
            <rules>
                <rule name="Modify Depth Header" enabled="true" patternSyntax="Wildcard">
                    <match url="*" />
                    <serverVariables>
                        <set name="HTTP_DEPTH" value="0" />
                    </serverVariables>
                    <action type="None" />
                </rule>
            </rules>
            <allowedServerVariables>
                <add name="HTTP_DEPTH" />
            </allowedServerVariables>
        </rewrite>
        
    </system.webServer>
</location>

In all likelihood, some of these settings will be stored in your applicationHost.config file, and the remaining settings will be stored in the web.config file of your website.

Testing the URL Rewrite Settings

If you did not have the URL Rewrite rule in place, or if you disabled the rule, then your web server might respond like the following example if you used the WebDAV Redirector to map a drive to your website from a command prompt:

C:\>net use z: http://www.contoso.com/
Enter the user name for 'www.contoso.com': www.contoso.com\robert
Enter the password for www.contoso.com:
The command completed successfully.

C:\>z:

Z:\>dir
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\

09/16/2013 08:55 PM <DIR> .
09/16/2013 08:55 PM <DIR> ..
09/14/2013 12:39 AM <DIR> aspnet_client
09/16/2013 08:06 PM <DIR> scripts
09/16/2013 07:55 PM 66 default.aspx
09/14/2013 12:38 AM 98,757 iis-85.png
09/14/2013 12:38 AM 694 iisstart.htm
09/16/2013 08:55 PM 75 web.config
              4 File(s) 99,592 bytes
              8 Dir(s) 956,202,631,168 bytes free

Z:\>

However, when you have the URL Rewrite correctly configured and enabled, connecting to the same website will resemble the following example - notice how no files or folders are listed:

C:\>net use z: http://www.contoso.com/
Enter the user name for 'www.contoso.com': www.contoso.com\robert
Enter the password for www.contoso.com:
The command completed successfully.

C:\>z:

Z:\>dir
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\

09/16/2013 08:55 PM <DIR> .
09/16/2013 08:55 PM <DIR> ..
              0 File(s) 0 bytes
              2 Dir(s) 956,202,803,200 bytes free

Z:\>

Despite the blank directory listing, you can still retrieve the properties for any file or folder if you know that it exists. So if you were to use the mapped drive from the preceding example, you could still use an explicit directory command for any object that you had uploaded or created:

Z:\>dir default.aspx
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\

09/16/2013 07:55 PM 66 default.aspx
              1 File(s) 66 bytes
              0 Dir(s) 956,202,799,104 bytes free

Z:\>dir scripts
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\scripts

09/16/2013 07:52 PM <DIR> .
09/16/2013 07:52 PM <DIR> ..
              0 File(s) 0 bytes
              2 Dir(s) 956,202,799,104 bytes free

Z:\>

The same is true for creating directories and files; you can create them, but they will not show up in the directory listings after you have created them unless you reference them explicitly:

Z:\>md foobar

Z:\>dir
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\

09/16/2013 11:52 PM <DIR> .
09/16/2013 11:52 PM <DIR> ..
              0 File(s) 0 bytes
              2 Dir(s) 956,202,618,880 bytes free

Z:\>cd foobar

Z:\foobar>copy NUL foobar.txt
        1 file(s) copied.

Z:\foobar>dir
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\foobar

09/16/2013 11:52 PM <DIR> .
09/16/2013 11:52 PM <DIR> ..
              0 File(s) 0 bytes
              2 Dir(s) 956,202,303,488 bytes free

Z:\foobar>dir foobar.txt
Volume in drive Z has no label.
Volume Serial Number is 0000-0000

Directory of Z:\foobar

09/16/2013 11:53 PM 0 foobar.txt
              1 File(s) 0 bytes
              0 Dir(s) 956,202,299,392 bytes free

Z:\foobar>

That wraps it up for today's post, although I should point out that if you see any errors when you are using the WebDAV Redirector, you should take a look at the Troubleshooting the WebDAV Redirector section of my Using the WebDAV Redirector article; I have done my best to list every error and resolution that I have discovered over the past several years.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

What's New in IIS 8.5?

11 July 2013 • by Bob • IIS, IIS 8.5, Windows Server 2012 R2

Microsoft released the Windows 8.1 Preview on June 26, but some of the articles that the IIS team was working on for IIS 8.5 needed a little more work before we published them. That being said, those articles were published on IIS.net earlier today, and here is a quick preview of those articles and the new features in IIS 8.5 which they describe:

For examples of these technologies in action, you can watch one of the following presentations:

Special thanks go to Gary Ericson for putting together and publishing these articles on IIS.net, and a big round of thanks to the IIS team for adding all of these new features!

(One quick note in closing: functionality similar to IIS 8.5's Enhanced Logging was available in the past by installing separate modules, but IIS 8.5 now makes this functionality available out-of-the-box.)

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Blog Navigation

You are on page 1 of 9 pages.

1 2 3 4 5 6 7 8 9