Geeky Bob

Converting W3C log files to NCSA format

07 November 2006 • by Bob • IIS, LogParser

Around a year ago I wrote a blog entry titled "Converting NCSA log files to W3C format", which showed how to use the MSWC.IISLog object to convert log files in the NCSA format back to W3C format. I wrote that blog entry to make up for the fact that the CONVLOG.EXE utility only converts log files to NCSA format, which some older log analysis software packages require. So what happens if you have a bunch of log files in W3C format and you don't have a copy of CONVLOG.EXE on your computer?

This blog entry is something of a reverse direction on my previous post, and shows you how to use the MSUtil.LogQuery object to convert W3C log files to NCSA format. The MSUtil.LogQuery object is shipped with LogParser, which you can download from one of the following URLs:

Version 2.2 and 2.0: http://www.microsoft.com/download, search for "Log Parser"
Version 2.1: http://www.microsoft.com/download, search for "IIS 6.0 Resource Kit Tools"

Once you've downloaded and installed the LogParser package, you will need to manually register the LogParser.dll file in order to use the MSUtil.LogQuery object. Having done so, you can use the Windows Script Host (WSH) code in this blog article to convert a folder filled with W3C log files to NCSA format.

To use this code, just copy the code into notepad, and save it with a ".vbs" file extension on your system. To run it, copy the script to a folder that contains your W3C log files, (named "ex*.log"), then double-click it.

Option Explicit
Dim objFSO
Dim objFolder
Dim objInputFile
Dim objOutputFile
Dim objLogQuery
Dim objLogRecordSet
Dim objLogRecord
Dim strInputPath
Dim strOutputPath
Dim strLogRecord
Dim strLogTemp

Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(".")

For Each objInputFile In objFolder.Files
 strInputPath = LCase(objInputFile.Name)
 If Left(strInputPath,2) = "ex" And Right(strInputPath,4) = ".log" Then
  strOutputPath = objFolder.Path & "\" & "nc" & Mid(strInputPath,3)
  strInputPath = objFolder.Path & "\" & strInputPath
  Set objLogQuery = CreateObject("MSUtil.LogQuery")
  Set objLogRecordSet = objLogQuery.Execute("SELECT * FROM " & strInputPath)
  Set objOutputFile = objFSO.CreateTextFile(strOutputPath)
  Do While Not objLogRecordSet.atEnd
  
   Set objLogRecord = objLogRecordSet.getRecord
   strLogRecord = FormatField(objLogRecord.getValue("c-ip"))
   strLogRecord = strLogRecord & " " & FormatField("")
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("cs-username"))
   strLogTemp = BuildDateTime(objLogRecord.getValue("date"),objLogRecord.getValue("time"))
   strLogRecord = strLogRecord & " " & FormatField(strLogTemp)
   strLogRecord = strLogRecord & " """ & FormatField(objLogRecord.getValue("cs-method"))
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("cs-uri-stem"))
   strLogTemp = FormatField(objLogRecord.getValue("cs-version"))
   If strLogTemp = "-" Then
    strLogRecord = strLogRecord & " HTTP/1.0"""
   Else
    strLogRecord = strLogRecord & " " & strLogTemp & """"
   End If   
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("sc-status"))
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("sc-bytes"))
    objOutputFile.WriteLine strLogRecord
   objLogRecordSet.moveNext
  Loop
  
  Set objLogQuery = Nothing
  objOutputFile.Close
 
 End If
Next

Function FormatField(tmpField)
 On Error Resume Next
 FormatField = "-"
 If Len(tmpField) > 0 Then FormatField = Trim(tmpField)
End Function

Function BuildDateTime(tmpDate,tmpTime)
 On Error Resume Next
 BuildDateTime = "[" & _
  Right("0" & Day(tmpDate),2) & "/" & _
  Left(MonthName(Month(tmpDate)),3) & "/" & _
  Year(tmpDate) & ":" & _
  Right("0" & Hour(tmpTime),2) & ":" & _
  Right("0" & Minute(tmpTime),2) & ":" & _
  Right("0" & Second(tmpTime),2) & _
  " +0000]"
End Function

I hope this helps!

Programmatically Enumerating Installations of the FrontPage Server Extensions

12 July 2006 • by Bob • FrontPage, Scripting

I had a great question from a customer the other day: "How do you programmatically enumerate how many web sites on a server have the FrontPage Server Extensions installed?" Of course, that's one of those questions that sounds so simple at first, and then you start to think about how to actually go about it and it gets a little more complicated.

The first thought that came to mind was to just look for all the "W3SVCnnnn" subfolders that are located in the "%ALLUSERSPROFILE%\Application Data\Microsoft\Web Server Extensions\50" folder. (These folders contain the "ROLES.INI" files for each installation.) The trouble with this solution is that some folders and files do not get cleaned up when the server extensions are uninstalled, so you'd get erroneous results.

The next thought that came to mind was to check the registry, because each installation of the server extensions will create a string value and subkey named "Port /LM/W3SVC/nnnn:" under the "[HKLM\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\Ports]" key. Enumerating these keys will give you the list of web sites that have the server extensions or SharePoint installed. The string values that are located under the subkey contain some additional useful information, so I thought that as long as I was enumerating the keys, I might as well enumerate those values.

The resulting script is listed below, and when run it will create a log file that lists all of the web sites that have the server extensions or SharePoint installed on the server that is specified by the "strComputer" constant.

Option Explicit

Const strComputer = "localhost"

Dim objFSO, objFile
Dim objRegistry
Dim strRootKeyPath, strSubKeyPath, strValue
Dim arrRootValueTypes, arrRootValueNames
Dim arrSubValueTypes, arrSubValueNames
Dim intLoopA, intLoopB

Const HKEY_LOCAL_MACHINE = &H80000002
Const REG_SZ = 1

strRootKeyPath = "Software\Microsoft\" & _
  "Shared Tools\Web Server Extensions\Ports"

Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.CreateTextFile("ServerExtensions.Log")

objFile.WriteLine String(40,"-")
objFile.WriteLine "Report for server: " & UCase(strComputer)
objFile.WriteLine String(40,"-")

Set objRegistry = GetObject(_
  "winmgmts:{impersonationLevel=impersonate}!\\" & _
  strComputer & "\root\default:StdRegProv")
objRegistry.EnumValues HKEY_LOCAL_MACHINE, strRootKeyPath, _
  arrRootValueNames, arrRootValueTypes

For intLoopA = 0 To UBound(arrRootValueTypes)
  If arrRootValueTypes(intLoopA) = REG_SZ Then
    objFile.WriteLine arrRootValueNames(intLoopA)
    strSubKeyPath = strRootKeyPath & _
      "\" & arrRootValueNames(intLoopA)
    objRegistry.EnumValues HKEY_LOCAL_MACHINE, _
      strSubKeyPath, arrSubValueNames, arrSubValueTypes
    For intLoopB = 0 To UBound(arrSubValueTypes)
      If arrSubValueTypes(intLoopB) = REG_SZ Then
        objRegistry.GetStringValue HKEY_LOCAL_MACHINE, _
          strSubKeyPath, arrSubValueNames(intLoopB), strValue
        objFile.WriteLine vbTab & _
          arrSubValueNames(intLoopB) & "=" & strValue
      End If
    Next
    objFile.WriteLine String(40,"-")
  End If
Next

objFile.Close

The script should be fairly easy to understand, and you can customize it to suit your needs. For example, you could change the "strComputer" constant to a string array and loop through an array of servers.

Note: More information about the WMI objects used in the script can be found on the following pages:

Hope this helps!

IIS 6: Reverting log files back to the default W3C fields

27 March 2006 • by Bob • IIS, Scripting

Recently I had to work with a customer that was trying to use a 3rd-party utility that read W3C log files and it was failing to complete processing. I had the customer send me his log files, and upon examination I discovered that the trouble was occuring because the customer had been experimenting with adding and removing the different fields from their log files and this was causing the log parsing utility to crash.

As luck would have it, IIS provides a useful logging utility object that you can read more about at the following URL:

I had used this logging utility object for an earlier project, so I was familiar with how it worked. With that knowledge in mind, I wrote the following script that loops through all of the log files in a folder and creates new log files in a subfolder that contain only the default W3C fields. (BTW - I sent this script to the customer and he was able to parse all of his log files successfully. ;-] )

Option Explicit
Randomize Timer
 
' Declare variables.
 
Dim objIISLog
Dim objFSO, objFolder, objFile
Dim objOutputFile, strInputFile
Dim strOutputFile, strOutputPath
Dim strLogRecord
Dim blnExists
 
' Create file system object.
Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
' Retrieve an object For the current folder.
Set objFolder = objFSO.GetFolder(".")
 
' Create a subfolder with a random name.
blnExists = True
Do While blnExists = True
 strOutputPath = objFolder.Path & "\" & CreateRandomName(20)
 blnExists = objFSO.FolderExists(strOutputPath)
Loop
objFSO.CreateFolder strOutputPath
 
' Loop through the log files in the current folder.
For Each objFile In objFolder.Files
 
 ' Test for a log file.
 If Right(LCase(objFile.Name),4) = ".log" Then
 
  ' Format the file names/paths.
  strInputFile = objFolder.Path & "\" & objFile.Name
  strOutputFile = strOutputPath & "\" & objFile.Name
 
  ' Create and open an IIS logging object.
  Set objIISLog = CreateObject("MSWC.IISLog")
  ' Open the input log file.
  objIISLog.OpenLogFile strInputFile, 1, "", 0, ""  
  ' Open the output log file.
  Set objOutputFile = objFSO.CreateTextFile(strOutputFile)
 
  ' Read the initial record from the log file.
  objIISLog.ReadLogRecord
 
  ' Write the headers to the output log file.
  objOutputFile.WriteLine "#Software: Microsoft Internet Information Services 5.0"
  objOutputFile.WriteLine "#Version: 1.0"
  objOutputFile.WriteLine "#Date: " & BuildDateTime(objIISLog.DateTime)
  objOutputFile.WriteLine "#Fields: date time c-ip cs-username s-ip s-port " & _
   "cs-method cs-uri-stem cs-uri-query sc-status cs(User-Agent)"
 
  ' Loop through the records in the log file.
  Do While Not objIISLog.AtEndOfLog
 
   ' Format the log file fields.
   strLogRecord = BuildDateTime(objIISLog.DateTime)
   strLogRecord = strLogRecord & _
    " " & FormatField(objIISLog.ClientIP) & _
    " " & FormatField(objIISLog.UserName) & _
    " " & FormatField(objIISLog.ServerIP) & _
    " " & FormatField(objIISLog.ServerPort) & _
    " " & FormatField(objIISLog.Method) & _
    " " & FormatField(objIISLog.URIStem) & _
    " " & FormatField(objIISLog.URIQuery) & _
    " " & FormatField(objIISLog.ProtocolStatus) & _
    " " & FormatField(objIISLog.UserAgent)
   
   ' Write the output log file record.
   objOutputFile.WriteLine strLogRecord
 
   ' Read the next record from the log file.
   objIISLog.ReadLogRecord
 
  Loop
 
  ' Close the input log file.
  objIISLog.CloseLogFiles 1
  objIISLog = Null
 
 End If
 
Next
 
' Inform the user that the operation has completed.
MsgBox "Finished!"
 
' Format a log file field.
Function FormatField(tmpField)
 On Error Resume Next
 FormatField = "-"
 If Len(tmpField) > 0 Then FormatField = Trim(tmpField)
End Function
 
' Format a log file date.
Function BuildDateTime(tmpDateTime)
 On Error Resume Next
 tmpDateTime = CDate(tmpDateTime)
 BuildDateTime = Year(tmpDateTime) & "-" & _
  Right("0" & Month(tmpDateTime),2) & "-" & _
  Right("0" & Day(tmpDateTime),2) & " " & _
  Right("0" & Hour(tmpDateTime),2) & ":" & _
  Right("0" & Minute(tmpDateTime),2) & ":" & _
  Right("0" & Second(tmpDateTime),2)
End Function
 
' Create a random name.
Function CreateRandomName(intNameLength)
 Const strValidChars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
 Dim tmpX, tmpY, tmpZ
 For tmpX = 1 To intNameLength
  tmpY = Mid(strValidChars,Int(Rnd(1)*Len(strValidChars))+1,1)
  tmpZ = tmpZ & tmpY
 Next
 CreateRandomName = tmpZ
End Function

Happy coding!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

IIS 6: Creating an Itemized List of Server Bindings

21 February 2006 • by Bob • IIS, Scripting

One of my servers has a large number of individual web sites on it, and each of these web sites has several server bindings for different IP addresses, Port Assignments, and Host Headers. As I continue to add more web sites on the server, it becomes increasingly difficult to keep track of all the details using the IIS user interface.

With that in mind, I wrote the following ADSI script which creates a text file that contains an itemized list of all server bindings on a server.

Option Explicit
On Error Resume Next

Dim objBaseNode, objChildNode
Dim objBindings, intBindings
Dim objFSO, objFile, strOutput

' get a base object
Set objBaseNode = GetObject("IIS://LOCALHOST/W3SVC")
Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.CreateTextFile("ServerBindings.txt")

' check if if have an error ...
If (Err.Number <> 0) Then

    ' ... and output the error.
    strOutput = "Error " & Hex(Err.Number) & "("
    strOutput = strOutput & Err.Description & ") occurred."

' ... otherwise, continue processing.
Else
    ' loop through the child nodes
    For Each objChildNode In objBaseNode
        ' is this node for a web site?
        If objChildNode.class = "IIsWebServer" Then
            ' get the name of the node
            strOutput = strOutput & "LM/W3SVC/" & _
                objChildNode.Name
            ' get the server comment
            strOutput = strOutput & " (" & _
                objChildNode.ServerComment & ")" & vbCrLf
            ' get the bindings
            objBindings = objChildNode.ServerBindings
            ' loop through the bindings
            For intBindings = 0 To UBound(objBindings)
                strOutput = strOutput & vbTab & _
                    Chr(34) & objBindings(intBindings) & _
                    Chr(34) & vbCrLf
            Next
        End If
    ' try not to be a CPU hog
    Wscript.Sleep 10
    Next
End If

objFile.Write strOutput
objFile.Close

Set objBaseNode = Nothing
Set objFSO = Nothing

Hope this helps!

Updating several IP addresses using ADSI

11 January 2006 • by Bob • IIS, Scripting

(Note: I had originally posted this information on a blog that I kept on http://weblogs.asp.net, but it makes more sense to post it here. [:)] )

Like many web programmers, I host several hobby web sites for fun. (They make a wonderful test bed for new code. ;-] )

And like many computer enthusiasts, I sometimes change my ISP for one reason or another. If you are hosting web sites in a similar situation, I'm sure that you can identify the pain of trying to manually update each old IP address to your new IP address. This situation can be made even more difficult when any number of your web sites are using several host headers because the user interface for the IIS administration tool only lists the first host header. This means that you have to manually view the properties for every site just to locate the IP addresses that you are required to change.

Well, I'm a big believer in replacing any repetitive task with code when it is possible, and a recent change of ISP provided just the right level of inspiration for me to write a simple Active Directory Service Interfaces (ADSI) script that locates IP addresses that have to be changed and updates them to their new values.

To use the example script, I would first suggest that you make a backup copy of your metabase. (The script works fine, but it is always better to have a backup. ;-] ) As soon as your metabase has been backed up, copy the example script into notepad or some other text editor, update the old and new IP addresses that are defined as constants, and then run the script.

Option Explicit
On Error Resume Next
 
Dim objIIS
Dim objSite
Dim varBindings
Dim intBindings
Dim blnChanged
 
Const strOldIP = "10.0.0.1"
Const strNewIP = "192.168.0.1"
 
Set objIIS = GetObject("IIS://LOCALHOST/W3SVC")
 
If (Err <> 0) Then
  WScript.Echo "Error " & Hex(Err.Number) & "(" & _
    Err.Description & ") occurred."
  WScript.Quit
Else
  For Each objSite In objIIS
    blnChanged = False
    If objSite.class = "IIsWebServer" Then
      varBindings = objSite.ServerBindings
      For intBindings = 0 To UBound(varBindings)
        If InStr(varBindings(intBindings),strOldIP) Then
          blnChanged = True
          varBindings(intBindings) = Replace(varBindings(intBindings),strOldIP,strNewIP)
        End If
      Next
    End If
    If blnChanged = True Then
      objSite.ServerBindings = varBindings      
      objSite.Setinfo
    End If
  Next
End If
MsgBox "Finished!"

That's all for now. Happy coding!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Converting NCSA log files to W3C format

01 December 2005 • by Bob • IIS, Scripting

One of the great utilities that ships with IIS is the CONVLOG.EXE application, which converts W3C or MS Internet Standard log files to NCSA format, where they can be processed by any of the applications that only parse NCSA log file information. The trouble is, what happens when you already have NCSA log files and you want W3C log files? You can't use the CONVLOG.EXE application, it only works in the opposite direction.

With that in mind, I wrote the following Windows Script Host (WSH) script that will read the current directory and convert all NCSA-formatted log files to W3C format. To use this code, just copy the code into notepad, and save it with a ".vbs" file extension on your system. To run it, copy the script to a folder that contains NCSA log files, (named "nc*.log"), then double-click it.

Option Explicit

Dim objIISLog
Dim objFSO
Dim objFolder
Dim objFile
Dim objOutputFile
Dim strInputPath
Dim strOutputPath
Dim strLogRecord

Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")

Set objFolder = objFSO.GetFolder(".")

For Each objFile In objFolder.Files

 strInputPath = LCase(objFile.Name)

 If Left(strInputPath,2) = "nc" And Right(strInputPath,4) = ".log" Then

  strOutputPath = objFolder.Path & "\" & "ex" & Mid(strInputPath,3)
  strInputPath = objFolder.Path & "\" & strInputPath

  Set objIISLog = CreateObject("MSWC.IISLog")
  objIISLog.OpenLogFile strInputPath, 1, "", 0, ""
  Set objOutputFile = objFSO.CreateTextFile(strOutputPath)

  objIISLog.ReadLogRecord

  objOutputFile.WriteLine "#Software: Microsoft Internet Information Services 5.0"
  objOutputFile.WriteLine "#Version: 1.0"
  objOutputFile.WriteLine "#Date: " & BuildDateTime(objIISLog.DateTime)
  objOutputFile.WriteLine "#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status cs(User-Agent)"

  Do While Not objIISLog.AtEndOfLog

   strLogRecord = BuildDateTime(objIISLog.DateTime)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ClientIP)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.UserName)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ServerIP)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ServerPort)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.Method)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.URIStem)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.URIQuery)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ProtocolStatus)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.UserAgent)
   objOutputFile.WriteLine strLogRecord

   objIISLog.ReadLogRecord

  Loop

  objIISLog.CloseLogFiles 1
  objIISLog = Null
 
 End If

Next

Function FormatField(tmpField)
 On Error Resume Next
 FormatField = "-"
 If Len(tmpField) > 0 Then FormatField = Trim(tmpField)
End Function

Function BuildDateTime(tmpDateTime)
 On Error Resume Next
 tmpDateTime = CDate(tmpDateTime)
 BuildDateTime = Year(tmpDateTime) & "-" & _
  Right("0" & Month(tmpDateTime),2) & "-" & _
  Right("0" & Day(tmpDateTime),2) & " " & _
  Right("0" & Hour(tmpDateTime),2) & ":" & _
  Right("0" & Minute(tmpDateTime),2) & ":" & _
  Right("0" & Second(tmpDateTime),2)
End Function

I hope this helps!

Blog Navigation

You are on page 5 of 5 pages.

1 2 3 4 5