NXLog Legacy Documentation

Microsoft IIS

Microsoft Internet Information Server supports several logging formats. This chapter provides information about configuring IIS logging and NXLog collection. The recommended W3C format is documented below as well as other supported IIS formats.

This chapter also includes sections about collecting logs from the SMTP server and about Automatic retrieval of IIS site log locations.

Configuring logging

IIS logging can be configured at the site level or server level as follows. For more detailed information, see Configure Logging in IIS on Microsoft Docs.

  1. Open IIS Manager, which can be accessed from the Tools menu in the Server Manager or from Administrative Tools.

  2. In the Connections pane on the left, select the server or site for which to configure logging. Select a server to configure logging server-wide, or a site to configure logging for that specific site.

  3. Double-click the Logging icon in the center pane.

    Logging icon selected
  4. Modify the logging configuration as required. The W3C format is recommended.

    Logging configuration options

The resulting logs can be collected by NXLog as shown in the following sections.

W3C extended log file format

IIS can write logs in the W3C format, and the logged fields can be configured via the Select Fields…​ button (see the Configuring logging section). W3C is the recommended format for use with NXLog.

Log sample
#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2017-10-02 17:11:27
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2017-10-02 17:11:27 fe80::b5d8:132c:cec9:daef%6 RPC_IN_DATA /rpc/rpcproxy.dll 1d4026cb-6730-43bf-91eb-df80f41c050f@test.com:6001&CorrelationID=<empty>;&RequestId=11d6a78a-7c34-4f43-9400-ad23b114aa62&cafeReqId=11d6a78a-7c34-4f43-9400-ad23b114aa62; 80 TEST\HealthMailbox418406e fe80::b5d8:132c:cec9:daef%6 MSRPC - 500 0 0 7990
2017-10-02 17:12:57 fe80::a425:345a:7143:3b15%2 POST /powershell clientApplication=ActiveMonitor;PSVersion=5.1.14393.1715 80 - fe80::a425:345a:7143:3b15%2 Microsoft+WinRM+Client - 500 0 0 11279

Note that field names with special characters must be referenced with curly braces (for example, ${s-ip} and ${cs(User-Agent)}).

Example 1. Collecting W3C format logs with xm_w3c

This configuration reads from file with im_file and parses with xm_w3c.

nxlog.conf
<Extension w3c_parser>
    Module      xm_w3c
</Extension>

<Input iis_w3c>
    Module      im_file
    File        'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
    InputType   w3c_parser
</Input>

For NXLog Community Edition, the xm_csv module can be used instead for parsing the records.

Example 2. Collecting W3C format logs with xm_csv

This configuration parses the logs with the xm_csv module. The header lines are discarded and the $date and $time fields are parsed in order to set an $EventTime field.

The field list must be set according to the configured IIS fields. The fields shown here correspond with the default field selection in IIS versions 8.5 and 10.
nxlog.conf
<Extension w3c_parser>
    Module          xm_csv
    Fields          date, time, s-ip, cs-method, cs-uri-stem, cs-uri-query, \
                    s-port, cs-username, c-ip, cs(User-Agent), cs(Referer), \
                    sc-status, sc-substatus, sc-win32-status, time-taken
    FieldTypes      string, string, string, string, string, string, integer, \
                    string, string, string, string, integer, integer, integer, \
                    integer
    Delimiter       ' '
    EscapeChar      '"'
    QuoteChar       '"'
    EscapeControl   FALSE
    UndefValue      -
</Extension>

<Input iis_w3c>
    Module          im_file
    File            'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
    <Exec>
        if $raw_event =~ /^#/ drop();
        else
        {
            w3c_parser->parse_csv();
            $EventTime = parsedate($date + "T" + $time + ".000Z");
        }
    </Exec>
</Input>

Configuring IIS HTTP API error logs

IIS can be configured to write HTTP Server API Error logs. There are three registry values that control HTTP API error logging. These keys are located at the following registry key:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\HTTP\Parameters

For detailed information about this registry key’s specific values, please see Error logging in HTTP APIs on Microsoft Support.

Log sample
#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2018-10-01 22:10:02
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri sc-status s-siteid s-reason s-queuename
2018-10-01 22:10:02 ::1%0 49211 ::1%0 47001 - - - - - Timer_ConnectionIdle -
2018-10-01 22:10:02 ::1%0 49212 ::1%0 47001 - - - - - Timer_ConnectionIdle -
2018-10-01 23:45:09 172.31.77.6 2094 172.31.77.6 80 HTTP/1.1 GET /qos/1kbfile.txt 503 – ConnLimit
Example 3. Collecting IIS HTTP API logs with xm_csv

This configuration parses the logs with the xm_w3c module. The header lines are discarded and the $date and $time fields are parsed in order to set an $EventTime field.

nxlog.conf
<Extension w3c_parser>
    Module      xm_w3c
</Extension>

<Input iis_http>
    Module      im_file
    File        'C:\Windows\System32\LogFiles\HTTPERR\httperr1.log'
    InputType   w3c_parser
</Input>
The xm_w3c module is not included in NXLog Community Edition, so the xm_csv module should be used.

IIS log file format

The IIS format is line-based, with comma-separated fields and no header. See IIS Log File Format (IIS 6.0) on TechNet for more information.

Log sample
::1, HealthMailbox418406e8ac5b4b61a6b731ac4c660553@test.com, 9/28/2017, 14:49:00, W3SVC1, WINEXC, ::1, 7452, 592, 2538, 302, 0, POST, /OWA/auth.owa, &CorrelationID=<empty>;&cafeReqId=728beb5e-98de-4680-acb2-45968bef533c;&encoding=;,
127.0.0.1, -, 9/28/2017, 14:49:01, W3SVC1, WINEXC, 127.0.0.1, 6798, 2502, 682, 302, 0, GET, /ecp/, &CorrelationID=<empty>;&cafeReqId=0ed28871-4083-492f-99c2-2fbdb06a9466;&LogoffReason=NoCookiesGetOrE14AuthPost,
Example 4. Collecting logs from the IIS format

This configuration reads from file with im_file and parses the fields with xm_csv. The $Date and $Time fields are parsed in order to set an $EventTime field.

nxlog.conf
<Extension iis_parser>
    Module      xm_csv
    Fields      ClientIPAddress, UserName, Date, Time, ServiceAndInstance, \
                ServerName, ServerIPAddress, TimeTaken, ClientBytesSent, \
                ServerBytesSent, ServerStatusCode, WindowsStatusCode, RequestType, \
                TargetOfOperation, Parameters
    FieldTypes  string, string, string, string, string, string, string, integer, \
                integer, integer, integer, integer, string, string, string
    UndefValue  -
</Extension>

<Input iis>
    Module      im_file
    File        'C:\inetpub\logs\LogFiles\W3SVC*\u_in*.log'
    <Exec>
        iis_parser->parse_csv();
        $EventTime = strptime($Date + " " + $Time, "%m/%d/%Y %H:%M:%S");
    </Exec>
</Input>

NCSA common log file format

The NCSA log format is a line-based plain text format that separates fields with spaces and uses hyphens (-) as placeholders for empty fields. See the Common & Combined Log Formats section for more information about this format. See NCSA Common Log File Format (IIS 6.0) on Microsoft TechNet for more information about this format as used by IIS.

Log sample
fe80::a425:345a:7143:3b15%2 - - [02/Oct/2017:13:16:18 -0700] "POST /mapi/emsmdb/?useMailboxOfAuthenticatedUser=true HTTP/1.1" 401 7226
fe80::a425:345a:7143:3b15%2 - TEST\HealthMailboxc0bafd1 [02/Oct/2017:13:16:20 -0700] "POST /mapi/emsmdb/?useMailboxOfAuthenticatedUser=true HTTP/1.1" 200 1482
Example 5. Collecting NCSA format logs

This configuration reads from file with the im_file module and uses a regular expression to parse each record.

nxlog.conf
<Input iis_ncsa>
    Module  im_file
    File    'C:\inetpub\logs\LogFiles\W3SVC*\u_nc*.log'
    <Exec>
        if $raw_event =~ /(?x)^(\S+)\ -\ (\S+)\ \[([^\]]+)\]\ \"(\S+)\ (.+)
                          \ HTTP\/\d\.\d\"\ (\S+)\ (\S+)/
        {
            $RemoteHostAddress = $1;
            if $2 != '-' $UserName = $2;
            $EventTime = parsedate($3);
            $HTTPMethod = $4;
            $HTTPURL = $5;
            $HTTPResponseStatus = $6;
            $BytesSent = $7;
        }
    </Exec>
</Input>

SMTP server

IIS 6.0 in Windows Server 2008 R2 includes an SMTP server. This SMTP server has been deprecated beginning with Windows Server 2012, but it is still available in Windows Server 2016.

During operation, the IIS SMTP Server pads the W3C log to 64 KiB with NUL characters. When the SMTP Server stops, it truncates the file to remove the padding, causing im_file to re-read the log file and generate duplicate events.

IIS SMTP Server logging can be configured as follows.

  1. Open Internet Information Services (IIS) 6.0 Manager from Administrative Tools.

  2. Right click on the corresponding SMTP Virtual Server and click Properties.

    Opening SMTP Server Properties
  3. Check Enable logging and choose the logging format from the Active log format drop-down menu. The W3C format is recommended.

    Enabling SMTP Server logging
  4. Click the Properties…​ button to configure the log location and other options.

    Editing SMTP Logging Properties
  5. If using the W3C format, adjust the logged fields under the Advanced tab. Include the Date and Time fields and whatever extended properties are required.

    Modifying W3C fields
Example 6. Collecting W3C logs from the IIS SMTP server

The following configuration retrieves W3C logs and parses them using the xm_w3c module.

nxlog.conf
<Extension w3c_parser>
    Module      xm_w3c
</Extension>

<Input smtp>
    Module      im_file
    File        'C:\Windows\System32\LogFiles\SmtpSvc1\ex*.log'
    InputType   w3c_parser
</Input>

See the preceding sections for more information about processing the other log formats or using xm_csv for processing W3C logs with NXLog Community Edition.

Automatic retrieval of IIS site log locations

The IIS per-site log file locations can be automatically fetched with a batch/PowerShell polyglot script via the include_stdout directive. For more details, see the PowerShell Generating configuration with PowerShell section.

Example 7. Retrieving log locations via script

The following polyglot script should be installed in the NXLog installation (or ROOT) directory. It uses the WebAdministration PowerShell module to return the configured log path for each site. If IIS is configured to use one log file per server, the path should instead be configured manually.

If there are multiple log formats in the log directory due to configuration changes, the wildcard path should be adjusted to match only those files that are in the corresponding format. For example, for W3C logging use u_ex*.log in the last line of the script.
get_iis_log_paths.cmd
@( Set "_= (
Rem " ) <#
)
@Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
if defined PROCESSOR_ARCHITEW6432 (
set powershell=%SystemRoot%\SysNative\WindowsPowerShell\v1.0\powershell.exe
) else (
set powershell=powershell.exe
)
%powershell% -ExecutionPolicy Bypass -NoProfile ^
-Command "iex ((gc '%~f0') -join [char]10)"
EndLocal & Exit /B %ErrorLevel%
#>
Import-Module -Name WebAdministration
foreach($Site in $(get-website)) {
$LogDir=$($Site.logFile.directory.replace("%SystemDrive%",$env:SystemDrive))

# WARNING: adjust path to match format (for example, for W3C use `u_ex*.log`).
Write-Output "File '$LogDir\W3SVC$($Site.id)\*.log'" }
nxlog.conf
<Extension w3c_parser>
    Module          xm_w3c
</Extension>

<Input iis>
    Module          im_file
    include_stdout  %ROOT%\get_iis_log_paths.cmd
    InputType       w3c_parser
</Input>
Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

Last revision: 14 February 2019