Microsoft IIS
Microsoft Internet Information Server supports several logging formats. This chapter provides information about configuring IIS logging and NXLog collection. The recommended W3C format is documented below as well as other supported IIS formats.
This chapter also includes sections about collecting logs from the SMTP server and about Automatic retrieval of IIS site log locations.
Configuring logging
IIS logging can be configured at the site level or server level as follows. For more detailed information, see Configure Logging in IIS on Microsoft Docs.
-
Open IIS Manager, which can be accessed from the Tools menu in the Server Manager or from Administrative Tools.
-
In the Connections pane on the left, select the server or site for which to configure logging. Select a server to configure logging server-wide, or a site to configure logging for that specific site.
-
Double-click the Logging icon in the center pane.
-
Modify the logging configuration as required. The W3C format is recommended.
The resulting logs can be collected by NXLog as shown in the following sections.
W3C extended log file format
IIS can write logs in the W3C format, and the logged fields can be configured via the Select Fields… button (see the Configuring logging section). W3C is the recommended format for use with NXLog.
#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2017-10-02 17:11:27
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2017-10-02 17:11:27 fe80::b5d8:132c:cec9:daef%6 RPC_IN_DATA /rpc/rpcproxy.dll 1d4026cb-6730-43bf-91eb-df80f41c050f@test.com:6001&CorrelationID=<empty>;&RequestId=11d6a78a-7c34-4f43-9400-ad23b114aa62&cafeReqId=11d6a78a-7c34-4f43-9400-ad23b114aa62; 80 TEST\HealthMailbox418406e fe80::b5d8:132c:cec9:daef%6 MSRPC - 500 0 0 7990
2017-10-02 17:12:57 fe80::a425:345a:7143:3b15%2 POST /powershell clientApplication=ActiveMonitor;PSVersion=5.1.14393.1715 80 - fe80::a425:345a:7143:3b15%2 Microsoft+WinRM+Client - 500 0 0 11279Note that field names with special characters must be referenced with curly
braces (for example, ${s-ip} and ${cs(User-Agent)}).
See also the W3C Extended Log File Format section and the W3C Extended Log File Format (IIS 6.0) and W3C Extended Log File Examples (IIS 6.0) articles on Microsoft TechNet.
For NXLog Community Edition, the xm_csv module can be used instead for parsing the records.
This configuration parses the logs with the xm_csv module. The header
lines are discarded and the $date and $time fields are parsed in order to
set an $EventTime field.
| The field list must be set according to the configured IIS fields. The fields shown here correspond with the default field selection in IIS versions 8.5 and 10. |
<Extension w3c_parser>
Module xm_csv
Fields date, time, s-ip, cs-method, cs-uri-stem, cs-uri-query, \
s-port, cs-username, c-ip, cs(User-Agent), cs(Referer), \
sc-status, sc-substatus, sc-win32-status, time-taken
FieldTypes string, string, string, string, string, string, integer, \
string, string, string, string, integer, integer, integer, \
integer
Delimiter ' '
EscapeChar '"'
QuoteChar '"'
EscapeControl FALSE
UndefValue -
</Extension>
<Input iis_w3c>
Module im_file
File 'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
<Exec>
if $raw_event =~ /^#/ drop();
else
{
w3c_parser->parse_csv();
$EventTime = parsedate($date + "T" + $time + ".000Z");
}
</Exec>
</Input>Configuring IIS HTTP API error logs
IIS can be configured to write HTTP Server API Error logs. There are three registry values that control HTTP API error logging. These keys are located at the following registry key:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\HTTP\Parameters
For detailed information about this registry key’s specific values, please see Error logging in HTTP APIs on Microsoft Support.
#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2018-10-01 22:10:02
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri sc-status s-siteid s-reason s-queuename
2018-10-01 22:10:02 ::1%0 49211 ::1%0 47001 - - - - - Timer_ConnectionIdle -
2018-10-01 22:10:02 ::1%0 49212 ::1%0 47001 - - - - - Timer_ConnectionIdle -
2018-10-01 23:45:09 172.31.77.6 2094 172.31.77.6 80 HTTP/1.1 GET /qos/1kbfile.txt 503 – ConnLimitThis configuration parses the logs with the xm_w3c module. The header
lines are discarded and the $date and $time fields are parsed in order to
set an $EventTime field.
<Extension w3c_parser>
Module xm_w3c
</Extension>
<Input iis_http>
Module im_file
File 'C:\Windows\System32\LogFiles\HTTPERR\httperr1.log'
InputType w3c_parser
</Input>IIS log file format
The IIS format is line-based, with comma-separated fields and no header. See IIS Log File Format (IIS 6.0) on TechNet for more information.
::1, HealthMailbox418406e8ac5b4b61a6b731ac4c660553@test.com, 9/28/2017, 14:49:00, W3SVC1, WINEXC, ::1, 7452, 592, 2538, 302, 0, POST, /OWA/auth.owa, &CorrelationID=<empty>;&cafeReqId=728beb5e-98de-4680-acb2-45968bef533c;&encoding=;, 127.0.0.1, -, 9/28/2017, 14:49:01, W3SVC1, WINEXC, 127.0.0.1, 6798, 2502, 682, 302, 0, GET, /ecp/, &CorrelationID=<empty>;&cafeReqId=0ed28871-4083-492f-99c2-2fbdb06a9466;&LogoffReason=NoCookiesGetOrE14AuthPost,
This configuration reads from file with im_file and parses the
fields with xm_csv. The $Date and $Time fields are parsed in
order to set an $EventTime field.
<Extension iis_parser>
Module xm_csv
Fields ClientIPAddress, UserName, Date, Time, ServiceAndInstance, \
ServerName, ServerIPAddress, TimeTaken, ClientBytesSent, \
ServerBytesSent, ServerStatusCode, WindowsStatusCode, RequestType, \
TargetOfOperation, Parameters
FieldTypes string, string, string, string, string, string, string, integer, \
integer, integer, integer, integer, string, string, string
UndefValue -
</Extension>
<Input iis>
Module im_file
File 'C:\inetpub\logs\LogFiles\W3SVC*\u_in*.log'
<Exec>
iis_parser->parse_csv();
$EventTime = strptime($Date + " " + $Time, "%m/%d/%Y %H:%M:%S");
</Exec>
</Input>NCSA common log file format
The NCSA log format is a line-based plain text format that separates fields with
spaces and uses hyphens (-) as placeholders for empty fields. See the
Common & Combined Log Formats section for more information about this
format. See NCSA Common Log
File Format (IIS 6.0) on Microsoft TechNet for more information about this
format as used by IIS.
fe80::a425:345a:7143:3b15%2 - - [02/Oct/2017:13:16:18 -0700] "POST /mapi/emsmdb/?useMailboxOfAuthenticatedUser=true HTTP/1.1" 401 7226 fe80::a425:345a:7143:3b15%2 - TEST\HealthMailboxc0bafd1 [02/Oct/2017:13:16:20 -0700] "POST /mapi/emsmdb/?useMailboxOfAuthenticatedUser=true HTTP/1.1" 200 1482
This configuration reads from file with the im_file module and uses a regular expression to parse each record.
<Input iis_ncsa>
Module im_file
File 'C:\inetpub\logs\LogFiles\W3SVC*\u_nc*.log'
<Exec>
if $raw_event =~ /(?x)^(\S+)\ -\ (\S+)\ \[([^\]]+)\]\ \"(\S+)\ (.+)
\ HTTP\/\d\.\d\"\ (\S+)\ (\S+)/
{
$RemoteHostAddress = $1;
if $2 != '-' $UserName = $2;
$EventTime = parsedate($3);
$HTTPMethod = $4;
$HTTPURL = $5;
$HTTPResponseStatus = $6;
$BytesSent = $7;
}
</Exec>
</Input>SMTP server
IIS 6.0 in Windows Server 2008 R2 includes an SMTP server. This SMTP server has been deprecated beginning with Windows Server 2012, but it is still available in Windows Server 2016.
| During operation, the IIS SMTP Server pads the W3C log to 64 KiB with NUL characters. When the SMTP Server stops, it truncates the file to remove the padding, causing im_file to re-read the log file and generate duplicate events. |
IIS SMTP Server logging can be configured as follows.
-
Open Internet Information Services (IIS) 6.0 Manager from Administrative Tools.
-
Right click on the corresponding SMTP Virtual Server and click Properties.
-
Check Enable logging and choose the logging format from the Active log format drop-down menu. The W3C format is recommended.
-
Click the Properties… button to configure the log location and other options.
-
If using the W3C format, adjust the logged fields under the Advanced tab. Include the Date and Time fields and whatever extended properties are required.
The following configuration retrieves W3C logs and parses them using the xm_w3c module.
<Extension w3c_parser>
Module xm_w3c
</Extension>
<Input smtp>
Module im_file
File 'C:\Windows\System32\LogFiles\SmtpSvc1\ex*.log'
InputType w3c_parser
</Input>See the preceding sections for more information about processing the other log formats or using xm_csv for processing W3C logs with NXLog Community Edition.
Automatic retrieval of IIS site log locations
The IIS per-site log file locations can be automatically fetched with a batch/PowerShell polyglot script via the include_stdout directive. For more details, see the PowerShell Generating configuration with PowerShell section.
The following polyglot script should be installed in the NXLog
installation (or ROOT) directory. It uses the WebAdministration PowerShell
module to return the configured log path for each site. If IIS is configured
to use one log file per server, the path should instead be configured
manually.
If there are multiple log formats in the log directory due to
configuration changes, the wildcard path should be adjusted to match
only those files that are in the corresponding format. For example, for
W3C logging use u_ex*.log in the last line of the script.
|
@( Set "_= (
Rem " ) <#
)
@Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
if defined PROCESSOR_ARCHITEW6432 (
set powershell=%SystemRoot%\SysNative\WindowsPowerShell\v1.0\powershell.exe
) else (
set powershell=powershell.exe
)
%powershell% -ExecutionPolicy Bypass -NoProfile ^
-Command "iex ((gc '%~f0') -join [char]10)"
EndLocal & Exit /B %ErrorLevel%
#>
Import-Module -Name WebAdministration
foreach($Site in $(get-website)) {
$LogDir=$($Site.logFile.directory.replace("%SystemDrive%",$env:SystemDrive))
# WARNING: adjust path to match format (for example, for W3C use `u_ex*.log`).
Write-Output "File '$LogDir\W3SVC$($Site.id)\*.log'" }<Extension w3c_parser>
Module xm_w3c
</Extension>
<Input iis>
Module im_file
include_stdout %ROOT%\get_iis_log_paths.cmd
InputType w3c_parser
</Input>
