Support

NXLog Documentation

Windows Event Log

Windows Event Log captures system, security, and application events on Windows operating systems. It serves as a repository of detailed events generated by the system and is the first resource IT administrators refer to when troubleshooting issues. Besides resolving problems, Windows events are also used to monitor, analyze, and satisfy compliance mandates.

About Windows Event Log

Unlike other event logs, such as the UNIX syslog, Windows Event Log is not stored as a plain text file but in a proprietary binary format. Therefore, it is impossible to view the logs in a text editor or send them as syslog messages while retaining their original format. However, the raw event data can be translated into XML using the Windows Event Log API.

The EVTX file format

Windows has stored Windows Event Log files in the EVTX file format since the release of Windows Vista and Windows Server 2008. Before that, event log files were stored in the EVT file format. Both are proprietary formats readable by the Microsoft Management Console (MMC) snap-in eventvwr.msc.

The EVTX format has many enhancements over its predecessor, including new event properties, the use of channels to publish events, a new Event Viewer, a rewritten Windows Event Log service, and support for the Extensible Markup Language (XML) format. From a log collection perspective, the added support for XML is the most important feature since it allows sharing or processing of event data in a structured format.

Windows saves EVTX files in the C:\Windows\System32\winevt\Logs\ directory for built-in channels. You can also export events manually from Event Viewer in four formats: EVTX, XML, TXT, and CSV.

NXLog can read EVTX and EVT files using the File directive of the im_msvistalog module. In addition, the CaptureEventXML directive of the same module allows you to store raw XML-formatted event data.

Viewing the Windows Event Log

The Windows Event Log can be viewed in the Event Viewer MMC snap-in included in Windows. Since the stored logs do not include the entire message—​only the event properties are stored—​Event Viewer inserts property values into a corresponding localized template to display the full event details.

Windows Event Viewer includes three views for displaying event data. These are shown on the preview pane or in the Event Properties window when you double-click on an event.

  • The General view is displayed by default. It includes the entire message rendered from a template and a set of standard event properties.

  • You can switch to the Friendly View by clicking on the Details tab. This view shows a hierarchical list of standard and event-specific properties. It does not show the descriptive message from the event template.

  • You can also switch to the XML View from the Details tab to see all the event properties in XML format.

    A Windows event in XML format
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing"
              Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
    <EventID>4624</EventID>
    [...]
    <Channel>Security</Channel>
    <Computer>USER-WORKSTATION</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="SubjectUserSid">S-1-5-18</Data>
    [...]
  </EventData>
</Event>

Consumers access events through the Event Log API. Refer to Windows Event Log Functions on Microsoft Learn for more information. In particular:

  • EvtQuery() to fetch events from a channel or log file that matches a given query. See Querying for Events.

  • EvtFormatMessage() generates the event message using the event properties and the localized template. See Formatting Event Messages.

Event channels

The EVTX format introduces event channels. A channel is a stream of events collected from a publisher and written to a log file.

Channels are organized into two groups:

  • Windows Logs contains five channels for Security, Setup, Forwarded Events, System, and Application event logs.

  • Applications and Services Logs contains channels created for individual applications and components.

Two types of channels for different purposes are available:

  • Serviced channels are relatively low volume and offer reliable event delivery. Event collectors can subscribe to these channels, and you can forward events from them to another system.

  • Direct channels are for high-performance event collection and are disabled by default. It is not possible to subscribe to a direct channel. To see these channels in Windows Event Viewer, enable Show Analytic and Debug Logs in the View menu. To enable logging for a direct channel, right-click on it and select Properties, then check the Enable logging option on the General tab.

The above channel types are divided further according to the intended audience:

  • Administrative channels are serviced channels that collect events intended for end users, administrators, and support.

  • Operational channels are serviced channels collecting events intended for troubleshooting.

  • Analytic channels are direct channels and contain events that describe program operations. These channels often collect a high volume of events.

  • Debug channels are direct channels intended to be used by developers only.

Table 1. Channel groups and types
Channel Groups Channels Channel Type

Windows Logs

Application

Administrative (serviced)

Security

Administrative (serviced)

Setup

Operational (serviced)

System

Administrative (serviced)

Forwarded Events

Operational (serviced)

Applications and Services Logs

DHCP-Server/Admin

Administrative (serviced)

DHCP-Server/AuditLogs

Analytic (direct)

DHCP-Server/DebugLogs

Debug (direct)

(And many more publisher-defined channels)

The im_msvistalog module can collect events from a specific channel with the Channel directive.

For more information about event channels, see Event Logs and Event Logs and Channels in Windows Event Log on Microsoft Learn.

Providers

Providers write events to the event logs. An event provider can be a service, driver, or program that runs on the computer and has the necessary instrumentation to write the events.

Event providers are categorized into four main types:

  • Manage Object Format (MOF) providers (also referred to as "classic")

  • Windows Software Trace Preprocessor (WPP) providers

  • Manifest-based providers

  • TraceLogging providers

For more information, see Providers on Microsoft Learn.

Collecting Windows events

NXLog provides the following modules for capturing Windows events.

Local Windows log collection with im_msvistalog

The im_msvistalog module can capture events locally from Windows 2008/Vista and later.

Because the Windows Event Log subsystem does not support subscribing to the Debug and Analytic channels, the im_msvistalog module cannot collect events from these channels.
Example 1. Collecting Windows logs locally from Windows 2008/Vista or later

This configuration reads all events from the local Windows Event Log. It then converts event records to JSON format using the to_json() procedure of the xm_json module and writes them to a file.

nxlog.conf
<Extension json>
    Module    xm_json
</Extension>

<Input eventlog>
    Module    im_msvistalog
</Input>

<Output file>
    Module    om_file
    File      'C:\logs\windows_events.log'
    Exec      to_json();
</Output>

For information about filtering Windows events, see Filtering events.

Remote Windows log collection with im_msvistalog

NXLog Enterprise Edition can collect events from remote Windows systems with the im_msvistalog module. In this mode, you do not need to install the NXLog agent on each Windows system; instead, a central agent collects the events from them via MSRPC.

Because the Windows Event Log subsystem does not support subscribing to the Debug and Analytic channels, the im_msvistalog module cannot collect events from these channels.
Example 2. Collecting Windows logs remotely via MSRPC

This configuration uses the im_msvistalog module to collect Windows events from a remote server named MYSERVER.

To adapt this example to your environment, specify the relevant RemoteServer, RemoteUser, RemoteDomain, and RemotePassword for your target machine.

nxlog.conf
<Input eventlog>
    Module            im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='1'>
                <Select Path='Application'>*</Select>
                <Select Path='Security'>*[System/Level=4]</Select>
                <Select Path='System'>*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    RemoteServer      MYSERVER
    RemoteUser        Administrator
    RemoteDomain      mydomain.local
    RemotePassword    secret
</Input>

Local Windows log collection with im_mseventlog

The im_mseventlog module captures event logs from Windows XP, Windows 2000, and Windows 2003.

The module retrieves the available log sources from the SYSTEM\CurrentControlSet\Services\Eventlog registry key and polls logs from all the sources. You can also specify the sources from which you want to collect logs with the Sources directive.

Example 3. Collecting Windows logs locally from Windows XP/2000/2003

This example shows the most basic im_mseventlog module configuration. It collects Windows logs from all registered sources and forwards them to a remote server via TCP.

nxlog.conf
<Input eventlog>
    Module    im_mseventlog
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.1:514
</Output>

Remote Windows log collection with im_wseventing

NXLog Enterprise Edition provides the im_wseventing module for receiving Windows event logs from remote machines via Windows Event Forwarding (WEF). You can use this module on both Windows and Linux hosts.

Example 4. Receiving Windows logs via WEF

This configuration listens for connections on port 5985 from all WEF clients. It expects to receive events over HTTPS.

nxlog.conf
<Input wseventing>
    Module              im_wseventing
    ListenAddr          0.0.0.0
    Port                5985
    Address             https://linux.example.com:5985/wsman
    HTTPSCertFile       %CERTDIR%/server-cert.pem
    HTTPSCertKeyFile    %CERTDIR%/server-key.pem
    HTTPSCAFile         %CERTDIR%/ca.pem
    <QueryXML>
        <QueryList>
            <Query Id="0" Path="Application">
                <Select Path="Application">*</Select>
                <Select Path="Microsoft-Windows-Winsock-AFD/Operational">*</Select>
                <Select Path="Microsoft-Windows-Wired-AutoConfig/Operational">
                    *
                </Select>
                <Select Path="Microsoft-Windows-Wordpad/Admin">*</Select>
                <Select Path="Windows PowerShell">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

You can filter for specific hosts by adding the <Computer> tag to the QueryXML block. This tag expects a pattern that NXLog will match against the name of the connecting Windows client. If the computer name does not match the specified pattern, NXLog will not collect its events.

The following QueryXML block, if added to the above configuration, would provide an alternative query for computer names matching the pattern foo*. In contrast, other computers will use the QueryXML without the <Computer> tag.

nxlog.conf
<QueryXML>
    <Computer>foo*</Computer>
    <QueryList>
        <Query Id="0" Path="Application">
            <Select Path="Application">*</Select>
        </Query>
    </QueryList>
</QueryXML>

Filtering events

Applications and services on Windows can generate a large volume of logs, and it is often necessary to collect a subset of the events. There are several ways to filter events from the Windows Event Log using the im_msvistalog module.

  • You can collect all events from a specific channel with the Channel directive.

  • You can specify an XPath query with the Query or QueryXML directives. An XPath query allows you to subscribe to multiple channels and filter events by various attributes. However, XPath queries have a maximum length, limiting the possibilities for detailed event subscriptions. See XPath filtering below.

  • You can process a log file with the File directive, in which case im_msvistalog will read all events from the .evtx file. This method is intended primarily for forensics, for example, using the nxlog-processor to process historical data.

  • After reading events from the source, you can selectively discard events using the drop() procedure within an Exec block.

Subscribing to a restricted set of events with an XPath query provides a performance advantage because NXLog will not collect unnecessary events in the first place. However, XPath queries have a maximum length and limited filtering capabilities, so you may need to combine XPath filtering with post-collection processing. For examples, see Example monitoring configurations in Event IDs to Monitor.

XPath filtering

Windows Event Log supports a subset of XPath 1.0. You can use XPath queries to subscribe to events matching specific criteria, both in Windows Event Viewer and with the im_msvistalog QueryXML directive. For more information, see Consuming Events on Microsoft Learn.

Event Viewer offers the most practical way to write and test queries. For example, you can test an XPath query by filtering the current log or creating a custom view.

  1. In Event Viewer, click on an event channel, then right-click on it and choose Filter Current Log…​ from the context menu. A dialog box showing basic filtering options will be displayed.

    Basic Filtering

  2. Specify the desired criteria. Switch to the XML tab to see the corresponding XPath query. You can copy this query to the im_msvistalog QueryXML directive.

  3. You can enable advanced filtering by selecting the Edit query manually checkbox on the XML tab. We recommend verifying that the query matches the correct events before copying it to the NXLog configuration.

    XPath XML
    Figure 1. A Custom View Querying the Application Channel for Events With ID 1008

Sometimes it is helpful to use a query with sources that may not be available. In this case, set the TolerateQueryErrors directive to TRUE to ensure that the module will continue to collect logs.

Example 5. Collecting Sysmon operational events

This configuration only collects Sysmon operational events from the local Windows Event Log.

nxlog.conf
<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>
Example 6. Collecting important events from the Windows System log

This configuration queries the System event log for events with levels below 4 (Critical, Error, and Warning).

nxlog.conf
<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path='System'>*[System/Level&lt;4]</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

Post-collection log filtering

You can use NXLog’s built-in log processing capabilities to filter events. For example, you can match events against any of the im_msvistalog_ fields and use the drop() procedure to delete unwanted events.

Example 7. Filtering Sysmon events in an exec block

This example discards all Sysmon events with event ID 3 for HTTP network connections to a particular server and all process creation and termination events (event IDs 1 and 5) for conhost.exe.

nxlog.conf
<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
        if ($EventID in (1, 5) and
            $Image == "C:\\Windows\\System32\\conhost.exe") or
           ($EventID == 3 and
            $DestinationPort == 80 and
            $DestinationIp == 10.0.0.1) {
           drop();
        }
    </Exec>
</Input>

Trimming events

Further to filtering for only necessary events, trimming helps you to reduce the size of the events. For example, starting from Windows Server 2008, some events include a message describing the event’s purpose. While such messages might be helpful for manual troubleshooting, they are unnecessary for archiving and processing by SIEMs and log analytics platforms. Consider, for example, event ID 4624. Each event logged contains the following text in addition to the event data:

This event is generated when a logon session is created. It is generated on the computer that was accessed.

The subject fields indicate the account on the local system which requested the logon.
This is most commonly a server service or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for which the new logon was created, i.e., the account logged on.

The network fields indicate where a remote logon request originated.
The workstation name is not always available and may sometimes be left blank.

The impersonation level field indicates the extent to which a process in the logon session can impersonate.

The authentication information fields provide detailed information about this specific logon request.
	- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
	- Transited services indicate which intermediate services have participated in this logon request.
	- Package name indicates which sub-protocol was used among the NTLM protocols.
	- Key length indicates the length of the generated session key.
This would be 0 if no session key were requested.

When dealing with thousands or millions of events, processing and storing this data for every event unnecessarily increases the network load and storage requirements. Removing descriptive messages and other unnecessary information can reduce the amount of data by half, helping to drive down costs related to network bandwidth and disk space, and making a substantial difference for SIEMs that charge by the amount of ingested data.

NXLog provides several options to trim event data depending on your use case and SIEM requirements.

Truncating the Message field

The im_msvistalog module populates the $Message field with the entire event message, including any descriptive text. If your SIEM accepts unstructured data, such as Syslog or Snare format, NXLog uses this field to format the output data. However, the event descriptions are usually not required by SIEMs and can be removed to reduce the event size significantly. For example, the following table shows data for a sample event with ID 4624 in Syslog format.

Table 2. Trimming unstructured event ID 4624
Event Size Decrease

Original event

2.17KB

-

Event with truncated Message field

1006B

57%

There are two methods you can use to truncate text fields with NXLog:

  • You can define rules and apply actions to matching events using the xm_pattern module. This is the preferred method since xm_pattern does not compare rules linearly, making it more efficient.

  • If you require linear comparison, you can use Regular Expressions.

Deleting standard fields

SIEMs capable of ingesting structured data are often pre-loaded with standard event information, such as the event type, category, and severity, mainly for security events. Therefore, fields like the $TaskValue, $Opcode, and $OpcodeValue can be removed before forwarding. In addition, for SIEMs that do not require the original event message, the $Message field can also be removed since it is redundant. The following table contains data for a sample event with ID 4624 in JSON format.

Table 3. Trimming structured event ID 4624
Event Size Fields Decrease

Original event

3.78KB

50

-

Event with truncated Message

2.50KB

50

34%

Event with truncated Message + deleted standard fields

2.18KB

38

42%

Deleted Message

1.45KB

49

62%

Deleted Message + standard fields

1.12KB

37

70%

There are two methods you can use to delete fields with NXLog:

  • You can define a field allowlist or blocklist using the xm_rewrite module. This is the preferred method when you need to delete several fields.

  • You can use the delete() procedure to delete individual fields.

Example trimming configurations

Example 8. Trimming unstructured events

This xm_pattern pattern file compares Windows security events based on the event ID. Each rule defines an Exec block so that the description is removed from the $Message field for each matching event. You can also define regex patterns for a more generic configuration; however, regex patterns are not as efficient as exact patterns and may delay log processing if used excessively. See the xm_pattern documentation for an example.

patterndb.xml
<?xml version='1.0' encoding='UTF-8'?>
<patterndb>
  <created>2022-07-23 11:18:18</created>
  <version>1</version>
  <group>
    <name>Security</name>
    <id>1</id>
    <matchfield>
      <name>Channel</name>
      <type>exact</type>
      <value>Security</value>
    </matchfield>

    <!-- Remove Event Summary -->
    <pattern>
      <id>1</id>
      <name>Security Event 4624</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4624</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>2</id>
      <name>Security Event 4625</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4625</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>3</id>
      <name>Security Event 4634</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4634</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>4</id>
      <name>Security Event 4648</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4648</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>5</id>
      <name>Security Event 4769</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4769</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>6</id>
      <name>Security Event 4616</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>4616</value>
      </matchfield>
      <exec>
        $Messge =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>7</id>
      <name>Security Event 4647</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>4647</value>
      </matchfield>
      <exec>
        $Messge =~ s/\s*This event is generated.*$//s;
      </exec>
    </pattern>

    <!-- Remove Certificate Summary -->
    <pattern>
      <id>8</id>
      <name>Security Event 4769</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4768</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*Certificate information is only provided.*$//s;
      </exec>
    </pattern>

    <pattern>
      <id>9</id>
      <name>Security Event 4769</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4771</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*Certificate information is only provided.*$//s;
      </exec>
    </pattern>

    <!-- Remove Token Elevation Summary -->
    <pattern>
      <id>10</id>
      <name>Security Event 4769</name>
      <matchfield>
        <name>EventID</name>
        <type>exact</type>
        <value>4688</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*Token Elevation Type indicates the type of.*$//s;
      </exec>
    </pattern>

    <!-- Remove Stored Credentials Summary -->
    <pattern>
      <id>11</id>
      <name>Security Event 5379</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>5379</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*This event occurs when a user performs a read operation on stored credentials in Credential Manager.*$//s;
      </exec>
    </pattern>

    <!-- Remove Authentication Summary -->
    <pattern>
      <id>12</id>
      <name>Security Event 4776</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>4776</value>
      </matchfield>
      <exec>
        $Message =~ s/^The computer attempted to validate the credentials for an account.\n\n//s;
      </exec>
    </pattern>

    <!-- Remove Network Share Summary -->
    <pattern>
      <id>13</id>
      <name>Security Event 5140</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>5140</value>
      </matchfield>
      <exec>
        $Message =~ s/^A network share object was accessed.\n\n//s;
      </exec>
    </pattern>

    <pattern>
      <id>14</id>
      <name>Security Event 5144</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>5144</value>
      </matchfield>
      <exec>
        $Message =~ s/^A network share object was deleted.\n\n//s;
      </exec>
    </pattern>

    <pattern>
      <id>15</id>
      <name>Security Event 5145</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>5145</value>
      </matchfield>
      <exec>
        $Message =~ s/^A network share object was checked to see whether client can be granted desired access.\n\n//s;
      </exec>
    </pattern>

    <!-- Remove User/Device Claims Summary -->
    <pattern>
      <id>16</id>
      <name>Security Event 4626</name>
      <matchfield>
        <name>EventID</name>
        <type>Exact</type>
        <value>4626</value>
      </matchfield>
      <exec>
        $Message =~ s/\s*The subject fields indicate.*$//s;
      </exec>
    </pattern>
  </group>
</patterndb>

The NXLog configuration below collects security events using the im_msvistalog input module. It processes each event according to the rules file above and converts the record to Syslog format.

nxlog.conf
<Extension syslog>
    Module         xm_syslog
</Extension>

<Extension pattern>
    Module         xm_pattern
    PatternFile    'C:\Program Files\nxlog\conf\patterndb.xml'
</Extension>

<Input SecurityEvents>
    Module         im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='0' Path="Security">
                <Select Path="Security">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
        match_pattern(); (1)
        to_syslog_bsd(); (2)
    </Exec>
</Input>
1 Processes the event against the rules defined in the pattern file of the xm_pattern module instance.
2 Converts the event to syslog using the to_syslog_bsd() procedure of the xm_syslog module.
Output sample
<14>Jul 26 10:12:02 WIN-F0RPR0BHTK0.domain.local Microsoft-Windows-Security-Auditing[0x0]: An account was successfully logged on.    Subject:  	Security ID:		S-1-0-0  	Account Name:		-  	Account Domain:		-  	Logon ID:		0x0    Logon Information:  	Logon Type:		3  	Restricted Admin Mode:	-  	Virtual Account:		No  	Elevated Token:		Yes    Impersonation Level:		Impersonation    New Logon:  	Security ID:		S-1-5-21-1241979678-2462307911-3370647597-1103  	Account Name:		WINAB-2JR3FR9RD$  	Account Domain:		DOMAIN.LOCAL  	Logon ID:		0x121355  	Linked Logon ID:		0x0  	Network Account Name:	-  	Network Account Domain:	-  	Logon GUID:		{72e0f6ab-7332-5b0f-7ecb-96939bff476e}    Process Information:  	Process ID:		0x0  	Process Name:		-    Network Information:  	Workstation Name:	-  	Source Network Address:	10.0.0.100  	Source Port:		49748    Detailed Authentication Information:  	Logon Process:		Kerberos  	Authentication Package:	Kerberos  	Transited Services:	-  	Package Name (NTLM only):	-  	Key Length:		0
Example 9. Trimming structured events

This configuration collects security events using the im_msvistalog input module. It processes each event to remove unnecessary fields and then converts the record to JSON format.

nxlog.conf
<Extension json>
    Module         xm_json
</Extension>

<Extension rewrite>
    Module         xm_rewrite
    Delete         SourceModuleName,SourceModuleType,Severity,SeverityValue,  \
                   Opcode,OpcodeValue,TaskValue,ProviderGuid,EventType,Level, \
                   LevelValue,Version,Message
</Extension>

<Input SecurityEvents>
    Module         im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='0' Path="Security">
                <Select Path="Security">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
        rewrite->process(); (1)
        to_json(); (2)
    </Exec>
</Input>
1 Processes the event according to the xm_rewrite module instance configuration.
2 Converts the event to JSON format using the to_json() procedure of the xm_json module.
Output sample
{
  "EventTime": "2022-07-26T10:12:02.577206+01:00",
  "Hostname": "WIN-F0RPR0BHTK0.domain.local",
  "Keywords": "9232379236109516800",
  "EventID": 4624,
  "SourceName": "Microsoft-Windows-Security-Auditing",
  "RecordNumber": 2593,
  "ActivityID": "{9F55F44E-A0CE-0003-C2F4-559FCEA0D801}",
  "ExecutionProcessID": 636,
  "ExecutionThreadID": 860,
  "Channel": "Security",
  "Category": "Logon",
  "SubjectUserSid": "S-1-0-0",
  "SubjectUserName": "-",
  "SubjectDomainName": "-",
  "SubjectLogonId": "0x0",
  "TargetUserSid": "S-1-5-21-1241979678-2462307911-3370647597-1103",
  "TargetUserName": "WINAB-2JR3FR9RD$",
  "TargetDomainName": "DOMAIN.LOCAL",
  "TargetLogonId": "0x121334",
  "LogonType": "3",
  "LogonProcessName": "Kerberos",
  "AuthenticationPackageName": "Kerberos",
  "WorkstationName": "-",
  "LogonGuid": "{72e0f6ab-7332-5b0f-7ecb-96939bff476e}",
  "TransmittedServices": "-",
  "LmPackageName": "-",
  "KeyLength": "0",
  "ProcessId": "0x0",
  "ProcessName": "-",
  "IpAddress": "10.0.0.100",
  "IpPort": "49732",
  "ImpersonationLevel": "%%1840",
  "RestrictedAdminMode": "-",
  "TargetOutboundUserName": "-",
  "TargetOutboundDomainName": "-",
  "VirtualAccount": "%%1843",
  "TargetLinkedLogonId": "0x0",
  "ElevatedToken": "%%1842",
  "EventReceivedTime": "2022-07-26T14:32:51.919931+01:00"
}

Event IDs to monitor

One of the most challenging tasks regarding Windows log collection is deciding which event IDs to monitor. Due to the sheer number of event IDs, this can be daunting at first sight. Therefore, this section will guide you in selecting the event IDs to monitor, and provide example configurations for collecting them.

Event IDs are unique per source but are not globally unique. Different sources may use the same event ID to identify unrelated operations.

Finding the right event IDs

An excellent general resource to start with is Windows 10 and Windows Server 2016 security auditing and monitoring reference. It provides detailed descriptions of event IDs used for security audit policies. Additional resources include:

The table below displays a small sample of important Security events to monitor on Windows servers. In addition, see the Security-focused Event IDs to Monitor configuration examples below.

Table 4. List of Windows security events to monitor
Event ID Description

1102

The audit log was cleared.

4625

An account failed to log on.

4648

A logon was attempted using explicit credentials.

4674

An operation was attempted on a privileged object.

4697

A service was installed in the system.

4704

A user right was assigned.

4705

A user right was removed.

4717

System security access was granted to an account.

4719

System audit policy was changed.

4723

An attempt was made to change an account’s password.

4732

A member was added to a security-enabled local group.

4738

A user account was changed.

4798

A user’s local group membership was enumerated.

4946

A change has been made to Windows Firewall exception list. A rule was added.

4950

A Windows Firewall setting has changed.

6416

A new external device was recognized by the system.

6424

The installation of this device was allowed, after having previously been forbidden by policy.

Example monitoring configurations

Once you have selected the event IDs you will monitor, you can configure the im_msvistalog module with the relevant query. Modify the configuration examples in this section to suit your deployment.

Due to a bug or limitation of the Windows Event Log API, 23 or more clauses in a query will fail with the following error message: ERROR failed to subscribe to msvistalog events, the Query is invalid: This operator is unsupported by this implementation of the filter.; [error code: 15001]
An XPath query that does not specify a provider will collect all matching event IDs globally from all providers. Make sure to configure your SIEM to associate events with the correct provider accordingly.
Example 10. Basic Windows security event log collection

This example provides a basic configuration for monitoring Windows security events. It explicitly specifies a small number of event IDs to be collected.

nxlog.conf
<Input security_events>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Security">*[System[(Level=1  or Level=2 or Level=3 or Level=4 or Level=0)
                        and (EventID=1102 or EventID=4719 or EventID=4704 or EventID=4717 or EventID=4738
                        or EventID=4798 or EventID=4705 or EventID=4674 or EventID=4697 or EventID=4648
                        or EventID=4723 or EventID=4946 or EventID=4950 or EventID=6416 or EventID=6424
                        or EventID=4732)]]
                </Select>
            </Query>
         </QueryList>
    </QueryXML>
</Input>
Example 11. Extended Windows security event log collection

This extended configuration defines groups of event IDs based on their source. Next, it uses the QueryXML directive to collect events from specific providers. Finally, it uses an Exec block to filter out event IDs that are not in the lists.

nxlog.conf
# define Account Usage Events
define AccountUsage        4740, 4648, 4781, 4733, 1518, 4776, 5376, 5377, \
                           4625, 300, 4634, 4672, 4720, 4722, 4782, 4793, \
                           4731, 4735, 4766, 4765, 4624, 1511, 4726, 4725, \
                           4767, 4728, 4732, 4756, 4704

# define Application Crash Events
define AppCrashes          1000, 1002, 1001

# define Application Whitelisting Events
define AppWhitelisting     8023, 8020, 8002, 8003, 8004, 8006, 8007, 4688, \
                           4689, 8005, 865, 866, 867, 868, 882

# define Boot Events
define BootEvents          13, 12

# define Certificate Services Events
define CertServices        95, 4886, 4890, 4874, 4873, 4870, 4887, 4885, \
                           4899, 4896, 1006, 1004, 1007, 1003, 1001, 1002

# define Clearing Event Logs Events
define ClearingLogs        1100, 104, 1102

# define DNS and Directory Services Events
define DNSDirectoryServ    5137, 5141, 5136, 5139, 5138, 3008, 3020

# define External Media Detection events
define ExtMedia            400, 410

# define Group Policy Error Events
define GroupPolicyError    112, 1001, 1125, 1126, 1127, 1129

# define Kernel Driver Signing Events
define KernelDriver        3001, 3002, 3003, 3004, 3010, 3023, 5038, \
                           6281, 219

# define Microsoft Cryptography API Events
define MSFTCryptoAPI       11, 70, 90

# define Mobile Device Activities
define MobileDeviceEvents  10000, 10001


# define Network Host Activities
define NetworkHost         4714, 4713, 4769, 6273, 6275, 6274, 6272, \
                           6278, 6277, 6279, 6276, 6280, 5140, 5145, \
                           5142, 5144, 4706, 1024, 4897, 4719, 4716, \
                           4779, 4778, 5632

# define PassTheHash Detection Events
define PassTheHash         4624, 4625

# define PowerShell Activities
define PowerShell          800, 169, 4103, 4104, 4105, 4106

# define Printing Services Events
define PrintingServices    307

# define Logon Events
define LogonEvents         4624, 4634

# define Software Service Installation Events
define Installation        903, 904, 6, 1022, 1033, 7045, 907, 908, 7000, \
                           800, 2, 905, 906, 19

# define System Integrity Events
define SystemIntegrity     4657, 1, 4616

# define System or Service Failure Events
define SystemServiceFail   7022, 7023, 7024, 7026, 7031, 7032, 7034

# define Task Scheduler Activities
define TaskScheduler       106, 141, 142, 200

# define Windows Defender Activities
define WinDefender         1008, 1006, 1116, 1010, 2003, 2001, 1009, 1118, \
                           1119, 1007, 1117, 3002, 2004, 1005, 5008

# define Windows Firewall Events
define WinFirewall         2009, 2004, 2005, 2006, 2033

# define Windows Update Error Events
define WinUpdateError      1009, 20, 24, 25, 31, 34, 35

<Input security_events>
    Module                 im_msvistalog
    TolerateQueryErrors    TRUE
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Application">*</Select>
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Assistant">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Troubleshooter">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Inventory">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Telemetry">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Steps-Recorder">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/Packaged app-Deployment">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/Packaged app-Execution">*</Select>
                <Select Path="Microsoft-Windows-CAPI2/Operational">*</Select>
                <Select Path="Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational">*</Select>
                <Select Path="Microsoft-Windows-DNS-Client/Operational">*</Select>
                <Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select>
                <Select Path="Microsoft-Windows-Kernel-PnP/Configuration">*</Select>
                <Select Path="Microsoft-Windows-NetworkProfile/Operational">*</Select>
                <Select Path="Microsoft-Windows-NTLM/Operational">*</Select>
                <Select Path="Microsoft-Windows-PowerShell/Admin">*</Select>
                <Select Path="Microsoft-Windows-PowerShell/Operational">*</Select>
                <Select Path="Microsoft-Windows-PrintService/Admin">*</Select>
                <Select Path="Microsoft-Windows-PrintService/Operational">*</Select>
                <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-RDPClient/Operational">*</Select>
                <Select Path="Microsoft-Windows-User Profile Service/Operational">*</Select>
                <Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select>
                <Select Path="Microsoft-Windows-Windows Defender/WHC">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/Firewall">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallDiagnostics">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose">*</Select>
                <Select Path="Network Isolation Operational">*</Select>
                <Select Path="Microsoft-Windows-WindowsUpdateClient/Operational">*</Select>
                <Select Path="Windows PowerShell">*</Select>
                <Select Path="Microsoft-Windows-CodeIntegrity/Operational">*[System[Provider[@Name='Microsoft-Windows-CodeIntegrity']]]</Select>
                <Select Path="Microsoft-Windows-LSA/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
       if ($EventID NOT IN (%AccountUsage%)) and
          ($EventID NOT IN (%AppCrashes%)) and
          ($EventID NOT IN (%AppWhitelisting%)) and
          ($EventID NOT IN (%BootEvents%)) and
          ($EventID NOT IN (%CertServices%)) and
          ($EventID NOT IN (%ClearingLogs%)) and
          ($EventID NOT IN (%DNSDirectoryServ%)) and
          ($EventID NOT IN (%ExtMedia%)) and
          ($EventID NOT IN (%GroupPolicyError%)) and
          ($EventID NOT IN (%KernelDriver%)) and
          ($EventID NOT IN (%MSFTCryptoAPI%)) and
          ($EventID NOT IN (%MobileDeviceEvents%)) and
          ($EventID NOT IN (%NetworkHost%)) and
          ($EventID NOT IN (%PassTheHash%)) and
          ($EventID NOT IN (%PowerShell%)) and
          ($EventID NOT IN (%PrintingServices%)) and
          ($EventID NOT IN (%LogonEvents%)) and
          ($EventID NOT IN (%Installation%)) and
          ($EventID NOT IN (%SystemIntegrity%)) and
          ($EventID NOT IN (%SystemServiceFail%)) and
          ($EventID NOT IN (%TaskScheduler%)) and
          ($EventID NOT IN (%WinDefender%)) and
          ($EventID NOT IN (%WinFirewall%)) and
          ($EventID NOT IN (%WinUpdateError%)) {
            drop();
        }
    </Exec>
</Input>
Example 12. Collecting Windows event logs indicating lateral movements

Similar to the extended configuration above, this configuration defines groups of event IDs associated with detecting malicious lateral movements. It is based on the Computer Emergency Response Team (CERT) research on Detecting Lateral Movement through Tracking Event Logs.

nxlog.conf
# define Security Events
define SecurityEvents          4624, 4634, 4648, 4656, 4658, 4660, 4663, 4672, \
                               4673, 4688, 4689, 4698, 4720, 4768, 4769, 4946, \
                               5140, 5142, 5144, 5145, 5154, 5156, 5447, 8222

# define Sysmon Events
define SysmonEvents             1, 2, 5, 8, 9

# define Application Management event
define ApplicationMgmt          104

# define Windows Remote Management Events
define WRMEvents                80, 132, 143, 166, 81

# define Task Scheduler - Operational Events
define TaskSchedEvents          106, 129, 200, 201

# define Local Session Manager - Operational Events
define LocalSessionMgrEvents    21, 24

#define BitsClient Events
define BitsClientsEvents        60

<Input lateral_security_events>
    Module                 im_msvistalog
    TolerateQueryErrors    TRUE
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
                <Select Path="Microsoft-Windows-Bits-Client/Operational">*</Select>
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
                <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Admin">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Operational">*</Select>
                <Select Path="Microsoft-Windows-WinRM/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
       if ($EventID NOT IN (%SecurityEvents%)) and
          ($EventID NOT IN (%SysmonEvents%)) and
          ($EventID NOT IN (%WRMEvents%)) and
          ($EventID NOT IN (%TaskSchedEvents%)) and
          ($EventID NOT IN (%ApplicationMgmt%)) and
          ($EventID NOT IN (%LocalSessionMgrEvents%)) and
          ($EventID NOT IN (%BitsClientsEvents%)) {
           drop();
       }
    </Exec>
</Input>

Forwarding Windows events

After collecting events from a Windows system with NXLog, you may need to send them to another host. This section provides details and configuration examples for forwarding event data.

Descriptions in Windows events may contain tabs and newlines, but some formats like BSD syslog do not support these characters. In this case, you can use a regular expression to remove them.

Example 13. Removing tabs and newlines from Windows events

This configuration uses a regular expression to replace all tab characters and newline sequences in the event description with spaces.

nxlog.conf
<Input eventlog>
    Module    im_mseventlog
    Exec      $Message =~ s/(\t|\R)/ /g;
</Input>

Forwarding Windows events in BSD syslog format

You can convert Windows events to BSD syslog format with the to_syslog_bsd() procedure of the xm_syslog module. For more information, see Forward syslog to a remote logger via UDP, TCP, or TLS.

Example 14. Converting Windows events to BSD syslog format

This configuration removes tab characters and newline sequences from the $Message field, converts the event record to BSD syslog format, and forwards the event via UDP.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <Exec>
        $Message =~ s/(\t|\R)/ /g;
        to_syslog_bsd();
    </Exec>
</Input>

<Output udp>
    Module    om_udp
    Host      10.10.1.1:514
</Output>
The to_syslog_bsd() procedure will only use a subset of the Windows Event Log fields.
Output sample
<14>Sep 20 10:21:16 win7host Service_Control_Manager[448]: The Computer Browser service entered the running state.

Forwarding Windows events in JSON format

Most SIEM solutions accept logs in JSON format to preserve all event log fields. The xm_json module provides the to_json() procedure to convert data to JSON format. See the examples in the NXLog Enterprise Edition Reference Manual to better understand how NXLog parses and converts JSON.

Example 15. Converting Windows events to JSON format

This configuration converts the event record to JSON format and forwards the event via TCP.

nxlog.conf
<Extension json>
    Module    xm_json
</Extension>

<Input eventlog>
    Module    im_msvistalog
    Exec      to_json();
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.10.1:1514
</Output>
Output sample
{
  "EventTime": "2022-09-20 10:21:16",
  "Hostname": "win7host",
  "Keywords": -9187343239835812000,
  "EventType": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventID": 7036,
  "SourceName": "Service Control Manager",
  "ProviderGuid": "{525908D1-A6D5-5695-8E2E-26921D2011F3}",
  "Version": 0,
  "Task": 0,
  "OpcodeValue": 0,
  "RecordNumber": 2629,
  "ProcessID": 448,
  "ThreadID": 2872,
  "Channel": "System",
  "Message": "The Computer Browser service entered the running state.",
  "param1": "Computer Browser",
  "param2": "running",
  "EventReceivedTime": "2022-09-20 10:21:17",
  "SourceModuleName": "eventlog",
  "SourceModuleType": "im_msvistalog"
}

Some SIEMs require you to send data in syslog-encapsulated JSON.

Example 16. Converting Windows events to syslog-encapsulated JSON

This configuration converts the event record to JSON, adds a BSD syslog header, and forwards the event via UDP.

nxlog.conf
<Extension json>
    Module    xm_json
</Extension>

<Extension syslog>
    Module    xm_syslog
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <Exec>
        $Message = to_json();
        to_syslog_bsd();
    </Exec>
</Input>

<Output udp>
    Module    om_udp
    Host      192.168.2.1:514
</Output>
Output sample
<14>Sep 20 10:21:16 win7host Service_Control_Manager[448]: {"EventTime":"2022-09-20 10:21:16","Hostname":"win7host","Keywords":-9187343239835811840,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":7036,"SourceName":"Service Control Manager","ProviderGuid":"{525908D1-A6D5-5695-8E2E-26921D2011F3}","Version":0,"Task":0,"OpcodeValue":0,"RecordNumber":2629,"ProcessID":448,"ThreadID":2872,"Channel":"System","Message":"The Computer Browser service entered the running state.","param1":"Computer Browser","param2":"running","EventReceivedTime":"2022-09-20 10:21:17","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"}

Forwarding Windows events in Snare format

Snare is another format commonly used with Windows events. The xm_syslog module includes the to_syslog_snare() procedure to generate syslog-encapsulated Snare logs. See the Snare integration guide for more information.

Example 17. Converting Windows events to Snare format

This configuration removes tab characters and newline sequences from the $Message field, converts the event record to the Snare over syslog format, and forwards the event via UDP.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <Exec>
        $Message =~ s/(\t|\R)/ /g;
        to_syslog_snare();
    </Exec>
</Input>

<Output snare>
    Module  om_udp
    Host    192.168.1.1:514
</Output>
Output sample
<14>Sep 20 10:21:16 win7host MSWinEventLog	1	System	193	Mon Sep 20 10:21:16 2017	7036	Service Control Manager	N/A	N/A	Information	win7host	N/A		The Computer Browser service entered the running state.	2773

Processing EVTX files on Linux

EVTX files collected from Windows systems can be processed on Linux with NXLog using an external script.

Example 18. Reading EVTX files using Python

This configuration uses the im_python module to execute a script and converts the events to JSON format. The Python script uses the python-evtx module to read the events from the EVTX file.

nxlog.conf
<Extension xml>
    Module        xm_xml
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input evtx>
    Module        im_python
    PythonCode    /opt/nxlog/etc/process_evtx.py
    <Exec>
        parse_windows_eventlog_xml($Message);
        delete($Message);
        to_json();
    </Exec>
</Input>
process_evtx.py
import Evtx.Evtx as evtx
import Evtx.Views as e_views

import nxlog

def read_data(module):
    nxlog.log_debug('Starting Processing EVTX')
    with evtx.Evtx('/tmp/security.evtx') as log:
        for record in log.records():
            event = module.logdata_new()
            event.set_field('Message', record.xml())
            event.post()
Output sample
{
  "EventReceivedTime": "2022-09-20T18:47:06.548281+01:00",
  "SourceModuleName": "evtx",
  "SourceModuleType": "im_python",
  "SourceName": "Microsoft-Windows-Security-Auditing",
  "ProviderGuid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
  "EventID": 5379,
  "Version": 0,
  "LevelValue": 0,
  "TaskValue": 13824,
  "OpcodeValue": 0,
  "Keywords": "0x8020000000000000",
  "EventTime": "2022-09-16T17:36:15.751457+01:00",
  "RecordNumber": 342859,
  "ActivityID": "{d9a2c36d-bc3e-0003-83c3-a2d93ebcd601}",
  "ExecutionProcessID": 872,
  "ExecutionThreadID": 956,
  "Channel": "Security",
  "Hostname": "PC1",
  "SubjectUserSid": "S-1-5-21-774634756-2905300177-2392840219-1003",
  "SubjectUserName": "John",
  "SubjectDomainName": "PC1",
  "SubjectLogonId": "0x00000000000ccd46",
  "TargetName": "MicrosoftOffice*",
  "Type": "0",
  "CountOfCredentialsReturned": "1",
  "ReadOperation": "%%8100",
  "ReturnCode": "0",
  "ProcessCreationTime": "2022-09-16 17:36:12.644609",
  "ClientProcessId": "12088",
  "EventType": "AUDIT_SUCCESS",
  "SeverityValue": 2,
  "Severity": "INFO"
}
Sometimes, events from the Windows Event Log contain values that need to be resolved using an external reference. Processing EVTX files using the python-evtx library may result in some events containing unresolved values.
Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

Windows 10 Pro
NXLog 5.5.7535

Last revision: 3 October 2022

© Copyright NXLog Ltd.

PRIVACY POLICY TERMS OF USE