Windows Event Log

This section discusses the various details of Windows Event Logs.

About Windows Event Log

Windows Event Log captures the details of both system and application events. When such an event occurs, Windows records it in the event log. The event log is then used to find details about the event and can be helpful when troubleshooting problems. Beside their use for IT related purposes, Windows Event Logs are also used to satisfy compliance mandates.

Unlike other event logs, such as the UNIX Syslog, Windows Event Log is not stored as a plain text file, but in a proprietary binary format. It is not possible to view Windows Event Log in a text editor, nor is it possible to send it as a Syslog event while retaining its original format. However, the raw event data can be translated into XML using the Windows Event Log API and forwarded in that format.

The EVTX file format

Windows stores Windows Event Log files in the EVTX file format since the release of Windows Vista and Windows Server 2008. Prior to that, event log files were stored in the EVT file format. Both are proprietary formats readable by the Microsoft Management Console (MMC) snap-in eventvwr.msc.

The EVTX format includes many new features and enhancements: a number of new event properties, the use of channels to publish events, a new Event Viewer, a rewritten Windows Event Log service, and support for the Extensible Markup Language (XML) format. From a log processing perspective, the added support for XML is the most important addition, as it provides the possibility to share or further process the event data in a structured format.

For the built in channels, Windows automatically saves the corresponding EVTX file into the C:\Windows\System32\winevt\Logs\ directory. Events can also be saved manually from the Event Viewer MMC snap-in, in four different formats: EVTX, XML, TXT, and CSV.

NXLog can directly read EVTX and EVT files using the im_msvistalog File directive. In addition, the CaptureEventXML directive of the same module can be used to store and send raw XML-formatted event data in the $EventXML field.

Viewing the Windows Event Log

The Windows Event Log can be viewed in the Event Viewer MMC snap-in included in Windows. Windows Event Logs are stored in a binary source data format, which is the "source" or "on-disk" format. It does not include the full message, only the event properties. When an event is rendered, property values are inserted into the localized message template stored elsewhere on disk.

The Event Viewer includes three views for displaying the data for a selected event. These are shown on the preview pane or in the Event Properties window when an event is opened.

  • The general view is shown by default. It includes the full message rendered from template and the "System" set of key/value pairs.

  • The Friendly View is available on the Details tab. It shows a hierachical view of the System properties and additional EventData properties defined by the event provider. It does not show a rendered message.

  • The XML View can be selected under the Details tab. It shows the event properties in XML format. It does not show a rendered message.

    A Windows Event Log event in XML format
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing"
              Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
    <EventID>4624</EventID>
    [...]
    <Channel>Security</Channel>
    <Computer>USER-WORKSTATION</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="SubjectUserSid">S-1-5-18</Data>
    [...]
  </EventData>
</Event>

Events can be accessed through the Event Log API (see Windows Event Log Functions on Microsoft Docs). In particular:

  • EvtQuery() fetches events from a given channel or log file that match a given query—see Querying for Events.

  • EvtFormatMessage() generates a message string for an event using the event properties and the localized message template—see Formatting Event Messages.

Event channels

The EVTX format introduces event channels. A channel is a stream of events that collects events from a publisher and writes them to an event log file.

Channels are organized into two groups:

  • The Windows Logs group contains a set of exactly five channels, which are used for Windows system events.

  • The Applications and Services Logs group contains channels created for individual applications or components. These channels are further organized in a folder hierarchy.

There are two channel types indicating how the events are handled:

  • Serviced channels offer relatively low volume, reliable delivery of events. Events in these channels may be forwarded to another system, and these channels may be subscribed to.

  • Direct channels are for high-performance collection of events. It is not possible to subscribe to a a direct channel. By default, these channels are disabled. To see these channels in the Event Viewer, check Show Analytic and Debug Logs in the View menu. To enable logging for one of these channels, select the channel, open the Action menu, click Properties, and check Enable logging on the General tab.

Each of the above is subdivided into two more channel types according to the the intended audience for the events collected by that channel:

  • Administrative channels collects events for end users, administrators, and support. This is a serviced channel type.

  • Operational channels collect events used for diagnosing problems. This is a serviced channel type.

  • Analytic channels are for events that describe program operation. These channels often collect a high volume of events. This is a direct channel type.

  • Debug channels are intended to be used by developers only. This is a direct channel type.

Table 1. Channel groups and types
Channel Groups Channels Channel Type

Windows Logs

Application

Administrative (serviced)

Security

Administrative (serviced)

Setup

Operational (serviced)

System

Administrative (serviced)

Forwarded Events

Operational (serviced)

Applications and Services Logs

DHCP-Server/Admin

Administrative (serviced)

DHCP-Server/AuditLogs

Analytic (direct)

DHCP-Server/DebugLogs

Debug (direct)

(And many more publisher-defined channels)

The im_msvistalog module can be configured to collect events from a specific channel with the Channel directive.

For more information about event channels, see these two pages on Microsoft Docs: Event Logs and Event Logs and Channels in Windows Event Log.

Providers

Event log providers write events to event logs. An event log provider can be a service, driver, or program that runs on the computer and has the necessary instrumentation to write to the event log.

Event providers are categorized into four main types.

  • Manage Object Format (MOF) providers (also referred to as "classic")

  • Windows Software Trace Preprocessor (WPP) providers

  • Manifest-based providers

  • TraceLogging providers

For more information on providers, see the Providers section in the Microsoft Windows documentation.

Collecting Event Log Data

This section lists and discusses the NXLog modules that can be used to collect Windows Event Log data.

NXLog Modules for Windows Event Log

NXLog provides the following modules for capturing Windows Event Log data.

  • The im_msvistalog module is available on Windows only, and captures event log data from Windows 2008/Vista and later. It can be configured to collect event log data from the local system or from a remote system via MSRPC (MSRPC is supported by NXLog Enterprise Edition only). See Local collection with im_msvistalog and Remote collection with im_msvistalog.

  • The im_wseventing module is available on both Linux and Windows (NXLog Enterprise Edition only). With it, event log data can be received from remote Windows systems using Windows Event Forwarding. This is the recommended module for most cases where remote capturing is required, because it is not necessary to specify each host that Event Log data will be captured from. See Remote collection with im_wseventing.

  • The im_mseventlog module is available on Windows only, and captures event log data locally from Windows XP, Windows 2000, and Windows 2003. See Local collection with im_mseventlog.

Local collection with im_msvistalog

The im_msvistalog module can capture Event Log data from the local system running Windows 2008/Vista or later.

Because the Windows Event Log subsystem does not support subscriptions to the Debug and Analytic channels, these types of events can not be collected with the im_msvistalog module.
Example 1. Collecting Windows Event Log locally from Windows 2008/Vista or later

In this example, NXLog reads all events from the local Windows Event Log. The data is converted to JSON format and written to a local file.

nxlog.conf
<Extension _json>
    Module  xm_json
</Extension>

<Input eventlog>
    Module  im_msvistalog
</Input>

<Output file>
    Module  om_file
    File    'C:\test\sysmon.json'
    Exec    to_json();
</Output>

For information about filtering events, particularly when using im_msvistalog, see Filtering events.

Remote collection with im_msvistalog

NXLog Enterprise Edition can be configured with the im_msvistalog module for collection of events generated on remote Windows systems. In this mode, it is not necessary to run an NXLog agent on the Windows systems. Instead, MSRPC is used to receive the events.

Because the Windows Event Log subsystem does not support subscriptions to the Debug and Analytic channels, these types of events can not be collected with the im_msvistalog module.
Example 2. Receiving Event Log data over MSRPC

In this example configuration, the im_msvistalog module is used to get events from a remote server named mywindowsbox using MSRPC.

To replicate this example in your environment, modify the RemoteServer, RemoteUser, RemoteDomain, and RemotePassword to reflect the access credentials for the target machine.

nxlog.conf
<Input in>
    Module          im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='1'>
                <Select Path='Application'>*</Select>
                <Select Path='Security'>*[System/Level=4]</Select>
                <Select Path='System'>*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    RemoteServer   mywindowsbox
    RemoteUser     Administrator
    RemoteDomain   Workgroup
    RemotePassword secret
</Input>

Local collection with im_mseventlog

The im_mseventlog module can capture Event Log data from Windows XP, Windows 2000, and Windows 2003.

The module looks up the available Event Log sources stored under the registry key SYSTEM\CurrentControlSet\Services\Eventlog, and polls logs from each of these or only the sources defined with the Sources directive of the NXLog configuration.

Example 3. Forwarding Event Log Data from Windows to a Remote Host

This example shows the most basic configuration of the im_mseventlog module. This configuration forwards all Event Log sources listed in the Windows registry over the network to a remote NXLog instance at the IP address 192.168.1.1.

nxlog.conf
<Input eventlog>
    Module  im_mseventlog
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.1.1
    Port    514
</Output>

Remote collection with im_wseventing

NXLog Enterprise Edition can use the im_wseventing module to receive Windows Event Log data from remote machines over WEF (Windows Event Forwarding). It works on both Windows and Linux hosts.

Example 4. Receiving Windows Event Log data using the im_wseventing module

This configuration receives data from all source computers, by listening on port 5985 for connections from all sources. It also shows a configured HTTPS certificate, used to secure the transfer of Event Log data.

nxlog.conf
<Input in>
    Module           im_wseventing
    ListenAddr       0.0.0.0
    Port             5985
    Address          https://linux.corp.domain.com:5985/wsman
    HTTPSCertFile    %CERTDIR%/server-cert.pem
    HTTPSCertKeyFile %CERTDIR%/server-key.pem
    HTTPSCAFile      %CERTDIR%/ca.pem
    <QueryXML>
        <QueryList>
            <Query Id="0" Path="Application">
                <Select Path="Application">*</Select>
                <Select Path="Microsoft-Windows-Winsock-AFD/Operational">*</Select>
                <Select Path="Microsoft-Windows-Wired-AutoConfig/Operational">
                    *
                </Select>
                <Select Path="Microsoft-Windows-Wordpad/Admin">*</Select>
                <Select Path="Windows PowerShell">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

A query for specific hosts can be set by adding an additional QueryXML block with a <Computer> tag. This tag contains a pattern that NXLog matches against the name of the connecting Windows client. Computer names not matching the pattern will use the default QueryXML block (containing no <Computer> tag). The following QueryXML block, if added to the above configuration, would provide an alternate query for computer names matching the pattern foo*.

nxlog.conf
<QueryXML>
    <Computer>foo*</Computer>
    <QueryList>
        <Query Id="0" Path="Application">
            <Select Path="Application">*</Select>
        </Query>
    </QueryList>
</QueryXML>

Filtering events

Systems and services on Windows can generate a large volume of logs, and it is often necessary to collect only a certain portion of those events. There are several ways to implement filtering of events from the Windows Event Log when using the im_msvistalog module.

  • A specific channel can be specified with the Channel directive to collect all the events written to a single channel.

  • An XPath query can be given with the QueryXML block (or Query directive). The specified query is then used to subscribe to events. An XPath query can be used to subscribe to multiple channels and/or limit events by various attributes. However, XPath queries have a maximum length, limiting the possibilities for detailed event subscriptions. See XPath filtering below.

  • A log file can be read by setting the File directive, in which case im_msvistalog will read all events from the file (for example, Security.evtx). This is intended primarily for forensics purposes, such as with nxlog-processor.

  • After being read from the source, events can be discarded by matching events in an Exec block and discarding them selectively with the drop() procedure.

Subscribing to a restricted set of events with an XPath query can offer a performance advantage because the events are never received by NXLog. However, XPath queries have a maximum length and limited filtering capabilities, so in some cases it is necessary to combine an XPath query with Exec block filtering in an im_msvistalog configuration. For examples, see examples in Event IDs to Monitor.

XPath filtering

XPath queries can be used to subscribe to events matching certain criteria, both in the Event Viewer and with the im_msvistalog QueryXML directive. Windows Event Log supports a subset of XPath 1.0. For more information, see Consuming Events on Microsoft Docs.

The Event Viewer offers the most practical way to write and test query strings. An XPath query can be generated and/or tested by filtering the current log or creating a custom view.

  1. In the Event Viewer, click an event channel to open it, then right-click the channel and choose Filter Current Log from the context menu. Or, click Create Custom View in the context menu. Either way, a dialog box will open and options for basic filtering will be shown in the Filter tab.

    Basic Filtering

  2. Specify the desired criteria. The corresponding XPath query on the XML tab will be updated automatically.

  3. To view the query string, switch to the XML tab. This string can be copied into the im_msvistalog QueryXML directive.

  4. If required, advanced filtering can be done by selecting the Edit query manually checkbox and editing the query. The query can then be tested to be sure it matches the correct events and finally copied to the NXLog configuration with the QueryXML block.

    XPath XML
    Figure 1. A Custom View Querying the Application Channel for Events With ID 1008

Sometimes it is helpful to use a query with sources that may not be available. In this case, set the TolerateQueryErrors directive to TRUE to ensure that the module will continue to collect logs.

Example 5. Collecting operational events only

Here, NXLog queries the local Windows Event Log for operational events only.

nxlog.conf
<Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>
Example 6. Collecting important system events

This query collects System channel events with levels below 4 (Critical, Error, and Warning).

nxlog.conf
<Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path='System'>*[System/Level&lt;4]</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

Exec block filtering

NXLog’s built-in filtering capabilities can also be used to filter events, by matching events and using the drop() procedure. Events can be matched against any of the im_msvistalog fields.

Example 7. Filtering Sysmon events in an exec block

This example discards all Sysmon network connection events (event ID 3) regarding HTTP network connections to a particular server and port, and all process creation and termination events (event IDs 1 and 5) for conhost.exe.

nxlog.conf
<Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
        if ($EventID in (1, 5) and
            $Image == "C:\\Windows\\System32\\conhost.exe") or
           ($EventID == 3 and
            $DestinationPort == 80 and
            $DestinationIp == 10.0.0.1)
           drop();
    </Exec>
</Input>

Event IDs to monitor

When it comes to Windows log collection, one of the most challenging tasks of a system administrator is deciding which event IDs to monitor. Due to the large number of event IDs in use, this can be daunting at first sight. Therefore, this section aims to provide guidance about selecting event IDs to monitor, with some example configurations.

Event IDs are unique per source but are not globally unique. The same event ID may be used by different sources to identify unrelated occurrences.

Finding the right event IDs

An excellent general source to start with is the Windows 10 and Windows Server 2016 security auditing and monitoring reference. It provides detailed descriptions about event IDs used for security audit policies. There are additional resources to find events to monitor, see below:

The table below displays a small sample of important events to monitor in the Windows Server Security Log for a local server. See the Security-focused Event IDs to Monitor section for the configuration file holding these event IDs.

Table 2. Example list of security-focused event IDs to monitor
Event ID Description

1102

The audit log was cleared.

4719

System audit policy was changed.

4704

A user right was assigned.

4717

System security access was granted to an account.

4738

A user account was changed.

4798

A user’s local group membership was enumerated.

4705

A user right was removed.

4674

An operation was attempted on a privileged object.

4732

A member was added to a security-enabled local group.

4697

A service was installed in the system.

4625

An account failed to log on.

4648

A logon was attempted using explicit credentials.

4723

An attempt was made to change an account’s password.

4946

A change has been made to Windows Firewall exception list. A rule was added.

4950

A Windows Firewall setting has changed.

6416

A new external device was recognized by the system.

6424

The installation of this device was allowed, after having previously been forbidden by policy.

Example monitoring configurations

Once a set of event IDs has been selected for monitoring, the im_msvistalog module can be configured.

The example configurations in this section are likely to require further modifications to suit each individual deployment.
Due to a bug or limitation of the Windows Event Log API, 23 or more clauses in a query will result in a failure with the following error message: ERROR failed to subscribe to msvistalog events, the Query is invalid: This operator is unsupported by this implementation of the filter.; [error code: 15001]
Event IDs are globally applied to all providers of a given XPath expression so events that match these IDs will be collected. You should tweak your chosen dashboard or alerting system to ensure that the right Event IDs and its subsequent providers are appropriately associated.
Example 8. Basic configuration example of security-focused event IDs to monitor

This configuration provides a basic example of Windows Security events to monitor. Since only a small number of IDs are presented, this configuration explicitly provides the actual event IDs to be collected.

nxlog.conf
<Input MonitorWindowsSecurityEvents>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Security">*[System[(Level=1  or Level=2 or Level=3 or Level=4 or Level=0) and (EventID=1102 or EventID=4719 or EventID=4704 or EventID=4717 or EventID=4738 or EventID=4798 or EventID=4705 or EventID=4674 or EventID=4697 or EventID=4648 or EventID=4723 or EventID=4946 or EventID=4950 or EventID=6416 or EventID=6424 or EventID=4732)]]</Select>
            </Query>
         </QueryList>
    </QueryXML>
</Input>
Example 9. Extended configuration example of security-focused event IDs to monitor

This extended configuration provides a much wider scope of log collection. Note that this approach for specifying the event IDs requires defining the event IDs based on groups of events first. The QueryXML paths are added in the QueryXML block in bulk. Then the Exec block will filter for the defined event IDs, but only within the paths specified. It also drops event IDs that are not defined.

nxlog.conf
# define Account Usage Events
define AccountUsage        4740, 4648, 4781, 4733, 1518, 4776, 5376, 5377, \
                           4625, 300, 4634, 4672, 4720, 4722, 4782, 4793, \
                           4731, 4735, 4766, 4765, 4624, 1511, 4726, 4725, \
                           4767, 4728, 4732, 4756, 4704

# define Application Crash Events
define AppCrashes          1000, 1002, 1001

# define Application Whitelisting Events
define AppWhitelisting     8023, 8020, 8002, 8003, 8004, 8006, 8007, 4688, \
                           4689, 8005, 865, 866, 867, 868, 882

# define Boot Events
define BootEvents          13, 12

# define Certificate Services Events
define CertServices        95, 4886, 4890, 4874, 4873, 4870, 4887, 4885, \
                           4899, 4896, 1006, 1004, 1007, 1003, 1001, 1002

# define Clearing Event Logs Events
define ClearingLogs        1100, 104, 1102

# define DNS and Directory Services Events
define DNSDirectoryServ    5137, 5141, 5136, 5139, 5138, 3008, 3020

# define External Media Detection events
define ExtMedia            400, 410

# define Group Policy Error Events
define GroupPolicyError    112, 1001, 1125, 1126, 1127, 1129

# define Kernel Driver Signing Events
define KernelDriver        3001, 3002, 3003, 3004, 3010, 3023, 5038, \
                           6281, 219

# define Microsoft Cryptography API Events
define MSFTCryptoAPI       11, 70, 90

# define Mobile Device Activities
define MobileDeviceEvents  10000, 10001


# define Network Host Activities
define NetworkHost         4714, 4713, 4769, 6273, 6275, 6274, 6272, \
                           6278, 6277, 6279, 6276, 6280, 5140, 5145, \
                           5142, 5144, 4706, 1024, 4897, 4719, 4716, \
                           4779, 4778, 5632

# define PassTheHash Detection Events
define PassTheHash         4624, 4625

# define PowerShell Activities
define PowerShell          800, 169, 4103, 4104, 4105, 4106

# define Printing Services Events
define PrintingServices    307

# define Logon Events
define LogonEvents         4624, 4634

# define Software Service Installation Events
define Installation        903, 904, 6, 1022, 1033, 7045, 907, 908, 7000, \
                           800, 2, 905, 906, 19

# define System Integrity Events
define SystemIntegrity     4657, 1, 4616

# define System or Service Failure Events
define SystemServiceFail   7022, 7023, 7024, 7026, 7031, 7032, 7034

# define Task Scheduler Activities
define TaskScheduler       106, 141, 142, 200

# define Windows Defender Activities
define WinDefender         1008, 1006, 1116, 1010, 2003, 2001, 1009, 1118, \
                           1119, 1007, 1117, 3002, 2004, 1005, 5008

# define Windows Firewall Events
define WinFirewall         2009, 2004, 2005, 2006, 2033

# define Windows Update Error Events
define WinUpdateError      1009, 20, 24, 25, 31, 34, 35

<Input ExtendedWindowsToCollect>
    Module                 im_msvistalog
    TolerateQueryErrors    TRUE
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Application">*</Select>
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Assistant">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Troubleshooter">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Inventory">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Program-Telemetry">*</Select>
                <Select Path="Microsoft-Windows-Application-Experience/Steps-Recorder">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/Packaged app-Deployment">*</Select>
                <Select Path="Microsoft-Windows-AppLocker/Packaged app-Execution">*</Select>
                <Select Path="Microsoft-Windows-CAPI2/Operational">*</Select>
                <Select Path="Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational">*</Select>
                <Select Path="Microsoft-Windows-DNS-Client/Operational">*</Select>
                <Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select>
                <Select Path="Microsoft-Windows-Kernel-PnP/Configuration">*</Select>
                <Select Path="Microsoft-Windows-NetworkProfile/Operational">*</Select>
                <Select Path="Microsoft-Windows-NTLM/Operational">*</Select>
                <Select Path="Microsoft-Windows-PowerShell/Admin">*</Select>
                <Select Path="Microsoft-Windows-PowerShell/Operational">*</Select>
                <Select Path="Microsoft-Windows-PrintService/Admin">*</Select>
                <Select Path="Microsoft-Windows-PrintService/Operational">*</Select>
                <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-RDPClient/Operational">*</Select>
                <Select Path="Microsoft-Windows-User Profile Service/Operational">*</Select>
                <Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select>
                <Select Path="Microsoft-Windows-Windows Defender/WHC">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/Firewall">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallDiagnostics">*</Select>
                <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose">*</Select>
                <Select Path="Network Isolation Operational">*</Select>
                <Select Path="Microsoft-Windows-WindowsUpdateClient/Operational">*</Select>
                <Select Path="Windows PowerShell">*</Select>
                <Select Path="Microsoft-Windows-CodeIntegrity/Operational">*[System[Provider[@Name='Microsoft-Windows-CodeIntegrity']]]</Select>
                <Select Path="Microsoft-Windows-LSA/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
       if ($EventID NOT IN (%AccountUsage%)) and
          ($EventID NOT IN (%AppCrashes%)) and
          ($EventID NOT IN (%AppWhitelisting%)) and
          ($EventID NOT IN (%BootEvents%)) and
          ($EventID NOT IN (%CertServices%)) and
          ($EventID NOT IN (%ClearingLogs%)) and
          ($EventID NOT IN (%DNSDirectoryServ%)) and
          ($EventID NOT IN (%ExtMedia%)) and
          ($EventID NOT IN (%GroupPolicyError%)) and
          ($EventID NOT IN (%KernelDriver%)) and
          ($EventID NOT IN (%MSFTCryptoAPI%)) and
          ($EventID NOT IN (%MobileDeviceEvents%)) and
          ($EventID NOT IN (%NetworkHost%)) and
          ($EventID NOT IN (%PassTheHash%)) and
          ($EventID NOT IN (%PowerShell%)) and
          ($EventID NOT IN (%PrintingServices%)) and
          ($EventID NOT IN (%LogonEvents%)) and
          ($EventID NOT IN (%Installation%)) and
          ($EventID NOT IN (%SystemIntegrity%)) and
          ($EventID NOT IN (%SystemServiceFail%)) and
          ($EventID NOT IN (%TaskScheduler%)) and
          ($EventID NOT IN (%WinDefender%)) and
          ($EventID NOT IN (%WinFirewall%)) and
          ($EventID NOT IN (%WinUpdateError%)) drop();
    </Exec>
</Input>
Example 10. Configuration example of event IDs corresponding to lateral movements

This configuration, similar to the extended configuration above, lists event IDs associated with the detection of malicious lateral movements. It is based on the security research conducted by the CERT (Computer Emergency Response Team) cybersecurity researchers on Detecting Lateral Movement through Tracking Event Logs.

nxlog.conf
# define Security Events
define SecurityEvents          4624, 4634, 4648, 4656, 4658, 4660, 4663, 4672, \
                               4673, 4688, 4689, 4698, 4720, 4768, 4769, 4946, \
                               5140, 5142, 5144, 5145, 5154, 5156, 5447, 8222

# define Sysmon Events
define SysmonEvents             1, 2, 5, 8, 9

# define Application Management event
define ApplicationMgmt          104

# define Windows Remote Management Events
define WRMEvents                80, 132, 143, 166, 81

# define Task Scheduler - Operational Events
define TaskSchedEvents          106, 129, 200, 201

# define Local Session Manager - Operational Events
define LocalSessionMgrEvents    21, 24

#define BitsClient Events
define BitsClientsEvents        60

<Input LateralMovementEvents>
    Module                 im_msvistalog
    TolerateQueryErrors    TRUE
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
                <Select Path="Microsoft-Windows-Bits-Client/Operational">*</Select>
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
                <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Admin">*</Select>
                <Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Operational">*</Select>
                <Select Path="Microsoft-Windows-WinRM/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
       if ($EventID NOT IN (%SecurityEvents%)) and
          ($EventID NOT IN (%SysmonEvents%)) and
          ($EventID NOT IN (%WRMEvents%)) and
          ($EventID NOT IN (%TaskSchedEvents%)) and
          ($EventID NOT IN (%ApplicationMgmt%)) and
          ($EventID NOT IN (%LocalSessionMgrEvents%)) and
          ($EventID NOT IN (%BitsClientsEvents%)) drop();
    </Exec>
</Input>

Forwarding Event Log Data

After collecting the Event Log data from a Windows system with NXLog, it may need to be sent to another host. This section provides details and examples for configuring this.

Event descriptions in Event Log data may contain tabs and newlines, but these are not supported by some formats like BSD Syslog. In this case, a regular expression can be used to remove them.

Example 11. Removing tabs and newline sequences

This input instance is configured to modify the $Message field (the event description) by replacing all tab characters and newline sequences with spaces.

nxlog.conf
<Input in>
    Module  im_mseventlog
    Exec    $Message =~ s/(\t|\R)/ /g;
</Input>

Forwarding Event Log in BSD Syslog format

Event Log data is commonly sent in the BSD Syslog format. This can be generated with the to_syslog_bsd() procedure provided by the xm_syslog module. For more information, see Sending Syslog to a Remote Logger via UDP, TCP, or TLS.

Example 12. Sending Event Log in BSD Syslog format

This example configuration removes tab characters and newline sequences from the $Message field, converts the event record to BSD Syslog format, and forwards the event via UDP.

nxlog.conf
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
    Exec    $Message =~ s/(\t|\R)/ /g; to_syslog_bsd();
</Input>

<Output udp>
    Module  om_udp
    Host    10.10.1.1
    Port    514
</Output>
The to_syslog_bsd() procedure will use only a subset of the Event Log fields.
Output sample
<14>Jan  2 10:21:16 win7host Service_Control_Manager[448]: The Computer Browser service entered the running state.

Forwarding Windows Event Log in JSON format

To preserve all event log fields, the logs can be formatted as JSON. The xm_json module provides a to_json() procedure for this purpose. For more information about generating logs in JSON format, see JSON.

Example 13. Sending Event Log in JSON format

This example configuration converts the event record to JSON format and forwards the event via TCP.

nxlog.conf
<Extension _json>
    Module  xm_json  
</Extension>

<Input eventlog>
    Module  im_msvistalog
    Exec    to_json();
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.10.1
    Port    1514
</Output>
Output sample
{
  "EventTime": "2017-01-02 10:21:16",
  "Hostname": "win7host",
  "Keywords": -9187343239835812000,
  "EventType": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventID": 7036,
  "SourceName": "Service Control Manager",
  "ProviderGuid": "{525908D1-A6D5-5695-8E2E-26921D2011F3}",
  "Version": 0,
  "Task": 0,
  "OpcodeValue": 0,
  "RecordNumber": 2629,
  "ProcessID": 448,
  "ThreadID": 2872,
  "Channel": "System",
  "Message": "The Computer Browser service entered the running state.",
  "param1": "Computer Browser",
  "param2": "running",
  "EventReceivedTime": "2017-01-02 10:21:17",
  "SourceModuleName": "eventlog",
  "SourceModuleType": "im_msvistalog"
}

For compatibility with logging systems that require BSD Syslog, the JSON format can be used with a BSD Syslog header.

Example 14. Encapsulating JSON Event Log in BSD Syslog

This example configuration converts the event record to JSON, adds a BSD Syslog header, and forwards the event via UDP.

nxlog.conf
<Extension _json>
    Module  xm_json
</Extension>

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
    Exec    $Message = to_json(); to_syslog_bsd();
</Input>

<Output udp>
    Module  om_udp
    Host    192.168.2.1
    Port    514
</Output>
Output sample
<14>Jan  2 10:21:16 win7host Service_Control_Manager[448]: {"EventTime":"2017-01-02 10:21:16","Hostname":"win7host","Keywords":-9187343239835811840,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":7036,"SourceName":"Service Control Manager","ProviderGuid":"{525908D1-A6D5-5695-8E2E-26921D2011F3}","Version":0,"Task":0,"OpcodeValue":0,"RecordNumber":2629,"ProcessID":448,"ThreadID":2872,"Channel":"System","Message":"The Computer Browser service entered the running state.","param1":"Computer Browser","param2":"running","EventReceivedTime":"2017-01-02 10:21:17","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"}

Forwarding Windows Event Log in the Snare format

The Snare format is often used for Windows Event Log data. The xm_syslog module includes a to_syslog_snare() procedure which can generate the Snare format with a Syslog header. For more information about the Snare format, see Snare.

Example 15. Sending Event Log in Snare format

This example configuration removes tab characters and newline sequences from the $Message field, converts the event record to the Snare over Syslog format, and forwards the event via UDP.

nxlog.conf
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
    Exec    $Message =~ s/(\t|\R)/ /g; to_syslog_snare();
</Input>

<Output snare>
    Module  om_udp
    Host    192.168.1.1
    Port    514
</Output>
Output sample
<14>Jan  2 10:21:16 win7host MSWinEventLog	1	System	193	Mon Jan 02 10:21:16 2017	7036	Service Control Manager	N/A	N/A	Information	win7host	N/A		The Computer Browser service entered the running state.	2773

Processing EVTX files on Linux

EVTX files collected from Windows systems can be processed on Linux with NXLog by using an external script.

Example 16. Reading EVTX files using Python

This example configuration uses the im_python module to execute a script and formats the events to JSON. The Python script uses the python-evtx module to read the events from the EVTX file.

nxlog.conf
<Extension xml>
    Module        xm_xml
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input evtx>
    Module        im_python
    PythonCode    /opt/nxlog/etc/process_evtx.py
    <Exec>
        parse_windows_eventlog_xml($Message);
        delete($Message);
        to_json();
    </Exec>
</Input>
process_evtx.py
import Evtx.Evtx as evtx
import Evtx.Views as e_views

import nxlog

def read_data(module):
    nxlog.log_debug('Starting Processing EVTX')
    with evtx.Evtx('/tmp/security.evtx') as log:
        for record in log.records():
            event = module.logdata_new()
            event.set_field('Message', record.xml())
            event.post()
Output sample
{
  "EventReceivedTime": "2020-11-19T18:47:06.548281+01:00",
  "SourceModuleName": "evtx",
  "SourceModuleType": "im_python",
  "SourceName": "Microsoft-Windows-Security-Auditing",
  "ProviderGuid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
  "EventID": 5379,
  "Version": 0,
  "LevelValue": 0,
  "TaskValue": 13824,
  "OpcodeValue": 0,
  "Keywords": "0x8020000000000000",
  "EventTime": "2020-11-19T17:36:15.751457+01:00",
  "RecordNumber": 342859,
  "ActivityID": "{d9a2c36d-bc3e-0003-83c3-a2d93ebcd601}",
  "ExecutionProcessID": 872,
  "ExecutionThreadID": 956,
  "Channel": "Security",
  "Hostname": "PC1",
  "SubjectUserSid": "S-1-5-21-774634756-2905300177-2392840219-1003",
  "SubjectUserName": "John",
  "SubjectDomainName": "PC1",
  "SubjectLogonId": "0x00000000000ccd46",
  "TargetName": "MicrosoftOffice*",
  "Type": "0",
  "CountOfCredentialsReturned": "1",
  "ReadOperation": "%%8100",
  "ReturnCode": "0",
  "ProcessCreationTime": "2020-11-16 17:36:12.644609",
  "ClientProcessId": "12088",
  "EventType": "AUDIT_SUCCESS",
  "SeverityValue": 2,
  "Severity": "INFO"
}
Sometimes events from the <<>> contain values which need to be resolved using an external reference. Processing EVTX files using the python-evtx library may result in some events containing unresolved values.
Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here.

Last revision: 23 November 2020

© Copyright NXLog Ltd.

PRIVACY POLICY TERMS OF USE