Windows Event Log
This section discusses the various details of Windows Event Logs.
About Windows Event Log
Windows Event Log captures the details of both system and application events. When such an event occurs, Windows records it in the event log. The event log is then used to find details about the event and can be helpful when troubleshooting problems. Beside their use for IT related purposes, Windows Event Logs are also used to satisfy compliance mandates.
Unlike other event logs, such as the UNIX Syslog, Windows Event Log is not stored as a plain text file, but in a proprietary binary format. It is not possible to view Windows Event Log in a text editor, nor is it possible to send it as a Syslog event while retaining its original format. However, the raw event data can be translated into XML using the Windows Event Log API and forwarded in that format.
The EVTX file format
Windows stores Windows Event Log files in the EVTX file format since the
release of Windows
Vista and Windows Server 2008. Prior to that, event log files were
stored in the EVT file format. Both are proprietary formats readable by
the Microsoft Management Console (MMC) snap-in eventvwr.msc
.
The EVTX format includes many new features and enhancements: a number of new event properties, the use of channels to publish events, a new Event Viewer, a rewritten Windows Event Log service, and support for the Extensible Markup Language (XML) format. From a log processing perspective, the added support for XML is the most important addition, as it provides the possibility to share or further process the event data in a structured format.
For the built in channels, Windows automatically saves the corresponding EVTX
file into the C:\Windows\System32\winevt\Logs\
directory. Events can also be
saved manually from the Event Viewer MMC snap-in, in four different formats:
EVTX, XML, TXT, and CSV.
NXLog can directly read EVTX and EVT files using the im_msvistalog
File directive. In addition, the
CaptureEventXML directive of the same
module can be used to store and send raw XML-formatted event data in the
$EventXML
field.
Viewing the Windows Event Log
The Windows Event Log can be viewed in the Event Viewer MMC snap-in included in Windows. Windows Event Logs are stored in a binary source data format, which is the "source" or "on-disk" format. It does not include the full message, only the event properties. When an event is rendered, property values are inserted into the localized message template stored elsewhere on disk.
The Event Viewer includes three views for displaying the data for a selected event. These are shown on the preview pane or in the Event Properties window when an event is opened.
-
The general view is shown by default. It includes the full message rendered from template and the "System" set of key/value pairs.
-
The Friendly View is available on the Details tab. It shows a hierachical view of the System properties and additional EventData properties defined by the event provider. It does not show a rendered message.
-
The XML View can be selected under the Details tab. It shows the event properties in XML format. It does not show a rendered message.
A Windows Event Log event in XML format<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" /> <EventID>4624</EventID> [...] <Channel>Security</Channel> <Computer>USER-WORKSTATION</Computer> <Security /> </System> <EventData> <Data Name="SubjectUserSid">S-1-5-18</Data> [...] </EventData> </Event>
Events can be accessed through the Event Log API (see Windows Event Log Functions on Microsoft Docs). In particular:
-
EvtQuery() fetches events from a given channel or log file that match a given query—see Querying for Events.
-
EvtFormatMessage() generates a message string for an event using the event properties and the localized message template—see Formatting Event Messages.
Event channels
The EVTX format introduces event channels. A channel is a stream of events that collects events from a publisher and writes them to an event log file.
Channels are organized into two groups:
-
The Windows Logs group contains a set of exactly five channels, which are used for Windows system events.
-
The Applications and Services Logs group contains channels created for individual applications or components. These channels are further organized in a folder hierarchy.
There are two channel types indicating how the events are handled:
-
Serviced channels offer relatively low volume, reliable delivery of events. Events in these channels may be forwarded to another system, and these channels may be subscribed to.
-
Direct channels are for high-performance collection of events. It is not possible to subscribe to a a direct channel. By default, these channels are disabled. To see these channels in the Event Viewer, check Show Analytic and Debug Logs in the View menu. To enable logging for one of these channels, select the channel, open the Action menu, click Properties, and check Enable logging on the General tab.
Each of the above is subdivided into two more channel types according to the the intended audience for the events collected by that channel:
-
Administrative channels collects events for end users, administrators, and support. This is a serviced channel type.
-
Operational channels collect events used for diagnosing problems. This is a serviced channel type.
-
Analytic channels are for events that describe program operation. These channels often collect a high volume of events. This is a direct channel type.
-
Debug channels are intended to be used by developers only. This is a direct channel type.
Channel Groups | Channels | Channel Type |
---|---|---|
Windows Logs |
Application |
Administrative (serviced) |
Security |
Administrative (serviced) |
|
Setup |
Operational (serviced) |
|
System |
Administrative (serviced) |
|
Forwarded Events |
Operational (serviced) |
|
Applications and Services Logs |
DHCP-Server/Admin |
Administrative (serviced) |
DHCP-Server/AuditLogs |
Analytic (direct) |
|
DHCP-Server/DebugLogs |
Debug (direct) |
|
(And many more publisher-defined channels) |
The im_msvistalog module can be configured to collect events from a specific channel with the Channel directive.
For more information about event channels, see these two pages on Microsoft Docs: Event Logs and Event Logs and Channels in Windows Event Log.
Providers
Event log providers write events to event logs. An event log provider can be a service, driver, or program that runs on the computer and has the necessary instrumentation to write to the event log.
Event providers are categorized into four main types.
-
Manage Object Format (MOF) providers (also referred to as "classic")
-
Windows Software Trace Preprocessor (WPP) providers
-
Manifest-based providers
-
TraceLogging providers
For more information on providers, see the Providers section in the Microsoft Windows documentation.
Collecting Event Log Data
This section lists and discusses the NXLog modules that can be used to collect Windows Event Log data.
NXLog Modules for Windows Event Log
NXLog provides the following modules for capturing Windows Event Log data.
-
The im_msvistalog module is available on Windows only, and captures event log data from Windows 2008/Vista and later. It can be configured to collect event log data from the local system or from a remote system via MSRPC (MSRPC is supported by NXLog Enterprise Edition only). See Local collection with im_msvistalog and Remote collection with im_msvistalog.
-
The im_wseventing module is available on both Linux and Windows (NXLog Enterprise Edition only). With it, event log data can be received from remote Windows systems using Windows Event Forwarding. This is the recommended module for most cases where remote capturing is required, because it is not necessary to specify each host that Event Log data will be captured from. See Remote collection with im_wseventing.
-
The im_mseventlog module is available on Windows only, and captures event log data locally from Windows XP, Windows 2000, and Windows 2003. See Local collection with im_mseventlog.
Local collection with im_msvistalog
The im_msvistalog module can capture Event Log data from the local system running Windows 2008/Vista or later.
Because the Windows Event Log subsystem does not support subscriptions to the Debug and Analytic channels, these types of events can not be collected with the im_msvistalog module. |
In this example, NXLog reads all events from the local Windows Event Log. The data is converted to JSON format and written to a local file.
<Extension _json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
</Input>
<Output file>
Module om_file
File 'C:\test\sysmon.json'
Exec to_json();
</Output>
For information about filtering events, particularly when using im_msvistalog, see Filtering events.
Remote collection with im_msvistalog
NXLog Enterprise Edition can be configured with the im_msvistalog module for collection of events generated on remote Windows systems. In this mode, it is not necessary to run an NXLog agent on the Windows systems. Instead, MSRPC is used to receive the events.
Because the Windows Event Log subsystem does not support subscriptions to the Debug and Analytic channels, these types of events can not be collected with the im_msvistalog module. |
In this example configuration, the im_msvistalog
module is used to get events from a remote server named mywindowsbox
using MSRPC.
To replicate this example in your environment, modify the
RemoteServer
, RemoteUser
, RemoteDomain
, and RemotePassword
to
reflect the access credentials for the target machine.
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id='1'>
<Select Path='Application'>*</Select>
<Select Path='Security'>*[System/Level=4]</Select>
<Select Path='System'>*</Select>
</Query>
</QueryList>
</QueryXML>
RemoteServer mywindowsbox
RemoteUser Administrator
RemoteDomain Workgroup
RemotePassword secret
</Input>
Local collection with im_mseventlog
The im_mseventlog module can capture Event Log data from Windows XP, Windows 2000, and Windows 2003.
The module looks up the available Event Log sources stored under the
registry key SYSTEM\CurrentControlSet\Services\Eventlog
, and polls
logs from each of these or only the sources defined with the Sources
directive of the NXLog configuration.
This example shows the most basic configuration of the
im_mseventlog module. This configuration forwards
all Event Log sources listed in the Windows registry over the network
to a remote NXLog instance at the IP address 192.168.1.1
.
<Input eventlog>
Module im_mseventlog
</Input>
<Output tcp>
Module om_tcp
Host 192.168.1.1
Port 514
</Output>
Remote collection with im_wseventing
NXLog Enterprise Edition can use the im_wseventing module to receive Windows Event Log data from remote machines over WEF (Windows Event Forwarding). It works on both Windows and Linux hosts.
This configuration receives data from all source computers, by listening on port 5985 for connections from all sources. It also shows a configured HTTPS certificate, used to secure the transfer of Event Log data.
<Input in>
Module im_wseventing
ListenAddr 0.0.0.0
Port 5985
Address https://linux.corp.domain.com:5985/wsman
HTTPSCertFile %CERTDIR%/server-cert.pem
HTTPSCertKeyFile %CERTDIR%/server-key.pem
HTTPSCAFile %CERTDIR%/ca.pem
<QueryXML>
<QueryList>
<Query Id="0" Path="Application">
<Select Path="Application">*</Select>
<Select Path="Microsoft-Windows-Winsock-AFD/Operational">*</Select>
<Select Path="Microsoft-Windows-Wired-AutoConfig/Operational">
*
</Select>
<Select Path="Microsoft-Windows-Wordpad/Admin">*</Select>
<Select Path="Windows PowerShell">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
A query for specific hosts can be set by adding an additional
QueryXML block with a <Computer>
tag. This tag contains a pattern that NXLog matches against
the name of the connecting Windows client. Computer names not matching
the pattern will use the default QueryXML block (containing no
<Computer>
tag).
The following QueryXML block, if
added to the above configuration, would provide an alternate query for
computer names matching the pattern foo*
.
<QueryXML>
<Computer>foo*</Computer>
<QueryList>
<Query Id="0" Path="Application">
<Select Path="Application">*</Select>
</Query>
</QueryList>
</QueryXML>
Filtering events
Systems and services on Windows can generate a large volume of logs, and it is often necessary to collect only a certain portion of those events. There are several ways to implement filtering of events from the Windows Event Log when using the im_msvistalog module.
-
A specific channel can be specified with the Channel directive to collect all the events written to a single channel.
-
An XPath query can be given with the QueryXML block (or Query directive). The specified query is then used to subscribe to events. An XPath query can be used to subscribe to multiple channels and/or limit events by various attributes. However, XPath queries have a maximum length, limiting the possibilities for detailed event subscriptions. See XPath filtering below.
-
A log file can be read by setting the File directive, in which case im_msvistalog will read all events from the file (for example,
Security.evtx
). This is intended primarily for forensics purposes, such as with nxlog-processor. -
After being read from the source, events can be discarded by matching events in an Exec block and discarding them selectively with the drop() procedure.
Subscribing to a restricted set of events with an XPath query can offer a performance advantage because the events are never received by NXLog. However, XPath queries have a maximum length and limited filtering capabilities, so in some cases it is necessary to combine an XPath query with Exec block filtering in an im_msvistalog configuration. For examples, see examples in Event IDs to Monitor.
XPath filtering
XPath queries can be used to subscribe to events matching certain criteria, both in the Event Viewer and with the im_msvistalog QueryXML directive. Windows Event Log supports a subset of XPath 1.0. For more information, see Consuming Events on Microsoft Docs.
The Event Viewer offers the most practical way to write and test query strings. An XPath query can be generated and/or tested by filtering the current log or creating a custom view.
-
In the Event Viewer, click an event channel to open it, then right-click the channel and choose Filter Current Log from the context menu. Or, click Create Custom View in the context menu. Either way, a dialog box will open and options for basic filtering will be shown in the Filter tab.
-
Specify the desired criteria. The corresponding XPath query on the XML tab will be updated automatically.
-
To view the query string, switch to the XML tab. This string can be copied into the im_msvistalog QueryXML directive.
-
If required, advanced filtering can be done by selecting the Edit query manually checkbox and editing the query. The query can then be tested to be sure it matches the correct events and finally copied to the NXLog configuration with the QueryXML block.
Figure 1. A Custom View Querying the Application Channel for Events With ID 1008
Sometimes it is helpful to use a query with sources that may not be available. In this case, set the TolerateQueryErrors directive to TRUE to ensure that the module will continue to collect logs.
Here, NXLog queries the local Windows Event Log for operational events only.
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
This query collects System channel events with levels below 4 (Critical, Error, and Warning).
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path='System'>*[System/Level<4]</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
Exec block filtering
NXLog’s built-in filtering capabilities can also be used to filter events, by matching events and using the drop() procedure. Events can be matched against any of the im_msvistalog fields.
This example discards all Sysmon network connection events (event ID 3)
regarding HTTP network connections to a particular server and port, and all
process creation and termination events (event IDs 1 and 5) for conhost.exe
.
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if ($EventID in (1, 5) and
$Image == "C:\\Windows\\System32\\conhost.exe") or
($EventID == 3 and
$DestinationPort == 80 and
$DestinationIp == 10.0.0.1)
drop();
</Exec>
</Input>
Event IDs to monitor
When it comes to Windows log collection, one of the most challenging tasks of a system administrator is deciding which event IDs to monitor. Due to the large number of event IDs in use, this can be daunting at first sight. Therefore, this section aims to provide guidance about selecting event IDs to monitor, with some example configurations.
Event IDs are unique per source but are not globally unique. The same event ID may be used by different sources to identify unrelated occurrences. |
Finding the right event IDs
An excellent general source to start with is the Windows 10 and Windows Server 2016 security auditing and monitoring reference. It provides detailed descriptions about event IDs used for security audit policies. There are additional resources to find events to monitor, see below:
-
The Microsoft Events and Errors page on Microsoft Docs provides a directory of events grouped by area. Start by navigating through the areas listed in the Available Documentation section.
-
Palantir has published a Windows Event Forwarding Guidance repository, which contains a comprehensive WEF Event Mappings table with categorized event IDs and details.
-
The NSA Spotting the Adversary with Windows Event Log Monitoring paper provides event IDs for security monitoring. See the example configuration here.
-
The JPCERT/CC Detecting Lateral Movements Tool Analysis resource provides a collection of event codes that are observed to indicate lateral movements. See the example configuration here.
-
See the NXLog User Guide on Active Directory Domain Services for a list and configuration sample of security event IDs relevant to Active Directory.
The table below displays a small sample of important events to monitor in the Windows Server Security Log for a local server. See the Security-focused Event IDs to Monitor section for the configuration file holding these event IDs.
Event ID | Description |
---|---|
1102 |
The audit log was cleared. |
4719 |
System audit policy was changed. |
4704 |
A user right was assigned. |
4717 |
System security access was granted to an account. |
4738 |
A user account was changed. |
4798 |
A user’s local group membership was enumerated. |
4705 |
A user right was removed. |
4674 |
An operation was attempted on a privileged object. |
4732 |
A member was added to a security-enabled local group. |
4697 |
A service was installed in the system. |
4625 |
An account failed to log on. |
4648 |
A logon was attempted using explicit credentials. |
4723 |
An attempt was made to change an account’s password. |
4946 |
A change has been made to Windows Firewall exception list. A rule was added. |
4950 |
A Windows Firewall setting has changed. |
6416 |
A new external device was recognized by the system. |
6424 |
The installation of this device was allowed, after having previously been forbidden by policy. |
Example monitoring configurations
Once a set of event IDs has been selected for monitoring, the im_msvistalog module can be configured.
The example configurations in this section are likely to require further modifications to suit each individual deployment. |
Due to a bug or limitation of the Windows Event Log API, 23 or more
clauses in a query will result in a failure with the following error
message:
ERROR failed to subscribe to msvistalog events, the Query is invalid:
This operator is unsupported by this implementation of the filter.;
[error code: 15001]
|
Event IDs are globally applied to all providers of a given XPath expression so events that match these IDs will be collected. You should tweak your chosen dashboard or alerting system to ensure that the right Event IDs and its subsequent providers are appropriately associated. |
This configuration provides a basic example of Windows Security
events to
monitor. Since only a small number of IDs are presented, this configuration
explicitly provides the actual event IDs to be collected.
<Input MonitorWindowsSecurityEvents>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Security">*[System[(Level=1 or Level=2 or Level=3 or Level=4 or Level=0) and (EventID=1102 or EventID=4719 or EventID=4704 or EventID=4717 or EventID=4738 or EventID=4798 or EventID=4705 or EventID=4674 or EventID=4697 or EventID=4648 or EventID=4723 or EventID=4946 or EventID=4950 or EventID=6416 or EventID=6424 or EventID=4732)]]</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
This extended configuration provides a much wider scope of log collection.
Note that this approach for specifying the event IDs requires defining the
event IDs based on groups of events first. The QueryXML paths are added in
the QueryXML
block in bulk. Then the Exec
block will filter for the defined
event IDs, but only within the paths specified. It also drops event IDs that
are not defined.
# define Account Usage Events
define AccountUsage 4740, 4648, 4781, 4733, 1518, 4776, 5376, 5377, \
4625, 300, 4634, 4672, 4720, 4722, 4782, 4793, \
4731, 4735, 4766, 4765, 4624, 1511, 4726, 4725, \
4767, 4728, 4732, 4756, 4704
# define Application Crash Events
define AppCrashes 1000, 1002, 1001
# define Application Whitelisting Events
define AppWhitelisting 8023, 8020, 8002, 8003, 8004, 8006, 8007, 4688, \
4689, 8005, 865, 866, 867, 868, 882
# define Boot Events
define BootEvents 13, 12
# define Certificate Services Events
define CertServices 95, 4886, 4890, 4874, 4873, 4870, 4887, 4885, \
4899, 4896, 1006, 1004, 1007, 1003, 1001, 1002
# define Clearing Event Logs Events
define ClearingLogs 1100, 104, 1102
# define DNS and Directory Services Events
define DNSDirectoryServ 5137, 5141, 5136, 5139, 5138, 3008, 3020
# define External Media Detection events
define ExtMedia 400, 410
# define Group Policy Error Events
define GroupPolicyError 112, 1001, 1125, 1126, 1127, 1129
# define Kernel Driver Signing Events
define KernelDriver 3001, 3002, 3003, 3004, 3010, 3023, 5038, \
6281, 219
# define Microsoft Cryptography API Events
define MSFTCryptoAPI 11, 70, 90
# define Mobile Device Activities
define MobileDeviceEvents 10000, 10001
# define Network Host Activities
define NetworkHost 4714, 4713, 4769, 6273, 6275, 6274, 6272, \
6278, 6277, 6279, 6276, 6280, 5140, 5145, \
5142, 5144, 4706, 1024, 4897, 4719, 4716, \
4779, 4778, 5632
# define PassTheHash Detection Events
define PassTheHash 4624, 4625
# define PowerShell Activities
define PowerShell 800, 169, 4103, 4104, 4105, 4106
# define Printing Services Events
define PrintingServices 307
# define Logon Events
define LogonEvents 4624, 4634
# define Software Service Installation Events
define Installation 903, 904, 6, 1022, 1033, 7045, 907, 908, 7000, \
800, 2, 905, 906, 19
# define System Integrity Events
define SystemIntegrity 4657, 1, 4616
# define System or Service Failure Events
define SystemServiceFail 7022, 7023, 7024, 7026, 7031, 7032, 7034
# define Task Scheduler Activities
define TaskScheduler 106, 141, 142, 200
# define Windows Defender Activities
define WinDefender 1008, 1006, 1116, 1010, 2003, 2001, 1009, 1118, \
1119, 1007, 1117, 3002, 2004, 1005, 5008
# define Windows Firewall Events
define WinFirewall 2009, 2004, 2005, 2006, 2033
# define Windows Update Error Events
define WinUpdateError 1009, 20, 24, 25, 31, 34, 35
<Input ExtendedWindowsToCollect>
Module im_msvistalog
TolerateQueryErrors TRUE
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Application">*</Select>
<Select Path="Security">*</Select>
<Select Path="System">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Assistant">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Troubleshooter">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Inventory">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Telemetry">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Steps-Recorder">*</Select>
<Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>
<Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>
<Select Path="Microsoft-Windows-AppLocker/Packaged app-Deployment">*</Select>
<Select Path="Microsoft-Windows-AppLocker/Packaged app-Execution">*</Select>
<Select Path="Microsoft-Windows-CAPI2/Operational">*</Select>
<Select Path="Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational">*</Select>
<Select Path="Microsoft-Windows-DNS-Client/Operational">*</Select>
<Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select>
<Select Path="Microsoft-Windows-Kernel-PnP/Configuration">*</Select>
<Select Path="Microsoft-Windows-NetworkProfile/Operational">*</Select>
<Select Path="Microsoft-Windows-NTLM/Operational">*</Select>
<Select Path="Microsoft-Windows-PowerShell/Admin">*</Select>
<Select Path="Microsoft-Windows-PowerShell/Operational">*</Select>
<Select Path="Microsoft-Windows-PrintService/Admin">*</Select>
<Select Path="Microsoft-Windows-PrintService/Operational">*</Select>
<Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
<Select Path="Microsoft-Windows-TerminalServices-RDPClient/Operational">*</Select>
<Select Path="Microsoft-Windows-User Profile Service/Operational">*</Select>
<Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select>
<Select Path="Microsoft-Windows-Windows Defender/WHC">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/Firewall">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallDiagnostics">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose">*</Select>
<Select Path="Network Isolation Operational">*</Select>
<Select Path="Microsoft-Windows-WindowsUpdateClient/Operational">*</Select>
<Select Path="Windows PowerShell">*</Select>
<Select Path="Microsoft-Windows-CodeIntegrity/Operational">*[System[Provider[@Name='Microsoft-Windows-CodeIntegrity']]]</Select>
<Select Path="Microsoft-Windows-LSA/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if ($EventID NOT IN (%AccountUsage%)) and
($EventID NOT IN (%AppCrashes%)) and
($EventID NOT IN (%AppWhitelisting%)) and
($EventID NOT IN (%BootEvents%)) and
($EventID NOT IN (%CertServices%)) and
($EventID NOT IN (%ClearingLogs%)) and
($EventID NOT IN (%DNSDirectoryServ%)) and
($EventID NOT IN (%ExtMedia%)) and
($EventID NOT IN (%GroupPolicyError%)) and
($EventID NOT IN (%KernelDriver%)) and
($EventID NOT IN (%MSFTCryptoAPI%)) and
($EventID NOT IN (%MobileDeviceEvents%)) and
($EventID NOT IN (%NetworkHost%)) and
($EventID NOT IN (%PassTheHash%)) and
($EventID NOT IN (%PowerShell%)) and
($EventID NOT IN (%PrintingServices%)) and
($EventID NOT IN (%LogonEvents%)) and
($EventID NOT IN (%Installation%)) and
($EventID NOT IN (%SystemIntegrity%)) and
($EventID NOT IN (%SystemServiceFail%)) and
($EventID NOT IN (%TaskScheduler%)) and
($EventID NOT IN (%WinDefender%)) and
($EventID NOT IN (%WinFirewall%)) and
($EventID NOT IN (%WinUpdateError%)) drop();
</Exec>
</Input>
This configuration, similar to the extended configuration above, lists event IDs associated with the detection of malicious lateral movements. It is based on the security research conducted by the CERT (Computer Emergency Response Team) cybersecurity researchers on Detecting Lateral Movement through Tracking Event Logs.
# define Security Events
define SecurityEvents 4624, 4634, 4648, 4656, 4658, 4660, 4663, 4672, \
4673, 4688, 4689, 4698, 4720, 4768, 4769, 4946, \
5140, 5142, 5144, 5145, 5154, 5156, 5447, 8222
# define Sysmon Events
define SysmonEvents 1, 2, 5, 8, 9
# define Application Management event
define ApplicationMgmt 104
# define Windows Remote Management Events
define WRMEvents 80, 132, 143, 166, 81
# define Task Scheduler - Operational Events
define TaskSchedEvents 106, 129, 200, 201
# define Local Session Manager - Operational Events
define LocalSessionMgrEvents 21, 24
#define BitsClient Events
define BitsClientsEvents 60
<Input LateralMovementEvents>
Module im_msvistalog
TolerateQueryErrors TRUE
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Security">*</Select>
<Select Path="System">*</Select>
<Select Path="Microsoft-Windows-Bits-Client/Operational">*</Select>
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
<Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
<Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Admin">*</Select>
<Select Path="Microsoft-Windows-TerminalServices-LocalSessionManager/Operational">*</Select>
<Select Path="Microsoft-Windows-WinRM/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if ($EventID NOT IN (%SecurityEvents%)) and
($EventID NOT IN (%SysmonEvents%)) and
($EventID NOT IN (%WRMEvents%)) and
($EventID NOT IN (%TaskSchedEvents%)) and
($EventID NOT IN (%ApplicationMgmt%)) and
($EventID NOT IN (%LocalSessionMgrEvents%)) and
($EventID NOT IN (%BitsClientsEvents%)) drop();
</Exec>
</Input>
Forwarding Event Log Data
After collecting the Event Log data from a Windows system with NXLog, it may need to be sent to another host. This section provides details and examples for configuring this.
Event descriptions in Event Log data may contain tabs and newlines, but these are not supported by some formats like BSD Syslog. In this case, a regular expression can be used to remove them.
This input instance is configured to modify the $Message
field (the
event description) by replacing all tab characters and newline
sequences with spaces.
<Input in>
Module im_mseventlog
Exec $Message =~ s/(\t|\R)/ /g;
</Input>
Forwarding Event Log in BSD Syslog format
Event Log data is commonly sent in the BSD Syslog format. This can be generated with the to_syslog_bsd() procedure provided by the xm_syslog module. For more information, see Sending Syslog to a Remote Logger via UDP, TCP, or TLS.
This example configuration removes tab characters and newline
sequences from the $Message
field, converts the event record to BSD
Syslog format, and forwards the event via UDP.
<Extension _syslog>
Module xm_syslog
</Extension>
<Input eventlog>
Module im_msvistalog
Exec $Message =~ s/(\t|\R)/ /g; to_syslog_bsd();
</Input>
<Output udp>
Module om_udp
Host 10.10.1.1
Port 514
</Output>
The to_syslog_bsd() procedure will use only a subset of the Event Log fields. |
<14>Jan 2 10:21:16 win7host Service_Control_Manager[448]: The Computer Browser service entered the running state.
Forwarding Windows Event Log in JSON format
To preserve all event log fields, the logs can be formatted as JSON. The xm_json module provides a to_json() procedure for this purpose. For more information about generating logs in JSON format, see JSON.
This example configuration converts the event record to JSON format and forwards the event via TCP.
<Extension _json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
Exec to_json();
</Input>
<Output tcp>
Module om_tcp
Host 192.168.10.1
Port 1514
</Output>
{
"EventTime": "2017-01-02 10:21:16",
"Hostname": "win7host",
"Keywords": -9187343239835812000,
"EventType": "INFO",
"SeverityValue": 2,
"Severity": "INFO",
"EventID": 7036,
"SourceName": "Service Control Manager",
"ProviderGuid": "{525908D1-A6D5-5695-8E2E-26921D2011F3}",
"Version": 0,
"Task": 0,
"OpcodeValue": 0,
"RecordNumber": 2629,
"ProcessID": 448,
"ThreadID": 2872,
"Channel": "System",
"Message": "The Computer Browser service entered the running state.",
"param1": "Computer Browser",
"param2": "running",
"EventReceivedTime": "2017-01-02 10:21:17",
"SourceModuleName": "eventlog",
"SourceModuleType": "im_msvistalog"
}
For compatibility with logging systems that require BSD Syslog, the JSON format can be used with a BSD Syslog header.
This example configuration converts the event record to JSON, adds a BSD Syslog header, and forwards the event via UDP.
<Extension _json>
Module xm_json
</Extension>
<Extension _syslog>
Module xm_syslog
</Extension>
<Input eventlog>
Module im_msvistalog
Exec $Message = to_json(); to_syslog_bsd();
</Input>
<Output udp>
Module om_udp
Host 192.168.2.1
Port 514
</Output>
<14>Jan 2 10:21:16 win7host Service_Control_Manager[448]: {"EventTime":"2017-01-02 10:21:16","Hostname":"win7host","Keywords":-9187343239835811840,"EventType":"INFO","SeverityValue":2,"Severity":"INFO","EventID":7036,"SourceName":"Service Control Manager","ProviderGuid":"{525908D1-A6D5-5695-8E2E-26921D2011F3}","Version":0,"Task":0,"OpcodeValue":0,"RecordNumber":2629,"ProcessID":448,"ThreadID":2872,"Channel":"System","Message":"The Computer Browser service entered the running state.","param1":"Computer Browser","param2":"running","EventReceivedTime":"2017-01-02 10:21:17","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"}
Forwarding Windows Event Log in the Snare format
The Snare format is often used for Windows Event Log data. The xm_syslog module includes a to_syslog_snare() procedure which can generate the Snare format with a Syslog header. For more information about the Snare format, see Snare.
This example configuration removes tab characters and newline
sequences from the $Message
field, converts the event record to the
Snare over Syslog format, and forwards the event via UDP.
<Extension _syslog>
Module xm_syslog
</Extension>
<Input eventlog>
Module im_msvistalog
Exec $Message =~ s/(\t|\R)/ /g; to_syslog_snare();
</Input>
<Output snare>
Module om_udp
Host 192.168.1.1
Port 514
</Output>
<14>Jan 2 10:21:16 win7host MSWinEventLog 1 System 193 Mon Jan 02 10:21:16 2017 7036 Service Control Manager N/A N/A Information win7host N/A The Computer Browser service entered the running state. 2773
Processing EVTX files on Linux
EVTX files collected from Windows systems can be processed on Linux with NXLog by using an external script.
This example configuration uses the im_python module to execute a script and formats the events to JSON. The Python script uses the python-evtx module to read the events from the EVTX file.
<Extension xml>
Module xm_xml
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input evtx>
Module im_python
PythonCode /opt/nxlog/etc/process_evtx.py
<Exec>
parse_windows_eventlog_xml($Message);
delete($Message);
to_json();
</Exec>
</Input>
import Evtx.Evtx as evtx
import Evtx.Views as e_views
import nxlog
def read_data(module):
nxlog.log_debug('Starting Processing EVTX')
with evtx.Evtx('/tmp/security.evtx') as log:
for record in log.records():
event = module.logdata_new()
event.set_field('Message', record.xml())
event.post()
{
"EventReceivedTime": "2020-11-19T18:47:06.548281+01:00",
"SourceModuleName": "evtx",
"SourceModuleType": "im_python",
"SourceName": "Microsoft-Windows-Security-Auditing",
"ProviderGuid": "{54849625-5478-4994-a5ba-3e3b0328c30d}",
"EventID": 5379,
"Version": 0,
"LevelValue": 0,
"TaskValue": 13824,
"OpcodeValue": 0,
"Keywords": "0x8020000000000000",
"EventTime": "2020-11-19T17:36:15.751457+01:00",
"RecordNumber": 342859,
"ActivityID": "{d9a2c36d-bc3e-0003-83c3-a2d93ebcd601}",
"ExecutionProcessID": 872,
"ExecutionThreadID": 956,
"Channel": "Security",
"Hostname": "PC1",
"SubjectUserSid": "S-1-5-21-774634756-2905300177-2392840219-1003",
"SubjectUserName": "John",
"SubjectDomainName": "PC1",
"SubjectLogonId": "0x00000000000ccd46",
"TargetName": "MicrosoftOffice*",
"Type": "0",
"CountOfCredentialsReturned": "1",
"ReadOperation": "%%8100",
"ReturnCode": "0",
"ProcessCreationTime": "2020-11-16 17:36:12.644609",
"ClientProcessId": "12088",
"EventType": "AUDIT_SUCCESS",
"SeverityValue": 2,
"Severity": "INFO"
}
Sometimes events from the <<>> contain values which need to be resolved using an external reference. Processing EVTX files using the python-evtx library may result in some events containing unresolved values. |