Event Log for Windows 2008/Vista/later (im_msvistalog)
This module can be used to collect Windows Event Log messages on Microsoft Windows platforms that support the newer Event Log API (also known as the Crimson Event Log subsystem), namely Windows 2008/Vista and later. See the official Microsoft documentation about Event Logs. The module supports reading all System, Application, and Custom events. It looks up the available channels and monitors events in each unless the Query and Channel directives are explicitly defined. Event logs can be collected from remote servers over MSRPC.
For Windows 2003 and earlier, use the im_mseventlog module because the new Windows Event Log API is only available in Windows Vista, Windows 2008, and later. |
Use the im_etw module to collect Analytic and Debug logs as the Windows Event Log subsystem, which im_msvistalog uses, does not support subscriptions to Debug or Analytic channels. |
NXLog Agent can alter a field’s line ending according to W3C recommendations. Any CR LF and any CR that is not followed by an LF will be translated to a single LF. |
In addition to the standard set of fields that are listed under the System section, event providers can define additional schema which enables logging additional data under the EventData section. The Security log makes use of this new feature and such additional fields can be seen in the following XML snippet:
<EventData>
<Data Name="SubjectUserSid">S-1-5-18</Data>
<Data Name="SubjectUserName">WIN-OUNNPISDHIG$</Data>
<Data Name="SubjectDomainName">WORKGROUP</Data>
<Data Name="SubjectLogonId">0x3e7</Data>
<Data Name="TargetUserSid">S-1-5-18</Data>
<Data Name="TargetUserName">SYSTEM</Data>
<Data Name="TargetDomainName">NT AUTHORITY</Data>
<Data Name="TargetLogonId">0x3e7</Data>
<Data Name="LogonType">5</Data>
<Data Name="LogonProcessName">Advapi</Data>
<Data Name="AuthenticationPackageName">Negotiate</Data>
<Data Name="WorkstationName" />
<Data Name="LogonGuid">{00000000-0000-0000-0000-000000000000}</Data>
<Data Name="TransmittedServices">-</Data>
<Data Name="LmPackageName">-</Data>
<Data Name="KeyLength">0</Data>
<Data Name="ProcessId">0x1dc</Data>
<Data Name="ProcessName">C:\Windows\System32\services.exe</Data>
<Data Name="IpAddress">-</Data>
<Data Name="IpPort">-</Data>
</EventData>
NXLog Agent can extract this data when fields are logged using this schema. The values will be available in the fields of the internal NXLog Agent log structure. This is especially useful because there is no need to write pattern-matching rules to extract this data from the message. These fields can be used in filtering rules, be written into SQL tables, or be used to trigger actions. The Exec directive can be used for filtering:
<Input in>
Module im_msvistalog
Exec if ($TargetUserName == 'SYSTEM') OR \
($EventType == 'VERBOSE') drop();
</Input>
Configuration
The im_msvistalog module accepts the following directives in addition to the xcommon module directives.
Optional directives
If this boolean directive is set to |
|||||||||||||||||||||||||||||||||||||
This boolean directive defines whether the module should store raw XML-formatted event data.
If set to |
|||||||||||||||||||||||||||||||||||||
This boolean directive defines whether the module should create the $Message field.
The default value is |
|||||||||||||||||||||||||||||||||||||
The name of the Channel to query. If not specified, the module will read from all sources defined in the registry. See the MSDN documentation about Event Selection. |
|||||||||||||||||||||||||||||||||||||
This optional directive can be used to specify a full path to a log file.
Log file types that can be used have the
The File directive can be specified multiple times to read from multiple files. This module finds files only when the module instance is started; any files added later will not be read until it is restarted. If the log file specified by this directive is updated with new event records while NXLog Agent is running (the file size or modification date attribute changes), the module detects the newly appended records on the fly without requiring the module instance to be restarted. Reading a Windows Event Log file directly is mostly useful for forensics purposes. The System log would be read directly with the following: File "C:\Windows\System32\winevt\Logs\System.evtx" You can use wildcards to specify file names and directories. Wildcards are not regular expressions but are patterns commonly used by Unix shells to expand filenames (also known as "globbing").
|
|||||||||||||||||||||||||||||||||||||
This optional directive specifies a language to use for rendering the events.
The language should be given as a hyphen-separated language/region code (for example, |
|||||||||||||||||||||||||||||||||||||
This directive specifies how frequently the module will check for new events, in seconds.
If this directive is not specified, the default is 1 second.
Fractional seconds may be specified ( |
|||||||||||||||||||||||||||||||||||||
This directive specifies the query for returning only specific Windows Event Log sources. See the MSDN documentation about Event Selection. Note that this directive requires a single-line parameter, so multi-line query XML should be specified using line continuation:
|
|||||||||||||||||||||||||||||||||||||
This directive is the same as the Query directive above, except it can be used as a block. Multi-line XML queries can be used without line continuation, and the XML Query can be copied directly from Event Viewer.
|
|||||||||||||||||||||||||||||||||||||
This optional directive can be used to specify the number of events the Windows Event Log API will pass to the module for processing.
Larger sizes may increase throughput. Note that there is a known issue in the Windows Event Log subsystem: when this value is higher than 31 it may fail to retrieve some events on busy systems, returning the error "EvtNext failed with error 1734: The array bounds are invalid."
For this reason, increasing this value is not recommended.
The default is |
|||||||||||||||||||||||||||||||||||||
This optional boolean directive instructs the module to only read logs that arrive after NXLog Agent is started.
This directive comes into effect if a saved position is not found, for example on the first start, or when the SavePos directive is The following matrix shows the outcome of this directive in conjunction with the SavePos directive:
|
|||||||||||||||||||||||||||||||||||||
This optional directive specifies the authentication method to use.
Available values are |
|||||||||||||||||||||||||||||||||||||
Domain of the user used for authentication when logging on the remote server to collect event logs. |
|||||||||||||||||||||||||||||||||||||
Password of the user used for authentication when logging on to the remote server to collect event logs. |
|||||||||||||||||||||||||||||||||||||
This optional directive specifies the name of the remote server to collect event logs. If not specified, the module will collect locally. |
|||||||||||||||||||||||||||||||||||||
Name of the user used for authentication when logging on to the remote server to collect event logs. |
|||||||||||||||||||||||||||||||||||||
This optional boolean directive specifies that GUID values should be resolved to their object names in the |
|||||||||||||||||||||||||||||||||||||
This optional boolean directive specifies that SID values should be resolved to user names in the |
|||||||||||||||||||||||||||||||||||||
If this boolean directive is set to |
|||||||||||||||||||||||||||||||||||||
This boolean directive specifies that im_msvistalog should ignore any invalid sources in the query.
The default is |
Fields
The following fields are used by im_msvistalog.
$raw_event
(type: string)-
A list of event fields in key-value pairs.
$AccountName
(type: string)-
The username associated with the event.
$AccountType
(type: string)-
The type of the account. Possible values are:
User
,Group
,Domain
,Alias
,Well Known Group
,Deleted Account
,Invalid
,Unknown
, andComputer
.
$ActivityID
(type: string)-
A globally unique identifier for the current activity, as stored in EvtSystemActivityID.
$Category
(type: string)-
The category name resolved from Task.
$Channel
(type: string)-
The Channel of the event source (for example,
Security
orApplication
).
$Domain
(type: string)-
The domain name of the user.
$ERROR_EVT_UNRESOLVED
(type: boolean)-
This field is set to TRUE if the event message cannot be resolved and the insertion strings are not present.
$EventID
(type: integer)-
The event ID (specific to the event source) from the EvtSystemEventID field.
$EventTime
(type: datetime)-
The EvtSystemTimeCreated field.
$EventType
(type: string)-
The type of the event, which is a string describing the severity. This is translated to its string representation from EvtSystemLevel. Possible values are:
CRITICAL
,ERROR
,AUDIT_FAILURE
,AUDIT_SUCCESS
,INFO
,WARNING
, andVERBOSE
.
$EventXML
(type: string)-
The raw event data in XML format. This field is available if the module’s CaptureEventXML directive is set to TRUE.
$ExecutionProcessID
(type: integer)-
The process identifier of the event producer as in EvtSystemProcessID.
$Hostname
(type: string)-
The EvtSystemComputer field.
$Keywords
(type: string)-
The value of the Keywords field from EvtSystemKeywords.
$Level
(type: string)-
The level of the event that was generated.
$LevelValue
(type: string)-
An integer indicating the level of the event that was generated.
$Message
(type: string)-
The message from the event.
$Opcode
(type: string)-
The Opcode string resolved from OpcodeValue.
$OpcodeValue
(type: integer)-
The Opcode number of the event as in EvtSystemOpcode.
$ProviderGuid
(type: string)-
The globally unique identifier of the event’s provider as stored in EvtSystemProviderGuid. This corresponds to the name of the provider in the $SourceName field.
$RecordNumber
(type: integer)-
The number of the event record.
$Severity
(type: string)-
The normalized severity name of the event. See $SeverityValue.
$SeverityValue
(type: integer)-
The normalized severity number of the event, mapped as follows.
Event Log Severity Normalized Severity 0/Audit Success
2/INFO
0/Audit Failure
4/ERROR
1/Critical
5/CRITICAL
2/Error
4/ERROR
3/Warning
3/WARNING
4/Information
2/INFO
5/Verbose
1/DEBUG
$SourceName
(type: string)-
The event source which produced the event, from the EvtSystemProviderName field.
$TaskValue
(type: integer)-
The task number from the EvtSystemTask field.
$ThreadID
(type: integer)-
The thread identifier of the event producer as in EvtSystemThreadID.
$UserID
(type: string)-
The Security Identifier (SID) which resolves to $AccountName, stored in EvtSystemUserID.
$Version
(type: integer)-
The Version number of the event as in EvtSystemVersion.
Fields by providers
For the fields used by the providers, see the fields by providers list.
Examples
Due to a bug or limitation of the Windows Event Log API, 23 or more
clauses in a query will fail with the following error message:
ERROR failed to subscribe to msvistalog events, the Query is invalid: This
operator is unsupported by this implementation of the filter.;
[error code: 15001]
|
This configuration collects events from Windows Event Log with the specified query. BSD Syslog headers are added and the messages are forwarded to a remote host via TCP.
<Extension syslog>
Module xm_syslog
</Extension>
<Input eventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id='0'>
<Select Path='Application'>*</Select>
<Select Path='Security'>*[System/Level<4]</Select>
<Select Path='System'>*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
<Output tcp>
Module om_tcp
Host 192.168.1.1:514
Exec to_syslog_bsd();
</Output>
<Route eventlog_to_tcp>
Path eventlog => tcp
</Route>
This configuration collects events from Windows Event Log and uses the extract_xml() function of the xm_xml module to retain the original EventData
and UserData
XML.
<Extension xml>
Module xm_xml
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
CaptureEventXML TRUE (1)
<QueryXML>
<QueryList>
<Query Id='0'>
<Select Path='Application'>*</Select>
<Select Path='Security'>*[System/Level<4]</Select>
<Select Path='System'>*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
$EventData = extract_xml("/Event/EventData", $EventXML);
if $EventData == ""
{
delete($EventData);
}
$UserData = extract_xml("/Event/UserData", $EventXML);
if $UserData == ""
{
delete($UserData);
}
delete($EventXML); (2)
to_json(); (3)
</Exec>
</Input>
1 | The CaptureEventXML directive is set to TRUE to store the raw XML-formatted event data in the $EventXML field. |
2 | Use the delete() procedure to remove unnecessary fields after processing. |
3 | The to_json() procedure of the xm_json module is called to convert the record to JSON. |
{
"EventTime": "2022-07-05T08:24:39.522394+02:00",
"Hostname": "WINAB-2JR3FR9RD",
"Keywords": "9259400833873739776",
"LevelValue": 4,
"EventType": "INFO",
"SeverityValue": 2,
"Severity": "INFO",
"EventID": 7036,
"SourceName": "Service Control Manager",
"ProviderGuid": "{555908D1-A6D7-4695-8E1E-26931D2012F4}",
"Version": 0,
"TaskValue": 0,
"OpcodeValue": 0,
"RecordNumber": 3356,
"ExecutionProcessID": 528,
"ExecutionThreadID": 1640,
"Channel": "System",
"Message": "The nxlog service entered the running state.",
"Level": "Information",
"param1": "nxlog",
"param2": "running",
"EventData.Binary": "6E0078006C006F0067002F0034000000",
"EventReceivedTime": "2022-07-05T08:24:40.725436+02:00",
"SourceModuleName": "eventlog",
"SourceModuleType": "im_msvistalog",
"EventData": "<EventData><Data Name=\"param1\">nxlog</Data><Data Name=\"param2\">running</Data><Binary>6E0078006C006F0067002F0034000000</Binary></EventData>"
}
If the NXLog Agent service is running as a custom user, errors may be logged in the log file such as:
or
This happens when the user account does not have permission to access the specified Windows Event Log channels such as the Security log. |