NXLog search
Let's talk Start free
Let's talk Start free
1.03

Parse XML logs

Extensible Markup Language (XML) is an interoperable data and file format. It supports storing data in a structured format that is both human and machine-readable. NXLog Agent includes an XML log parser that significantly simplifies XML log processing.

Below, we provide examples of collecting and processing logs in the XML format.

Parse basic XML

NXLog Agent’s xm_xml module uses <Event> as the RootTag, does not parse attributes, and does not prefix fields with the root tag by default.

Example 1. Parsing a basic XML log structure

This configuration collects XML logs from a file with the im_file input module and parses records with the xm_xml module. It expects log records that comply with xm_xml's default settings.

nxlog.conf
<Extension xml_parser>
    Module    xm_xml
</Extension>

<Input xml_log>
    Module    im_file
    File      '/path/to/file'
    Exec      parse_xml(); (1)
</Input>
1 Calls the parse_xml() procedure to parse the record into structured data.

The following log sample adheres to the expected XML format. Although it is pretty printed for legibility, the configuration expects one record per line.

XML log sample
<Event>
  <Timestamp>2023-11-09T16:13:02.775963+01:00</Timestamp>
  <Hostname>SERVER-1</Hostname>
  <Severity>NOTICE</Severity>
  <Message>The server has started.</Message>
</Event>

When the NXLog Agent configuration above processes this log event, it adds the following fields to the log record in addition to the core fields.

Field Value

$Timestamp

2023-11-09T16:13:02.775963+01:00

$Hostname

SERVER-1

$Severity

NOTICE

$Message

The server has started.

Parse complex XML

You can customize the XML log parser settings to parse any XML structure. In addition, you can combine it with the xm_multiline module to parse multiline log records.

Example 2. Parsing pretty-printed XML

This configuration collects multiline logs from a file with the im_file input module and uses the xm_multiline and xm_xml modules to parse the records.

nxlog.conf
<Extension multiline> (1)
    Module             xm_multiline
    HeaderLine         /^<LogRecord>/
    EndLine            /^<\/LogRecord>/
</Extension>

<Extension xml_parser> 
    Module             xm_xml
    RootTag            LogRecord
    ParseAttributes    TRUE (2)
</Extension>

<Input xml_log>
    Module             im_file
    File               '/path/to/file'
    InputType          multiline (3)
    Exec               parse_xml(); (4)
</Input>
1 Defines regular expressions to detect the first and last line of a record.
2 Enables the ParseAttributes directive to create fields from element attributes.
3 Sets the InputType to the xm_multiline module instance.
4 Calls the parse_xml() procedure to parse the record into structured data.
XML log sample
<LogRecord>
  <Timestamp>2023-11-09T16:13:02.775963+01:00</Timestamp>
  <Hostname>SERVER-1</Hostname>
  <Event ID="4605" Category="System">
    <Severity>NOTICE</Severity>
    <Message>The server has started.</Message>
  </Event>
</LogRecord>

When the NXLog Agent configuration above processes this log event, it adds the following fields to the log record in addition to the core fields.

Field Value

$Timestamp

2023-11-09T16:13:02.775963+01:00

$Hostname

SERVER-1

$Event.ID

4605

$Event.Category

System

$Event.Severity

NOTICE

$Event.Message

The server has started.

Parse Windows Event Log XML

The XML log parser provides specialized procedures for parsing Windows events in XML format without requiring custom settings.

Example 3. Parsing Windows Event Log XML

This configuration collects Windows event logs from a file with the im_file input module and parses records with the xm_xml module. It expects log records in the Windows Event Log XML format.

nxlog.conf
<Extension xml> 
    Module            xm_xml
    PrefixWinEvent    TRUE (1)
</Extension>

<Input windows_event>
    Module            im_file
    File              'C:\logs\security_evt.xml'
    Exec              parse_windows_eventlog_xml(); (2)
</Input>
1 Enables the PrefixWinEvent directive to prefix EventData and UserData child elements with the parent field name.
2 Calls the parse_windows_eventlog_xml() procedure to parse the record into structured data.

The following is a Windows security event sample. Although it is pretty printed for legibility, the configuration expects one record per line.

Windows event in XML format
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-a5ba-3e3b0328c30d}" /> 
    <EventID>4672</EventID> 
    <Version>0</Version> 
    <Level>0</Level> 
    <Task>12548</Task> 
    <Opcode>0</Opcode> 
    <Keywords>0x8020000000000000</Keywords> 
    <TimeCreated SystemTime="2023-11-09T16:55:28.1200980Z" /> 
    <EventRecordID>1637767</EventRecordID> 
    <Correlation ActivityID="{485b6933-09f7-0002-ba69-5b48f709da01}" /> 
    <Execution ProcessID="1004" ThreadID="3152" /> 
    <Channel>Security</Channel> 
    <Computer>CLIENT-1</Computer> 
    <Security /> 
  </System>
  <EventData>
    <Data Name="SubjectUserSid">S-1-5-18</Data> 
    <Data Name="SubjectUserName">SYSTEM</Data> 
    <Data Name="SubjectDomainName">NT AUTHORITY</Data> 
    <Data Name="SubjectLogonId">0x3e7</Data> 
    <Data Name="PrivilegeList">SeAssignPrimaryTokenPrivilege SeTcbPrivilege SeSecurityPrivilege SeTakeOwnershipPrivilege SeLoadDriverPrivilege SeBackupPrivilege SeRestorePrivilege SeDebugPrivilege SeAuditPrivilege SeSystemEnvironmentPrivilege SeImpersonatePrivilege SeDelegateSessionUserImpersonatePrivilege</Data> 
  </EventData>
</Event>

When the NXLog Agent configuration above processes this log event, it adds the following fields to the log record in addition to the core fields. Notice the fields prefixed with EventData. If the PrefixWinEvent directive is not enabled, the configuration will still parse the fields but omit the prefix.

Field Value

$SourceName

Microsoft-Windows-Security-Auditing

$ProviderGuid

{54849625-5478-4994-a5ba-3e3b0328c30d}

$EventID

4672

$Version

0

$LevelValue

0

$TaskValue

12548

$OpcodeValue

0

$Keywords

0x8020000000000000

$EventTime

2023-11-09T17:55:28.120098+01:00

$RecordNumber

1637767

$ActivityID

{485b6933-09f7-0002-ba69-5b48f709da01}

$ExecutionProcessID

1004

$ExecutionThreadID

3152

$Channel

Security

$EventData.SubjectUserSid

S-1-5-18

$EventData.SubjectUserName

SYSTEM

$EventData.SubjectDomainName

NT AUTHORITY

$EventData.SubjectLogonId"

0x3e7

$EventData.PrivilegeList

SeAssignPrimaryTokenPrivilege SeTcbPrivilege SeSecurityPrivilege SeTakeOwnershipPrivilege SeLoadDriverPrivilege SeBackupPrivilege SeRestorePrivilege SeDebugPrivilege SeAuditPrivilege SeSystemEnvironmentPrivilege SeImpersonatePrivilege SeDelegateSessionUserImpersonatePrivilege

$EventType

AUDIT_SUCCESS

$SeverityValue

2

$Severity

INFO

© Copyright 2024 NXLog Ltd.

PRIVACY POLICY GENERAL TERMS OF BUSINESS