NXLog search
Let's talk Start free
Let's talk Start free

NXLog Legacy Documentation

6.04

Syslog (xm_syslog)

This module provides support for the legacy BSD Syslog protocol as defined in RFC 3164 and the current IETF standard defined by RFCs 5424-5426. This is achieved by exporting functions and procedures usable from the NXLog language. The transport is handled by the respective input and output modules (such as im_udp), this module only provides a parser and helper functions to create syslog messages and handle facility and severity values.

To examine the supported platforms, see the list of installer packages in the Available Modules chapter.

The older but still widespread BSD Syslog standard defines both the format and the transport protocol in RFC 3164. The transport protocol is UDP, but to provide reliability and security, this line-based format is also commonly transferred over TCP and SSL. There is a newer standard defined in RFC 5424, also known as the IETF Syslog format, which obsoletes the BSD Syslog format. This format overcomes most of the limitations of BSD Syslog and allows multiline messages and proper timestamps. The transport method is defined in RFC 5426 for UDP and RFC 5425 for TLS/SSL.

Because the IETF Syslog format supports multiline messages, RFC 5425 defines a special format to encapsulate these by prefixing the payload size in ASCII to the IETF Syslog message. Messages transferred in UDP packets are self-contained and do not need this additional framing. The following input reader and output writer functions are provided by the xm_syslog module to support this TLS transport defined in RFC 5425. While RFC 5425 explicitly defines that the TLS network transport protocol is to be used, pure TCP may be used if security is not a requirement. Syslog messages can also be written to file with this framing format using these functions.

InputType Syslog_TLS

This input reader function parses the payload size and then reads the message according to this value. It is required to support Syslog TLS transport defined in RFC 5425.

OutputType Syslog_TLS

This output writer function prepends the payload size to the message. It is required to support Syslog TLS transport defined in RFC 5425.

The Syslog_TLS InputType/OutputType can work with any input/output such as im_tcp or im_file and does not depend on SSL transport at all. The name Syslog_TLS was chosen to refer to the octet-framing method described in RFC 5425 used for TLS transport.

Structured data in IETF Syslog messages is parsed and put into NXLog fields. The SD-ID will be prepended to the field name with a dot unless it is NXLOG@XXXX. Consider the following Syslog message:

<30>1 2011-12-04T21:16:10.000000+02:00 host app procid msgid [exampleSDID@32473 eventSource="Application" eventID="1011"] Message part

After this IETF-formatted Syslog message is parsed with parse_syslog_ietf(), there will be two additional fields: $exampleSDID.eventID and $exampleSDID.eventSource. When SD-ID is NXLOG, the field name will be the same as the SD-PARAM name. The two additional fields extracted from the structured data part of the following IETF Syslog message are $eventID and $eventSource:

<30>1 2011-12-04T21:16:10.000000+02:00 host app procid msgid [NXLOG@32473 eventSource="Application" eventID="1011"] Message part

All fields in the structured data part are parsed as strings.

Configuration

The xm_syslog module accepts the following directives in addition to the common module directives.

Optional directives

IETFTimestampInGMT

This is an alias for the UTCTimestamp directive below.

ReplaceLineBreaks

This optional directive specifies a character with which to replace line breaks in the Syslog message when generating Syslog events with to_syslog_bsd(), to_syslog_ietf(), and to_syslog_snare(). The default is a space. To retain line breaks in Syslog messages, set this to \n.

SnareDelimiter

This optional directive takes a single character (see below) as argument. This character is used by the to_syslog_snare() procedure to separate fields. If this directive is not specified, the default delimiter character is the tab (\t). In latter versions of Snare 4 this has changed to the hash mark (#); this directive can be used to specify the alternative delimiter. Note that there is no delimiter after the last field.

SnareReplacement

This optional directive takes a single character (see below) as argument. This character is used by the to_syslog_snare() procedure to replace occurrences of the delimiter character inside the $Message field. If this directive is not specified, the default replacement character is the space.

UTCTimestamp

This optional boolean directive can be used to format the timestamps produced by to_syslog_ietf() in UTC/GMT instead of local time. The default is FALSE: local time is used with a timezone indicator.

Specifying Quote, Escape, and Delimiter Characters

The SnareDelimiter and SnareReplacement directives can be specified in several ways.

Unquoted single character

Any printable character can be specified as an unquoted character, except for the backslash (\):

Delimiter ;
Control characters

The following non-printable characters can be specified with escape sequences:

\a

audible alert (bell)

\b

backspace

\t

horizontal tab

\n

newline

\v

vertical tab

\f

formfeed

\r

carriage return

For example, to use TAB delimiting:

Delimiter \t
A character in single quotes

The configuration parser strips whitespace, so it is not possible to define a space as the delimiter unless it is enclosed within quotes:

Delimiter ' '

Printable characters can also be enclosed:

Delimiter ';'

The backslash can be specified when enclosed within quotes:

Delimiter '\'
A character in double quotes

Double quotes can be used like single quotes:

Delimiter " "

The backslash can be specified when enclosed within double quotes:

Delimiter "\"
A hexadecimal ASCII code

Hexadecimal ASCII character codes can be used prefixed with 0x. For example, the space can be specified as:

Delimiter 0x20

This is equivalent to:

Delimiter " "

Functions

The following functions are exported by xm_syslog.

string syslog_facility_string(integer arg)

Convert a Syslog facility value to a string.

integer syslog_facility_value(string arg)

Convert a Syslog facility string to an integer.

string syslog_severity_string(integer arg)

Convert a Syslog severity value to a string.

integer syslog_severity_value(string arg)

Convert a Syslog severity string to an integer.

Procedures

The following procedures are exported by xm_syslog.

parse_syslog();

Parse the $raw_event field as either BSD Syslog (RFC 3164) or IETF Syslog (RFC 5424) format.

parse_syslog(string source);

Parse the given string as either BSD Syslog (RFC 3164) or IETF Syslog (RFC 5424) format.

parse_syslog_bsd();

Parse the $raw_event field as BSD Syslog (RFC 3164) format.

parse_syslog_bsd(string source);

Parse the given string as BSD Syslog (RFC 3164) format.

parse_syslog_ietf();

Parse the $raw_event field as IETF Syslog (RFC 5424) format.

parse_syslog_ietf(string source);

Parse the given string as IETF Syslog (RFC 5424) format.

to_snare();

Create a SNARE formatted log message in $raw_event. The following fields are used to construct the $raw_event field: $EventTime, $Hostname, $SeverityValue, $FileName, $Channel, $SourceName, $AccountName, $AccountType, $EventType, $Category, $RecordNumber, and $Message.

to_syslog_bsd();

Create a BSD Syslog formatted log message in $raw_event from the fields of the event. The following fields are used to construct the $raw_event field: $EventTime; $Hostname; $SourceName; $ProcessID or $ExecutionProcessID; $Message or $raw_event; $SyslogSeverity, $SyslogSeverityValue, $Severity, or $SeverityValue; and $SyslogFacility or $SyslogFacilityValue. If the fields are not present, a sensible default is used.

to_syslog_ietf();

Create an IETF Syslog (RFC 5424) formatted log message in $raw_event from the fields of the event. The following fields are used to construct the $raw_event field: $EventTime; $Hostname; $SourceName; $ProcessID or $ExecutionProcessID; $Message or $raw_event; $SyslogSeverity, $SyslogSeverityValue, $Severity, or $SeverityValue; and $SyslogFacility or $SyslogFacilityValue. If the fields are not present, a sensible default is used.

to_syslog_snare();

Create a SNARE Syslog formatted log message in $raw_event. This procedure is compatible with the Snare format for Windows Event Log. The following fields are used to construct the $raw_event field: $EventTime, $Hostname, $SeverityValue, $FileName, $Channel, $SourceName, $AccountName, $AccountType, $EventType, $Category, $RecordNumber, and $Message.

Fields

The following fields are used by xm_syslog.

In addition to the fields listed below, the parse_syslog() and parse_syslog_ietf() procedures will create fields from the Structured Data part of an IETF Syslog message. If the SD-ID in this case is not "NXLOG", these fields will be prefixed by the SD-ID (for example, $mySDID.CustomField).

$raw_event (type: string)

A Syslog formatted string, set after to_syslog_bsd(), to_syslog_ietf(), or to_snare(), or to_syslog_snare() is invoked.

$EventTime (type: datetime)

The timestamp found in the Syslog message, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called. If the year value is missing, it is set as described in the core fix_year() function.

$Hostname (type: string)

The hostname part of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called.

$Message (type: string)

The message part of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called.

$MessageID (type: string)

The MSGID part of the syslog message, set after parse_syslog_ietf() is called.

$ProcessID (type: string)

The process ID in the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called.

$Severity (type: string)

The normalized severity name of the event. See $SeverityValue.

$SeverityValue (type: integer)

The normalized severity number of the event, mapped as follows.

Syslog Severity Normalized Severity

0/emerg

5/critical

1/alert

5/critical

2/crit

5/critical

3/err

4/error

4/warning

3/warning

5/notice

2/info

6/info

2/info

7/debug

1/debug
$SourceName (type: string)

The application/program part of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called.

$SyslogFacility (type: string)

The facility name of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called. The default facility is user.

$SyslogFacilityValue (type: integer)

The facility code of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called. The default facility is 1 (user).

$SyslogSeverity (type: string)

The severity name of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called. The default severity is notice. See $SeverityValue.

$SyslogSeverityValue (type: integer)

The severity code of the Syslog line, set after parse_syslog(), parse_syslog_bsd(), or parse_syslog_ietf() is called. The default severity is 5 (notice). See $SeverityValue.

Examples

When you are parsing syslog messages with NXLog, understanding the fields created by your input instance is crucial for effective log parsing and troubleshooting. If you’re unsure of the fields your input instance creates, use the to_json() procedure to write the log record in JSON format. This procedure converts all fields except the $raw_event field to a JSON object and writes it to the $raw_event field. Note that the procedure writes timestamp fields in a different format than the NXLog default but retains the original value of all other fields.
Example 1. Collect and parse syslog BSD logs

This configuration receives logs over UDP and parses log records with the parse_syslog_bsd() procedure.

nxlog.conf
<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:514
    Exec          parse_syslog_bsd();
</Input>

The following is a syslog message in BSD format.

Input sample
<30>May 15 17:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2

When the NXLog configuration above processes this message, it adds the following fields to the log record.

Field Value

$EventReceivedTime

2024-05-15 17:46:46

$SourceModuleName

udp_listen

$SourceModuleType

im_udp

$Hostname

myserver

$SyslogFacilityValue

3

$SyslogFacility

DAEMON

$SyslogSeverityValue

6

$SyslogSeverity

INFO

$SeverityValue

2

$Severity

INFO

$EventTime

2024-05-15 17:40:27

$SourceName

sshd

$ProcessID

26459

$Message

Accepted publickey for john from 192.168.1.1 port 41193 ssh2

$raw_event

<30>May 15 17:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2
Parsing syslog messages results in several fields related to the event severity, such as $SyslogSeverityValue, $SyslogSeverity, $SeverityValue, and $Severity. Depending on your SIEM or log analysis tool, you often only need one numeric or string severity field. In this case, you can reduce redundancy by deleting the unwanted fields.

xm_syslog also supports converting logs to the syslog BSD format. The configuration depends on whether the input logs are:

Unstructured

The input consists of plain text records in an unknown format.

Structured

NXLog receives logs in a known format, and the configuration parses log records into fields.

Example 2. Forward unstructured logs in syslog BSD format

This configuration collects nginx access logs from a file. It uses the to_syslog_bsd() procedure to convert log records to syslog and then forwards them over UDP.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input nginx>
    Module    im_file
    File      '/var/log/nginx/access.log'
</Input>

<Output udp>
    Module    om_udp
    Host      192.168.1.123:514
    Exec      to_syslog_bsd();
</Output>

The following is an HTTP access log record generated by nginx.

Input sample
192.168.1.5 - johndoe [16/May/2024:08:23:34 +0000] "GET /index.html HTTP/1.1" 200 612 "http://referrer.com/page" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"

In this case, NXLog uses the following core fields to convert the record to syslog BSD format.

Field Value

$EventReceivedTime

May 16 11:07:24

$Hostname

WEB-SVR

$raw_event

192.168.1.5 - johndoe [16/May/2024:08:23:34 +0000] "GET /index.html HTTP/1.1" 200 612 "http://referrer.com/page" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
Output sample
<13>May 16 11:07:24 WEB-SVR 192.168.1.5 - johndoe [16/May/2024:08:23:34 +0000] "GET /index.html HTTP/1.1" 200 612 "http://referrer.com/page" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
Example 3. Forward structured logs in syslog BSD format

This configuration collects Windows events with the im_msvistalog input module, automatically parsing events into structured data. The output instance converts the log records to syslog with the to_syslog_bsd() procedure and writes them to a file.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
       <QueryList>
          <Query Id="0" Path="Security">
          <Select Path="Security">*[System[(EventID=4625)]]</Select>
          </Query>
       </QueryList>
    </QueryXML>
</Input>

<Output file>
    Module    om_file
    File      'C:\logs\eventlog.txt'
    Exec      to_syslog_bsd();
</Output>

The following input sample shows Windows event ID 4625 concerning a failed login attempt.

Input sample
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-a5ba-3e3b0328c30d}" />
    <EventID>4625</EventID>
    <Version>0</Version>
    <Level>0</Level>
    <Task>12544</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8010000000000000</Keywords>
    <TimeCreated SystemTime="2024-05-07T20:34:38.1328923Z" />
    <EventRecordID>456703</EventRecordID>
    <Correlation ActivityID="{a9b5a229-9b0a-0008-7aa2-b5a90a9bda01}" />
    <Execution ProcessID="1124" ThreadID="7052" />
    <Channel>Security</Channel>
    <Computer>WIN-SERVER</Computer>
  <Security />
  </System>
    <EventData>
    <Data Name="SubjectUserSid">S-1-5-21-3603782361-128455402-3223720292-1002</Data>
    <Data Name="SubjectUserName">jdoe</Data>
    <Data Name="SubjectDomainName">NXLOG</Data>
    <Data Name="SubjectLogonId">0x9d84141</Data>
    <Data Name="TargetUserSid">S-1-0-0</Data>
    <Data Name="TargetUserName">jdoe</Data>
    <Data Name="TargetDomainName">NXLOG</Data>
    <Data Name="Status">0xc000006e</Data>
    <Data Name="FailureReason">%%2313</Data>
    <Data Name="SubStatus">0xc000006e</Data>
    <Data Name="LogonType">2</Data>
    <Data Name="LogonProcessName">Advapi</Data>
    <Data Name="AuthenticationPackageName">Negotiate</Data>
    <Data Name="WorkstationName">WIN-SERVER</Data>
    <Data Name="TransmittedServices">-</Data>
    <Data Name="LmPackageName">-</Data>
    <Data Name="KeyLength">0</Data>
    <Data Name="ProcessId">0x3990</Data>
    <Data Name="ProcessName">C:\Program Files\Google\Chrome\Application\chrome.exe</Data>
    <Data Name="IpAddress">-</Data>
    <Data Name="IpPort">-</Data>
  </EventData>
</Event>

In this case, NXLog uses the following fields to convert the record to syslog BSD format.

Field Value

$EventTime

May 7 22:34:38

$Hostname

WIN-SERVER

$SourceName

Microsoft-Windows-Security-Auditing

$ProcessID

0x3990

$raw_event

An account failed to log on. …​
Output sample
<11>May  7 22:34:38 WIN-SERVER Microsoft-Windows-Security-Auditing[0x3990]: An account failed to log on.    Subject:  	Security ID:		S-1-5-21-3603782361-128455402-3223720292-1002  	Account Name:		jdoe  	Account Domain:		NXLOG  	Logon ID:		0x9d84141    Logon Type:			2    Account For Which Logon Failed:  	Security ID:		S-1-0-0  	Account Name:		jdoe  	Account Domain:		NXLOG    Failure Information:  	Failure Reason:		Unknown user name or bad password.  	Status:			0xc000006e  	Sub Status:		0xc000006e    Process Information:  	Caller Process ID:	0x3990  	Caller Process Name:	C:\Program Files\Google\Chrome\Application\chrome.exe    Network Information:  	Workstation Name:	WIN-SERVER  	Source Network Address:	-  	Source Port:		-    Detailed Authentication Information:  	Logon Process:		Advapi    	Authentication Package:	Negotiate  	Transited Services:	-  	Package Name (NTLM only):	-  	Key Length:		0    This event is generated when a logon request fails. It is generated on the computer where access was attempted.    The Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).    The Process Information fields indicate which account and process on the system requested the logon.    The Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The authentication information fields provide detailed information about this specific logon request.  	- Transited services indicate which intermediate services have participated in this logon request.  	- Package name indicates which sub-protocol was used among the NTLM protocols.  	- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.
Example 4. Collect and parse syslog IETF logs

This configuration receives logs over UDP and parses log records with the parse_syslog_ietf() procedure.

nxlog.conf
<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:1514
    Exec          parse_syslog_ietf();
</Input>

The following is a syslog message in IETF format.

Input sample
<30>1 2024-05-15T17:46:27.000Z myserver sshd 26459 - [origin ip="192.168.1.1" software="sshd" swVersion="1.0"][auth method="publickey" user="john" srcPort="41193" protocol="ssh2"] Accepted publickey for john from 192.168.1.1 port 41193 ssh2

When the NXLog configuration above processes this message, it adds the following fields to the log record.

Field Value

$EventReceivedTime

2024-05-15 17:46:46

$SourceModuleName

udp_listen

$SourceModuleType

im_udp

$Hostname

myserver

$SyslogFacilityValue

3

$SyslogFacility

DAEMON

$SyslogSeverityValue

6

$SyslogSeverity

INFO

$SeverityValue

2

$Severity

INFO

$EventTime

2024-05-15 17:46:27

$SourceName

sshd

$ProcessID

26459

$Message

Accepted publickey for john from 192.168.1.1 port 41193 ssh2

$raw_event

<30>1 2024-05-15T17:46:27.000Z myserver sshd 26459 - [origin ip="192.168.1.1" software="sshd" swVersion="1.0"][auth method="publickey" user="john" srcPort="41193" protocol="ssh2"] Accepted publickey for john from 192.168.1.1 port 41193 ssh2
Parsing syslog messages results in several fields related to the event severity, such as $SyslogSeverityValue, $SyslogSeverity, $SeverityValue, and $Severity. Depending on your SIEM or log analysis tool, you often only need one numeric or string severity field. In this case, you can reduce redundancy by deleting the unwanted fields.

xm_syslog can convert unstructured and structured logs to the syslog IETF format.

Example 5. Forward unstructured logs in syslog IETF format

This configuration collects plain text logs from a file. It uses the to_syslog_ietf() procedure to convert log records to syslog and then forwards them over UDP.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input fin_logs>
    Module    im_file
    File      '/var/log/transactions.log'
</Input>

<Output udp>
    Module    om_udp
    Host      192.168.1.123:514
    Exec      to_syslog_ietf();
</Output>

The following is a log record of a financial transaction.

Input sample
2024-05-17 08:53:22, INFO, UserID:12345, Action: Transaction Attempt, Amount: $250.00, DeviceID: AB1234C, DeviceType: Mobile, Location: New York, IP: 192.168.1.1, Result: Success, Additional Info: Transaction completed without issues.

NXLog will use the following core fields to convert the record to syslog BSD format.

Field Value

$EventReceivedTime

2024-05-26 08:06:46

$Hostname

FIN-SERVER

$raw_event

2024-05-17 08:53:22, INFO, UserID:12345, Action: Transaction Attempt, Amount: $250.00, DeviceID: AB1234C, DeviceType: Mobile, Location: New York, IP: 192.168.1.1, Result: Success, Additional Info: Transaction completed without issues.
Output sample
<13>1 2024-05-26T08:06:46.337598+02:00 FIN-SERVER - - - [NXLOG@14506 EventReceivedTime="2024-05-26 08:06:46" SourceModuleName="fin_logs" SourceModuleType="im_file"] 2024-05-17 08:53:22, INFO, UserID:12345, Action: Transaction Attempt, Amount: $250.00, DeviceID: AB1234C, DeviceType: Mobile, Location: New York, IP: 192.168.1.1, Result: Success, Additional Info: Transaction completed without issues.
Example 6. Forward structured logs in syslog IETF format

This configuration collects Linux Audit logs with the im_linuxaudit input module, automatically parsing events into structured data. The output instance converts the log records to syslog with the to_syslog_ietf() procedure and writes them to a file.

nxlog.conf
<Extension syslog>
    Module           xm_syslog
</Extension>

<Input linuxaudit>
    Module           im_linuxaudit
    FlowControl      FALSE
    ResolveValues    TRUE
    <Rules>         
        -w /home/johndoe/privatekey -p wa -k privatekey_change
    </Rules>
</Input>

<Output file>
    Module           om_file
    File             '/var/log/messages'
    Exec             to_syslog_bsd();
</Output>

The following input sample shows a Linux audit event related to a file change.

Input sample
2024-05-29 13:41:13 NXLOG-LINUX INFO type="SYSCALL" time="2024-05-29 13:41:13" seq="336" arch="x86_64" syscall="openat" success="yes" exit="91" a0="ffffff9c" a1="55dfae2c3a98" a2="80241" a3="1b6" items="2" ppid="1412" pid="29957" auid="4294967295" uid="1000" gid="1000" euid="1000" suid="1000" fsuid="1000" egid="1000" sgid="1000" fsgid="1000" tty="(none)" ses="4294967295" comm="notepadqq-bin" exe="/usr/lib/notepadqq/notepadqq-bin" key="privatekeychange"

In this case, NXLog uses the following fields to convert the record to syslog IETF format.

Field Value

$type

SYSCALL

$time

2024-05-29 13:41:13

$seq

295

$arch

x86_64

$syscall

openat

$success

yes

$exit

97

$a0

ffffff9c

$a1

55dfae2c3a98

$a2

80241

$a3

1b6

$items

2

$ppid

1412

$pid

29957

$auid

4294967295

$uid

1000

$gid

1000

$euid

1000

$suid

1000

$fsuid

1000

$egid

1000

$sgid

1000

$fsgid

1000

$tty

(none)

$ses

4294967295

$comm

notepadqq-bin

$exe

/usr/lib/notepadqq/notepadqq-bin

$key

privatekeychange

$Hostname

NXLOG-LINUX

$EventReceivedTime

2024-05-29 13:42:45

$SourceModuleName

linuxaudit

$SourceModuleType

im_linuxaudit

$raw_event

2024-05-29 13:41:13 NXLOG-LINUX INFO type="SYSCALL" time="2024-05-29 13:41:13" seq="336" arch="x86_64" syscall="openat" success="yes" exit="91" a0="ffffff9c" a1="55dfae2c3a98" a2="80241" a3="1b6" items="2" ppid="1412" pid="29957" auid="4294967295" uid="1000" gid="1000" euid="1000" suid="1000" fsuid="1000" egid="1000" sgid="1000" fsgid="1000" tty="(none)" ses="4294967295" comm="notepadqq-bin" exe="/usr/lib/notepadqq/notepadqq-bin" key="privatekeychange"
Output sample
<13>1 2024-05-29T12:46:54.615621+02:00 NXLOG-LINUX - - - [NXLOG@14506 type="SYSCALL" time="2024-05-29 12:46:54" seq="283" arch="x86_64" syscall="openat" success="yes" exit="91" a0="ffffff9c" a1="55dfad8e4e28" a2="80241" a3="1b6" items="2" ppid="1412" pid="29957" auid="4294967295" uid="1000" gid="1000" euid="1000" suid="1000" fsuid="1000" egid="1000" sgid="1000" fsgid="1000" tty="(none)" ses="4294967295" comm="notepadqq-bin" exe="/usr/lib/notepadqq/notepadqq-bin" key="privatekeychange" EventReceivedTime="2024-05-29 13:42:45" SourceModuleName="linuxaudit" SourceModuleType="im_linuxaudit"] 2024-05-29 12:46:54 NXLOG-LINUX INFO type="SYSCALL" time="2024-05-29 12:46:54" seq="283" arch="x86_64" syscall="openat" success="yes" exit="91" a0="ffffff9c" a1="55dfad8e4e28" a2="80241" a3="1b6" items="2" ppid="1412" pid="29957" auid="4294967295" uid="1000" gid="1000" euid="1000" suid="1000" fsuid="1000" egid="1000" sgid="1000" fsgid="1000" tty="(none)" ses="4294967295" comm="notepadqq-bin" exe="/usr/lib/notepadqq/notepadqq-bin" key="privatekeychange"

When collecting logs from multiple sources, you may need to process syslog messages in both the BSD and IETF formats. xm_syslog provides a generic procedure that automatically detects the format of the syslog message.

Example 7. Collect and parse syslog BSD and IETF formats simultaneously

This configuration receives logs over UDP and parses log records with the parse_syslog() procedure.

nxlog.conf
<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:1514
    Exec          parse_syslog();
</Input>

This input sample consists of two lines, an IETF syslog message on the first line and a BSD syslog message on the second.

Input sample
<30>1 2024-05-30T09:40:27.000Z myserver sshd 26459 - [origin ip="192.168.1.1" software="sshd" swVersion="1.0"][auth method="publickey" user="john" srcPort="41193" protocol="ssh2"] Accepted publickey for john from 192.168.1.1 port 41193 ssh2
<30>May 30 09:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2

When the NXLog configuration above processes the first message in IETF format, it adds the following fields to the log record:

Table 1. Created fields for the first line (IETF syslog)
Field Value

$EventReceivedTime

2024-05-30 09:59:37

$SourceModuleName

udp_listen

$SourceModuleType

im_udp

$Hostname

myserver

$SyslogFacilityValue

3

$SyslogFacility

DAEMON

$SyslogSeverityValue

6

$SyslogSeverity

INFO

$SeverityValue

2

$Severity

INFO

$EventTime

2024-05-30 09:40:27

$SourceName

sshd

$ProcessID

26459

$origin.ip

192.168.1.1

$origin.software

sshd

$origin.swVersion

1.0

$auth.method

publickey

$auth.user

john

$auth.srcPort

41193

$auth.protocol

ssh2

$Message

Accepted publickey for john from 192.168.1.1 port 41193 ssh2

$raw_event

<30>1 2024-1105-30T09:40:27.000Z myserver sshd 26459 - [origin ip="192.168.1.1" software="sshd" swVersion="1.0"][auth method="publickey" user="john" srcPort="41193" protocol="ssh2"] Accepted publickey for john from 192.168.1.1 port 41193 ssh2

Processing the second message in BSD format results in the following fields:

Field Value

$EventReceivedTime

2024-05-30 09:59:37

$SourceModuleName

udp_listen

$SourceModuleType

im_udp

$Hostname

myserver

$SyslogFacilityValue

3

$SyslogFacility

DAEMON

$SyslogSeverityValue

6

$SyslogSeverity

INFO

$SeverityValue

2

$Severity

INFO

$EventTime

2024-05-30 09:40:27

$SourceName

sshd

$ProcessID

26459

$Message

Accepted publickey for john from 192.168.1.1 port 41193 ssh2

$raw_event

<30>May 30 09:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2