LEEF (xm_leef)

This module provides two functions to generate and parse data in the Log Event Extended Format (LEEF), which is used by IBM Security QRadar products. For more information about the format see the Log Event Extended Format (LEEF) Version 2 specification.

Configuration

The xm_leef module accepts the following directives in addition to the common module directives.

Optional directives

AddSyslogHeader

This optional boolean directive specifies whether the log record should be prefixed with an RFC 3164 (BSD-style) syslog header. This defaults to TRUE (a syslog header will be added by the to_leef() procedure).

IncludeHiddenFields

This boolean directive specifies that the to_leef() function or the to_leef() procedure should include fields having a leading dot (.) or underscore (_) in their names. The default is TRUE. If IncludeHiddenFields is set to TRUE, then generated LEEF text will contain these otherwise excluded fields.

LEEFHeader

This optional directive takes a string type expression and only affects how to_leef() formats the result. It should evaluate to the following format:

LEEF:1.0|Microsoft|MSExchange|2013 SP1|15345|

It should typically be used as follows:

LEEFHeader 'LEEF:1.0|Microsoft|MSExchange|2013 SP1|' + $EventID + '|'

When this directive is not specified, the LEEF header is constructed using the $Vendor, $SourceName (or $SourceModuleName), $Version, and $EventID fields.

Functions

The following functions are exported by xm_leef.

string to_leef(): Convert the internal fields to a single LEEF formatted string.

Note that directive IncludeHiddenFields has an effect on fields included in the output.

Procedures

The following procedures are exported by xm_leef.

parse_leef();: Parse the $raw_event field as key-value pairs and create the following NXLog fields (if possible): $Category, $AccountName, $AccountType, $Domain, $EventTime, $Hostname, $MessageSourceAddress, $SeverityValue (mapped from the sev attribute), $SourceName, $devTimeFormat, $LEEFVersion, $Vendor, $Version, $EventID, $DelimiterCharacter.

parse_leef(string source);: Parse the the given string as key-value pairs and create the following NXLog fields (if possible): $Category, $AccountName, $AccountType, $Domain, $EventTime, $Hostname, $MessageSourceAddress, $SeverityValue (mapped from the sev attribute), $SourceName, $devTimeFormat, $LEEFVersion, $Vendor, $Version, $EventID, $DelimiterCharacter.

to_leef();

Format the internal fields as LEEF and put this into the $raw_event field. to_leef() will automatically map the following fields to event attributes, if available:

NXLog Agent field LEEF attribute

NXLog Agent field	LEEF attribute
`$AccountName`	`accountName`
`$AccountType`	`role`
`$Category`	`cat`
`$Domain`	`domain`
`$EventTime`	`devTime`
`$Hostname`	`identHostName`
`$MessageSourceAddress`	`src`
$SeverityValue (mapped)	`sev`
`$SourceName`	`vSrcName`

$AccountName

accountName

$AccountType

role

$Category

cat

$Domain

domain

$EventTime

devTime

$Hostname

identHostName

$MessageSourceAddress

src

$SeverityValue (mapped)

sev

$SourceName

vSrcName

Fields

The following fields are used by xm_leef.

In addition to the fields listed below, the parse_leef() procedure will create a field for every LEEF attribute contained in the source LEEF message such as $srcPort, $cat, $identHostName, etc.

$AccountName (type: string): The name of the user account that created the event.

$AccountType (type: string): The type of the user account (e.g., Administrator, User, Domain Admin) that created the event. This field takes the value of the role LEEF attribute.

$Category (type: string): A text string that extends the LEEF EventID field with more specific information about the LEEF event. This field takes the value of the cat LEEF attribute.

$DelimiterCharacter (type: string): The character specified as a delimiter in the LEEF header.

$devTimeFormat (type: string): A string that defines the date format of the LEEF event, contained in the devTimeFormat LEEF attribute, for example, "yyyy-MM-dd HH:mm:ss".

$Domain (type: string): The name of the domain the user account belongs to.

$EventID (type: string): The ID of the event. This field takes the value of the EventID LEEF header.

$EventTime (type: datetime): The time when the event occurred. This field takes the value of the devTime LEEF attribute.

$Hostname (type: string): The name of the host that created the event. This field takes the value of the identHostname LEEF attribute.

$LEEFVersion (type: string): The LEEF format version contained in the LEEF header, for example, LEEF:1.0.

$MessageSourceAddress (type: ipaddr): The IP address of the device that created the event. This field takes the value of the src LEEF attribute.

$SeverityValue (type: string)

A numeric value between 1 and 5 that indicates the severity of the event. This value is mapped to or from the value of the sev LEEF attribute:

LEEF sev attribute	$SeverityValue
≤2	1
3	1
4	2
5	2
6	3
7	3
8	4
9	4
≥10	5

$SourceName (type: string): The name of the subsystem or application that generated the event. This field takes the value of the Product LEEF header field.

$Vendor (type: string): A text string that identifies the vendor or manufacturer of the device sending the syslog event in the LEEF format. This field takes the value of the Vendor LEEF header field.

$Version (type: string): A string that identifies the version of the software or appliance that sent the event log. This field takes the value of the Product version LEEF header field.

Creating and populating fields

The parse_leef() procedure parses LEEF log records into structured data. It expects the $raw_event field or the string passed as a parameter to be in the following format:

LEEF:Version|Vendor|Product|Version|EventID|key1=value1	key2=value2

Once a log record is parsed with this procedure, fields are created based on the LEEF header and the event attributes. The fields can be used for further processing or to convert the log record to a different output format. For an example of how to parse LEEF log records and manipulate fields, see Parsing LEEF below.

Input modules may create additional fields containing various information. When converting to a different format, such fields will be included in the output log record, which may consume additional memory and bandwidth. For efficient handling of log records, consult the Fields section in the documentation of input modules and test the configuration before deployment. To delete any unwanted fields, use the delete() procedure or the xm_rewrite extension.

Examples

Example 1. Sending Windows Event Log as LEEF over UDP

This configuration will collect Windows Event Log and NXLog Agent internal messages, convert them to LEEF, and forward them via UDP.

nxlog.conf

<Extension leef>
    Module  xm_leef
</Extension>

<Input internal>
    Module  im_internal
</Input>

<Input eventlog>
    Module  im_msvistalog
</Input>

<Output udp>
    Module  om_udp
    Host    192.168.168.2:1514
    Exec    to_leef();
</Output>

<Route qradar>
    Path    internal, eventlog => udp
</Route>

Example 2. Parsing LEEF

This configuration uses the im_file input module to collect LEEF logs from a file. Log records are parsed into structured data using the parse_leef() procedure. Core fields that are not required are deleted, and the log record is converted to JSON using the to_json() procedure of the xm_json module.

nxlog.conf

<Extension leef>
    Module    xm_leef
</Extension>

<Extension json>
    Module    xm_json
</Extension>

<Input file_input>
    Module    im_file
    File      '/path/to/log/file'
    <Exec>
        parse_leef();

        # Delete core fields that are not required
        delete($SourceModuleName);
        delete($SourceModuleType);

        to_json();
    </Exec>
</Input>

Input sample

LEEF:2.0|Microsoft|Service Control Manager|14393.4704|7036|devTime=2021-11-04 14:35:06	identHostName=SERVER-01	Keywords=9259400833873739776	EventType=INFO	sev=4	Severity=INFO	vSrcName=Service Control Manager	ProviderGuid={555908D1-A6D7-4695-8E1E-26931D2012F4}	TaskValue=0	OpcodeValue=0	RecordNumber=11169	ExecutionProcessID=532	ExecutionThreadID=1004	Channel=System	Message=The Remote Registry service entered the running state.	param1=Remote Registry	param2=running	EventData.Binary=6E0078006C006F0067002F0034000000	devTimeFormat=yyyy-MM-dd HH:mm:ss

Output sample in JSON format

{
  "EventReceivedTime": "2021-11-04T14:35:40.898900+03:00",
  "LEEFVersion": "LEEF:2.0",
  "Vendor": "Microsoft",
  "SourceName": "Service Control Manager",
  "Version": "14393.4704",
  "EventID": "7036",
  "EventTime": "2021-11-04T14:35:06.000000+03:00",
  "Hostname": "SERVER-01",
  "Keywords": "9259400833873739776",
  "EventType": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "ProviderGuid": "{555908D1-A6D7-4695-8E1E-26931D2012F4}",
  "TaskValue": "0",
  "OpcodeValue": "0",
  "RecordNumber": "11169",
  "ExecutionProcessID": "532",
  "ExecutionThreadID": "1004",
  "Channel": "System",
  "Message": "The Remote Registry service entered the running state.",
  "param1": "Remote Registry",
  "param2": "running",
  "EventData.Binary": "6E0078006C006F0067002F0034000000",
  "devTimeFormat": "yyyy-MM-dd HH:mm:ss"
}