Pattern Matcher (pm_pattern)

This module makes it possible to execute pattern matching with a patterndatabase file in XML format. The pm_pattern module has been replaced by an extension module, xm_pattern, which provides nearly identical functionality with the improved flexibility of an extension module.

The XML Schema Definition (XSD) for the pattern database file is available in the nxlog-public/contrib repository.

To examine the supported platforms, see the list of installer packages in the Available Modules chapter.

Configuration

The pm_pattern module accepts the following directives in addition to the common module directives.

PatternFile: This mandatory directive specifies the name of the pattern database file.

Fields

The following fields are used by pm_pattern.

$PatternID (type: integer): The ID number of the pattern which matched the message.

$PatternName (type: string): The name of the pattern which matched the message.

Examples

Example 1. Using the pm_pattern Module

This configuration reads BSD Syslog messages from the socket, processes the messages with a pattern file, and then writes them to file in JSON format.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Extension syslog>
    Module      xm_syslog
</Extension>

<Input uds>
    Module      im_uds
    UDS         /dev/log
    Exec        parse_syslog_bsd();
</Input>

<Processor pattern>
    Module      pm_pattern
    PatternFile /var/lib/nxlog/patterndb.xml
</Processor>

<Output file>
    Module      om_file
    File        "/var/log/out"
    Exec        to_json();
</Output>

<Route uds_to_file>
    Path        uds => pattern => file
</Route>

The following pattern database contains two patterns to match SSH authentication messages. The patterns are under a group named ssh which checks whether the $SourceName field is sshd and only tries to match the patterns if the logs are indeed from sshd. The patterns both extract AuthMethod, AccountName, and SourceIP4Address from the log message when the pattern matches the log. Additionally TaxonomyStatus and TaxonomyAction are set. The second pattern utilizes the Exec block, which is evaluated when the pattern matches.

For this pattern to work, the logs must be parsed with parse_syslog() prior to processing by the pm_pattern module (as in the above example), because it uses the $SourceName and $Message fields.

patterndb.xml

<?xml version='1.0' encoding='UTF-8'?>
<patterndb>
 <created>2010-01-01 01:02:03</created>
 <version>42</version>
 
 <group>
   <name>ssh</name>
   <id>42</id>
   <matchfield>
    <name>SourceName</name>
    <type>exact</type>
    <value>sshd</value>
   </matchfield>

   <pattern>
    <id>1</id>
    <name>ssh auth success</name>

    <matchfield>
     <name>Message</name>
     <type>regexp</type>
        <!-- Accepted publickey for nxlogfan from 192.168.1.1 port 4242 ssh2 -->
     <value>^Accepted (\S+) for (\S+) from (\S+) port \d+ ssh2</value>
     <capturedfield>
	<name>AuthMethod</name>
	<type>string</type>
     </capturedfield>
     <capturedfield>
	<name>AccountName</name>
	<type>string</type>
     </capturedfield>
     <capturedfield>
	<name>SourceIP4Address</name>
        <type>string</type>
     </capturedfield>
    </matchfield>

    <set>
     <field>
       <name>TaxonomyStatus</name>
       <value>success</value>
       <type>string</type>
     </field>
     <field>
       <name>TaxonomyAction</name>
       <value>authenticate</value>
       <type>string</type>
     </field>
    </set>
   </pattern>

   <pattern>
    <id>2</id>
    <name>ssh auth failure</name>

    <matchfield>
     <name>Message</name>
     <type>regexp</type>
     <value>^Failed (\S+) for invalid user (\S+) from (\S+) port \d+ ssh2</value>

     <capturedfield>
	<name>AuthMethod</name>
	<type>string</type>
     </capturedfield>
     <capturedfield>
	<name>AccountName</name>
	<type>string</type>
     </capturedfield>
     <capturedfield>
	<name>SourceIP4Address</name>
        <type>string</type>
     </capturedfield>
    </matchfield>

    <set>
     <field>
       <name>TaxonomyStatus</name>
       <value>failure</value>
       <type>string</type>
     </field>
     <field>
       <name>TaxonomyAction</name>
       <value>authenticate</value>
       <type>string</type>
     </field>
    </set>

    <exec>
      $TestField = 'test';
    </exec>
    <exec>
      $TestField = $Testfield + 'value';
    </exec>
   </pattern>

 </group>

</patterndb>