Pattern Matcher (pm_pattern)

This module makes it possible to execute pattern matching with a pattern database file in XML format.

This module is being phased out and will be removed in a future release. Use the xm_pattern module instead.

To examine the supported platforms, see the list of installation packages.

Configuration

The pm_pattern module accepts the following directives in addition to the common module directives.

Required directives

The following directives are required for the module to start.

PatternFile

This mandatory directive specifies the name of the pattern database file. The XML Schema Definition (XSD) for the pattern database file is available in the nxlog-public/contrib repository.

Fields

The following fields are used by pm_pattern.

$PatternID (type: integer): The ID number of the pattern which matched the message.

$PatternName (type: string): The name of the pattern which matched the message.

Examples

Example 1. Using the pm_pattern Module

This configuration reads BSD Syslog messages from the socket, processes the messages with a pattern file, and then writes them to file in JSON format.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Extension syslog>
    Module      xm_syslog
</Extension>

<Input uds>
    Module      im_uds
    UDS         /dev/log
    Exec        parse_syslog_bsd();
</Input>

<Processor pattern>
    Module      pm_pattern
    PatternFile /var/lib/nxlog/patterndb.xml
</Processor>

<Output file>
    Module      om_file
    File        "/var/log/out"
    Exec        to_json();
</Output>

<Route uds_to_file>
    Path        uds => pattern => file
</Route>

The following pattern database contains two patterns to match SSH authentication messages. The patterns are under a group named ssh which checks whether the $SourceName field is sshd and only tries to match the patterns if the logs are indeed from sshd. The patterns both extract AuthMethod, AccountName, and SourceIPAddress from the log message when the pattern matches the log. Additionally TaxonomyStatus and TaxonomyAction are set. The second pattern utilizes the Exec block, which is evaluated when the pattern matches.

For this pattern to work, the logs must be parsed with parse_syslog() prior to processing by the pm_pattern module (as in the above example), because it uses the $SourceName and $Message fields.

patterndb.xml

<?xml version='1.0' encoding='UTF-8'?>
<patterndb>
 <created>2010-01-01 01:02:03</created>
 <version>42</version>
 
 <group>
   <id>42</id>
   <name>ssh</name>
   <matchfield>
    <name>SourceName</name>
    <type>exact</type>
    <value>sshd</value>
   </matchfield>

   <pattern>
    <id>1</id>
    <name>ssh auth success</name>

    <matchfield>
     <name>Message</name>
     <type>regexp</type>
        <!-- Accepted publickey for nxlogfan from 192.168.1.1 port 4242 ssh2 -->
     <value>^Accepted (\S+) for (\S+) from (\S+) port \d+ ssh2</value>
     <capturedfield>
	<name>AuthMethod</name>
	<type>STRING</type>
     </capturedfield>
     <capturedfield>
	<name>AccountName</name>
	<type>STRING</type>
     </capturedfield>
     <capturedfield>
	<name>SourceIPAddress</name>
        <type>IPADDR</type>
     </capturedfield>
    </matchfield>

    <set>
     <field>
       <name>TaxonomyStatus</name>
       <type>STRING</type>
       <value>success</value>
     </field>
     <field>
       <name>TaxonomyAction</name>
       <type>STRING</type>
       <value>authenticate</value>
     </field>
    </set>
   </pattern>

   <pattern>
    <id>2</id>
    <name>ssh auth failure</name>

    <matchfield>
     <name>Message</name>
     <type>regexp</type>
     <value>^Failed (\S+) for invalid user (\S+) from (\S+) port \d+ ssh2</value>

     <capturedfield>
	<name>AuthMethod</name>
	<type>STRING</type>
     </capturedfield>
     <capturedfield>
	<name>AccountName</name>
	<type>STRING</type>
     </capturedfield>
     <capturedfield>
	<name>SourceIPAddress</name>
        <type>IPADDR</type>
     </capturedfield>
    </matchfield>

    <set>
     <field>
       <name>TaxonomyStatus</name>
       <type>STRING</type>
       <value>failure</value>
     </field>
     <field>
       <name>TaxonomyAction</name>
       <type>STRING</type>
       <value>authenticate</value>
     </field>
    </set>

    <exec>
      $TestField = 'test';
    </exec>
    <exec>
      $TestField = $Testfield + 'value';
    </exec>
   </pattern>

 </group>

</patterndb>