Perl (xm_perl)

The Perl programming language is widely used for log processing and comes with a broad set of modules bundled or available from CPAN. Code can be written more quickly in Perl than in C, and code execution is safer because exceptions (croak/die) are handled properly and will only result in an unfinished attempt at log processing rather than taking down the whole NXLog process.

While the NXLog language is already a powerful framework, it is not intended to be a fully featured programming language and does not provide lists, arrays, hashes, and other features available in many high-level languages. With this module, Perl can be used to process event data via a built-in Perl interpreter. See also the im_perl and om_perl modules.

The Perl interpreter is only loaded if the module is declared in the configuration. The module will parse the file specified in the PerlCode directive when NXLog starts the module. This file should contain one or more methods which can be called from the Exec directive of any module that will use Perl for log processing. See the example below.

Perl code defined via this module must not be called from the im_perl and om_perl modules as that would involve two Perl interpreters and will likely result in a crash.

To access event data, the Log::Nxlog module must be included, which provides the following methods.

log_debug(msg): Send the message msg to the internal logger on DEBUG log level. This method does the same as the log_debug() procedure in NXLog.
log_info(msg): Send the message msg to the internal logger on INFO log level. This method does the same as the log_info() procedure in NXLog.
log_warning(msg): Send the message msg to the internal logger on WARNING log level. This method does the same as the log_warning() procedure in NXLog.
log_error(msg): Send the message msg to the internal logger on ERROR log level. This method does the same as the log_error() procedure in NXLog.
delete_field(event, key): Delete the value associated with the field named key.
field_names(event): Return a list of the field names contained in the event data. This method can be used to iterate over all of the fields.
field_type(event, key): Return a string representing the type of the value associated with the field named key.
get_field(event, key): Retrieve the value associated with the field named key. This method returns a scalar value if the key exists and the value is defined, otherwise it returns undef.
set_field_boolean(event, key, value): Set the boolean value in the field named key.
set_field_integer(event, key, value): Set the integer value in the field named key.
set_field_string(event, key, value): Set the string value in the field named key.

For the full NXLog Perl API, see the POD documentation in Nxlog.pm. The documentation can be read with perldoc Log::Nxlog.

Perl prerequisites for Windows

You must install a separate Perl environment to use the xm_perl module on Windows. We only guarantee compatibility with Strawberry Perl 5.32.0.1. Newer versions will likely work too, but older versions (5.30.3.1 and before) are not supported. See the Strawberry Perl Releases page to manually download and install the required package.

The following PowerShell script automatically downloads and installs Perl from the MSI file.

windows-perl-install.ps1

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$baseuri="http://strawberryperl.com/download/5.32.0.1"
$msifile="strawberry-perl-5.32.0.1-64bit.msi"
Invoke-WebRequest -uri $baseuri/$msifile -OutFile $msifile
msiexec /passive /i $msifile

Configuration

The xm_perl module accepts the following directives in addition to the common module directives.

PerlCode

This mandatory directive expects a file containing valid Perl code. This file is read and parsed by the Perl interpreter. Methods defined in this file can be called with the call() procedure.

On Windows, the Perl script invoked by the PerlCode directive must define the Perl library paths at the beginning of the script to provide access to the Perl modules.

nxlog-windows.pl

use lib 'c:\Program Files\nxlog\data';

Config

This optional directive allows you to pass configuration strings to the script file defined by the PerlCode directive. This is a block directive and any text enclosed within <Config></Config> is submitted as a single string literal to the Perl code.

If you pass several values using this directive (for example, separated by the \n delimiter) be sure to parse the string correspondingly inside the Perl code.

Procedures

The following procedures are exported by xm_perl.

call(string subroutine);: Call the given Perl subroutine.

perl_call(string subroutine, varargs args);: Call the given Perl subroutine.

Examples

Example 1. Using the built-in Perl interpreter

In this example, the event field $EventData.Binary is parsed by a Perl method. The decoded and sanitized value is returned in the $EventData.String field.

nxlog.conf

<Extension perl>
    Module          xm_perl
    PerlCode        "../conf/nxlog.d/unhex.pl"
</Extension>

<Output fileout>
    Module          om_file
    File            'tmp/output'

    # Now call the 'unhex' subroutine defined in 'unhex.pl'
    Exec            perl_call("unhex");

    # You can also invoke this public procedure 'call' in case
    # of multiple xm_perl instances like this:
    # Exec          perl->call("unhex");
</Output>

unhex.pl

# Use FindBin to add the NXLog Perl module directory to Perl's search path
use FindBin;
use lib "$FindBin::Bin/../../../../opt/nxlog/Network/Library/Perl/5.30"; (1)
use Log::Nxlog;

BEGIN{
   # this runs on module start, sending a message to the internal log
   Log::Nxlog::log_info("unhex.pl initialized");
}

sub unhex
{
   # Load event into $event
   my ( $event ) = @_;
   # extract a single field, called EventData.Binary and store it in $hex
   my $hex = Log::Nxlog::get_field($event, 'EventData.Binary');
   # Decode hex encoded data in $hex and place it in $chars
   my $chars = pack('H*', $hex);
   # delete spurious NUL chars
   $chars =~ s/\0//g;
   # replace newlines w/ \n
   $chars =~ s/\n/\\n/g;
   # create a new field EventData.String and place $chars in it - as a string
   Log::Nxlog::set_field_string($event, 'EventData.String', $chars);
   # Optionally log the value of the variable $chars
   #Log::Nxlog::log_info($chars);
}

Perl may return a Can’t locate Log/Nxlog.pm in @INC (you may need to install the Log::Nxlog module) error if it can’t find the NXLog Perl module. Use FindBin to add NXLog’s Perl module directory to Perl’s search path. By default, the NXLog Perl module path is:

/opt/nxlog/Network/Library/Perl/<system_perl_version>/Log on macOS
/opt/nxlog/src/modules/extension/perl/version/Log on Linux
C:\Program Files\nxlog\data\Log on Microsoft Windows