Perl (im_perl)

The Perl programming language is widely used for log processing and comes with a broad set of modules bundled or available from CPAN. Code can be written more quickly in Perl than in C, and code execution is safer because exceptions (croak/die) are handled properly and will only result in an unfinished attempt at log processing rather than taking down the whole NXLog Agent process.

This module makes it possible to execute Perl code in an input module to capture and inject event data directly into NXLog Agent. See also the om_perl and xm_perl modules.

The module will parse the file specified in the PerlCode directive when NXLog Agent starts the module. The Perl code must implement the read_data subroutine which will be called by the module. To generate event data, the Log::Nxlog Perl module must be included, which provides the following methods.

log_debug(msg): Send the message msg to the internal logger on DEBUG log level. This method does the same as the log_debug() procedure in NXLog Agent.
log_info(msg): Send the message msg to the internal logger on INFO log level. This method does the same as the log_info() procedure in NXLog Agent.
log_warning(msg): Send the message msg to the internal logger on WARNING log level. This method does the same as the log_warning() procedure in NXLog Agent.
log_error(msg): Send the message msg to the internal logger on ERROR log level. This method does the same as the log_error() procedure in NXLog Agent.
add_input_data(event): Pass the event record to the next module instance in the route. Failure to call this method will result in a memory leak.
logdata_new(): Create a new event record. The return value can be used with the set_field_*() methods to insert data.
set_field_boolean(event, key, value): Set the boolean value in the field named key.
set_field_integer(event, key, value): Set the integer value in the field named key.
set_field_string(event, key, value): Set the string value in the field named key.
set_read_timer(delay): Set the timer in seconds to invoke the read_data method again.

The set_read_timer() method should be called to invoke read_data again. This is typically used for polling data. The read_data method must not block.

For the full NXLog Agent Perl API, see the POD documentation in Nxlog.pm. The documentation can be read with perldoc Log::Nxlog.

Perl prerequisites for Windows

You must install a separate Perl environment to use the im_perl module on Windows. We only guarantee compatibility with Strawberry Perl 5.32.0.1. Newer versions will likely work too, but older versions (5.30.3.1 and before) are not supported. See the Strawberry Perl Releases page to manually download and install the required package.

The following PowerShell script automatically downloads and installs Perl from the MSI file.

windows-perl-install.ps1

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$baseuri="http://strawberryperl.com/download/5.32.0.1"
$msifile="strawberry-perl-5.32.0.1-64bit.msi"
Invoke-WebRequest -uri $baseuri/$msifile -OutFile $msifile
msiexec /passive /i $msifile

Configuration

The im_perl module accepts the following directives in addition to the common module directives.

Required directives

The following directives are required for the module to start.

PerlCode

This mandatory directive expects a file containing valid Perl code that implements the read_data subroutine. This file is read and parsed by the Perl interpreter.

On Windows, the Perl script invoked by the PerlCode directive must define the Perl library paths at the beginning of the script to provide access to the Perl modules.

nxlog-windows.pl

use lib 'c:\Program Files\nxlog\data';

Optional directives

Call

This optional directive specifies the Perl subroutine to invoke. With this directive, you can call only specific subroutines from your Perl code. If the directive is not specified, the default subroutine read_data is invoked.

Config

This optional directive allows you to pass configuration strings to the script file defined by the PerlCode directive. This is a block directive and any text enclosed within <Config></Config> is submitted as a single string literal to the Perl code.

If you pass several values using this directive (for example, separated by the \n delimiter) be sure to parse the string correspondingly inside the Perl code.

Examples

Example 1. Using im_perl to Generate Event Data

In this example, logs are generated by a Perl function that increments a counter and inserts it into the generated line.

nxlog.conf

<Input perl>
    Module      im_perl
    PerlCode    "path/to/perl-input.pl"
    Call        read_data1
</Input>

<Output file>
    Module      om_file
    File        "/tmp/output"
</Output>

<Route r1>
    Path        perl => file
</Route>

perl-input.pl

# Use FindBin to add the NXLog Perl module directory to Perl's search path
use FindBin; 
use lib "$FindBin::Bin/../../../../opt/nxlog/Network/Library/Perl/5.30"; (1)
use strict;
use warnings;

use Log::Nxlog;

my $counter;

sub read_data1
{
    my $event = Log::Nxlog::logdata_new();
    $counter //= 1;
    my $line = "Input1: this is a test line ($counter) that should appear in the output";
    $counter++;
    Log::Nxlog::set_field_string($event, 'raw_event', $line);
    Log::Nxlog::add_input_data($event);
    if ( $counter <= 100 )
    {
        Log::Nxlog::set_read_timer(0);
    }
}

Perl may return a Can’t locate Log/Nxlog.pm in @INC (you may need to install the Log::Nxlog module) error if it can’t find the NXLog Agent Perl module. Use FindBin to add NXLog Agent’s Perl module directory to Perl’s search path. By default, the NXLog Agent Perl module path is:

/opt/nxlog/Network/Library/Perl/<system_perl_version>/Log on macOS
/opt/nxlog/src/modules/extension/perl/version/Log on Linux
C:\Program Files\nxlog\data\Log on Microsoft Windows