NXLog Legacy Documentation

Program (om_exec)

This module executes a program or script on startup and writes (pipes) log data to its standard input. By default, it will write the contents of the $raw_event core field one record per line. The output format can be changed with the OutputType common module directive. The program or script is expected to be able to read data from its standard input. Refer to the Examples below. When the module stops, which usually happens when NXLog exits and the pipe is closed, it will stop the program or script. How this work is explaned below.

To examine the supported platforms, see the list of installer packages in the Available Modules chapter.
The program or script is started when NXLog starts and must not exit until the module is stopped. To invoke a program or script for each log message, use xm_exec instead.
If you are using a Python script, we recommend disabling buffering of the stdout and stderr streams using the -u command-line option. This is because, under certain circumstances, Python’s internal buffering may result in loss of data.

Configuration

The om_exec module accepts the following directives in addition to the common module directives. The Command directive is required.

Required directives

The following directives are required for the module to start.

Command

This mandatory directive specifies the name of the program or script to be executed.

Programs, scripts, and commands are executed under the context of the user running NXLog. When NXLog is running as a service, the service user will be used. If the program, script, or command accesses environment variables, make sure that these are available for the NXLog user.

Optional directives

Arg

This is an optional parameter. Arg can be specified multiple times, once for each argument that needs to be passed to the Command. Note that specifying multiple arguments with one Arg directive, with arguments separated by spaces, will not work (the Command will receive it as one argument).

ExitTimeout

This directive determines the time, in seconds, that the module waits for the child process to terminate. Valid values range from -1 to 54 seconds, with a default of 2 seconds. A value of -1 indicates that the module waits indefinitely for the child process to complete its processing, using the default value of 2 seconds as the waiting time. For further details see how NXLog handles a child process exit.

Restart

Restart the process if it exits. There is a one-second delay before it is restarted to avoid a denial-of-service when a process is not behaving. Looping should be implemented in the script itself. This directive is only to provide some safety against malfunctioning scripts and programs. This boolean directive defaults to FALSE: the Command will not be restarted if it exits.

Child process exit handling

When the module receives a command to stop execution (exit) through a command via the xm_admin module or when the NXLog process exits entirely, the associated child process must terminate gracefully.

Windows operating systems
  1. The module closes the STDIN file handles of the child process.

  2. The module waits for the child process to terminate within the time specified by the ExitTimeout directive.

  3. If the child process continues running after ExitTimeout:

    • If ExitTimeout is not -1, the module terminates the child process using the Windows system function TerminateProcess.

    • If ExitTimeout is -1, the module does not forcefully terminate the child process.

Unix-like operating systems
  1. The module closes the STDIN file handles of the child process.

  2. The module sends a SIGTERM signal to the child process, requesting it to exit gracefully.

  3. The module waits for the child process to terminate within the time specified by the ExitTimeout directive.

  4. If needed, the module sends SIGTERM signals periodically until the child process exits.

  5. If the child process continues running after ExitTimeout:

    • If ExitTimeout is not -1, the module sends a SIGKILL signal to terminate the child process forcibly.

    • If ExitTimeout is -1, the module does not forcefully terminate the child process.

Ensure that the child process handles exceptions correctly when writing to file handlers, regardless of the operating system.

Examples

Example 1. Python 3 script with a graceful exit
import sys
import signal
import time


asked_to_exit = False


def put_exec_logger(st):
    with open("exec-logger.log", "a") as f:
        f.write(st)
        f.flush()


def to_handle(signum, frame):
    global asked_to_exit

    asked_to_exit = True
    auxst = 'om-exec Signal (' + str(signum) + ') received\n'
    put_exec_logger(auxst)


def prepare_and_do_shutdown():
    c = 0
    while c < 10:
        auxst = 'om-exec Asked to finish - waiting 10 seconds before exiting... Counting ' + str(c) + "\n"
        put_exec_logger(auxst)
        time.sleep(1)
        c += 1
    put_exec_logger("om-exec Exiting grancefully" + "\n")
    exit(0)


def core():
    global asked_to_exit

    signal.signal(signal.SIGINT, to_handle)
    signal.signal(signal.SIGTERM, to_handle)

    break_reason = "EOF"
    for line in sys.stdin:
        if asked_to_exit:
            break_reason = "signal"
            break
        with open("om-exec-out.out", "a") as f:
            f.write(line)
            f.flush()
        put_exec_logger(line)

    put_exec_logger("om-exec Exiting reason " + break_reason + "\n")
    prepare_and_do_shutdown()


if __name__ == '__main__':
    core()
Example 2. Piping logs to an external program

This configuration reads logs from a file and writes the log lines to the standard input of an application. The Command directive specifies the path to the application executable and the Arg directive specifies an application argument.

nxlog.conf
<Input log_file>
    Module      im_file
    File        '/path/to/log/file'
</Input>

<Output myapp>
    Module      om_exec
    Command     /path/to/myapp

    # On Windows the path to the application executable
    # should include the file extension.
    #Command    C:\Program Files\MyApp\myapp.exe

    Arg         --level=info
</Output>
Example 3. Piping logs to the standard output

This configuration reads logs from a file and writes the log lines to the standard output using the Linux cat command-line tool. The Command directive specifies the path to the Linux system shell. The first Arg directive passes the -c argument to the shell and the second Arg directive specifies the command to execute.

nxlog.conf
<Input log_file>
    Module      im_file
    File        '/path/to/log/file'
</Input>

<Output stdout>
    Module      om_exec
    Command     /bin/sh
    Arg         -c
    Arg         cat $1
</Output>
Example 4. Executing a Bash script

This configuration reads logs from a file and pipes the log lines to a Bash script. The Command directive specifies the path to a script and the Arg directives specify script arguments.

nxlog.conf
<Input log_file>
    Module    im_file
    File      '/path/to/log/file'
</Input>

<Output bash_script>
   Module     om_exec
   Command    /path/to/logreader.sh
   Arg        -o
   Arg        /path/to/output.log
</Output>
logreader.sh
#!/usr/bin/env bash

LOG=./output.log
PIDF=./pidf
ME=$(basename $0)
echo $$ > $PIDF

while getopts o: flag

do
    case "${flag}" in
        o)  # Path to the output log file
            LOG=${OPTARG};;
        \?) # Invalid option
            echo "Error: Invalid option"
            exit;;
    esac
done

printlog(){
   echo "$ME[$$]:$@" >> $LOG
}

trap_function(){
   rm $PIDF
   exit
}

trap trap_function 15 2

while read LINE; do
   printlog "INPUT:$LINE"
done

trap_function
Example 5. Executing a Python script

This configuration reads logs from a file and pipes the log lines to a Python script. The Command directive specifies the path to the Python executable. The first Arg directive specifies the -u command-line option to disable buffering for the stdout and stderr streams. It is recommended to disable buffering because, under certain circumstances, it may lead to loss of data. The second Arg directive specifies the path to the script, and the following Arg directives specify script arguments.

nxlog.conf
<Input log_file>
    Module    im_file
    File      'C:\Logs\input.log'
</Input>

<Output python_script>
   Module     om_exec
   Command    C:\Python39\python.exe
   Arg        -u
   Arg        C:\Scripts\logreader.py
   Arg        --output-file
   Arg        C:\Logs\output.log
</Output>
logreader.py
import argparse
import sys

arg_parser = argparse.ArgumentParser()
arg_parser.add_argument("--output-file",
                        type=str,
                        required=False,
                        dest="output_file",
                        default="./output.log",
                        help="Path to the output log file")
parsed_args = arg_parser.parse_args()

def read_stdin(file_path):
    with open(file_path, "a") as fh_stream:
        for line in sys.stdin:
            fh_stream.write(line)
            fh_stream.flush()

if __name__ == '__main__':
    read_stdin(parsed_args.output_file)
Example 6. Executing commands under a specific shell

To execute commands under a specific shell, the Command directive should specify the path to the shell executable. The commands to execute can be passed as arguments according to the shell being used. The configuration below executes PowerShell commands from a file. The first Arg directive specifies the path to the script, and the following Arg directives specify script arguments.

nxlog.conf
<Input log_file>
    Module    im_file
    File      'C:\Logs\input.log'
</Input>

<Output powershell_script>
   Module     om_exec
   Command    C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
   Arg        C:\Scripts\logreader.ps1
   Arg        -OutputFile
   Arg        C:\Logs\output.log
</Output>
logreader.ps1
param (
    [string]$OutputFile = "$PSScriptRoot\output.log"
)

while ($line = Read-Host)
{
    $line | Add-Content -Path $OutputFile
}