External programs (im_exec)
This module will execute a program or script on startup and read its standard output. It can be used to easily integrate with exotic log sources which can be read only with the help of an external script or program.
| To examine the supported platforms, see the list of installer packages in the Available Modules chapter. |
|
If you are using a Perl script, consider using im_perl instead or turning on Autoflush with If you are using a Python script, we recommend disabling buffering of the |
Configuration
The im_exec module accepts the following directives in addition to the common module directives. The Command directive is required.
Required directives
The following directives are required for the module to start.
This mandatory directive specifies the name of the program or script to be executed.
|
Optional directives
This is an optional parameter. Arg can be specified multiple times, once for each argument that needs to be passed to the Command. Note that specifying multiple arguments with one Arg directive, with arguments separated by spaces, will not work (the Command would receive it as one argument). |
|
See the InputType description in the global module configuration section. |
|
Restart the process if it exits. There is a one second delay before it is restarted to avoid a denial-of-service when a process is not behaving. Looping should be implemented in the script itself, this directive is only to provide some safety against malfunctioning scripts and programs. This boolean directive defaults to FALSE: the Command will not be restarted if it exits. |
Creating and populating fields
im_exec populates the $raw_event core field with the log message read from the program or script’s standard output.
Further processing of this field can be done to parse the message into structured data or convert it to a different output format, such as JSON or XML.
See Parsing and converting log records below for an example and Parsing various log formats in the NXLog User Guide for more information on parsing log records.
Examples
This configuration uses the Linux tail command-line tool to read lines from a log file.
The first Arg directive specifies the -f argument, which means that tail should monitor the file for new lines.
The second Arg directive specifies the path of the log file.
This is equivalent to executing the following command:
$ tail -f /var/log/messages| The im_file module should be used to read log messages from files. This example is only intended to demonstrate use of the im_exec module. |
<Input messages>
Module im_exec
Command /usr/bin/tail
Arg -f
Arg /var/log/messages
</Input>This configuration executes an application to read logs from a third-party source. The Command directive specifies the path to the application executable and the Arg directive specifies an application argument. This is equivalent to executing the following command:
$ /path/to/myapp --level=info<Input myapp>
Module im_exec
Command /path/to/myapp
# On Windows the path to the application executable
# should include the file extension.
#Command C:\Program Files\MyApp\myapp.exe
Arg --level=info
</Input>This configuration executes a Python script to read logs from a third-party source.
The Command directive specifies the path to the Python executable.
The first Arg directive specifies the -u command-line option to disable buffering for the stdout and stderr streams.
It is recommended to disable buffering because it may lead to a delay in receiving the logs.
The second Arg directive specifies the path to the script.
This is equivalent to executing the following command:
> python -u C:\Scripts\myscript.py<Input python_script>
Module im_exec
Command C:\Python39\python.exe
Arg -u
Arg C:\Scripts\myscript.py
</Input>To execute commands under a specific shell, the Command directive should specify the path to the shell executable. The commands to execute can be passed as arguments according to the shell being used. The configuration below executes PowerShell commands from a file.
<Input powershell_script>
Module im_exec
Command C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Arg C:\Scripts\myscript.ps1
</Input>This configuration executes a script and parses the $raw_event field with a regular expression.
If the regular expression matches, fields are created according to the captured groups, otherwise the log record is dropped.
Finally, the record is converted to JSON format using the to_json() procedure of the xm_json module.
<Extension json>
Module xm_json
</Extension>
<Input powershell_script>
Module im_exec
Command C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Arg C:\Scripts\myscript.ps1
<Exec>
if $raw_event =~ /(?x)^(\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d\+\d\d:\d\d),
(.+),(.+)$/
{
$EventTime = parsedate($1);
$Severity = $2;
$Message = $3;
}
else
{
drop();
}
to_json();
</Exec>
</Input>2021-11-05T14:03:40+01:00,INFO,The service started successfully{
"EventReceivedTime": "2021-11-05T14:04:24.244343+01:00",
"SourceModuleName": "powershell_script",
"SourceModuleType": "im_exec",
"EventTime": "2021-11-05T14:03:40.000000+01:00",
"Severity": "INFO",
"Message": "The service started successfully"
}
