Files (om_file)
This module can be used to write log messages to a file.
Configuration
The om_file module accepts the following directives in addition to the common module directives. The File directive is required.
Required directives
The following directives are required for the module to start.
This mandatory directive specifies the name of the output file to open. It must be a string type expression. If the expression in the File directive is not a constant string (it contains functions, field names, or operators), it will be evaluated before each event is written to the file (and after the Exec is evaluated). Note that the filename must be quoted to be a valid string literal, unlike in other directives which take a filename argument. For relative filenames, note that NXLog Agent changes its working directory to "/" unless the global SpoolDir is set to something else. Below are three variations for specifying the same output file on a Windows system: File 'C:\logs\logmsg.txt' File "C:\\logs\\logmsg.txt" File 'C:/logs/logmsg.txt'
|
File permissions
The following directives are for specifying permissions for the created socket, pipe, or, file.
This directive specifies the permissions to use for the created socket or pipe or file. This must be a four-digit octal value beginning with a zero. By default, OS default permissions will be set. This directive is not currently supported on Windows. |
|
Use this directive to set the user ownership for the created socket, pipe, or file. By default, this is the user NXLog Agent is running as (which may be specified by the global User directive). This directive is not currently supported on Windows. |
Optional directives
In the case of dynamic filenames, a cache can be utilized to keep files open. This increases performance by reducing the overhead caused by many open/close operations. It is recommended to set this to the number of expected files to be written. Note that this should not be set to more than the number of open files allowed by the system. This caching provides performance benefits on Windows only. Caching is disabled by default. |
|
If set to |
|
Use this directive to set the group ownership for the created socket, pipe, or file. By default, this is the group NXLog Agent is running as, (which may be specified by the global Group directive). This directive is not currently supported on Windows. |
|
See the OutputType directive in the list of common module directives. If this directive is not specified the default is LineBased (the module will use CRLF as the record terminator on Windows, or LF on Unix). This directive also supports data converters, see the description in the OutputType section. |
|
If set to |
|
This optional boolean directive instructs the module to sync the file after each log message is written, ensuring that it is written to disk from the buffers. Because this can hurt performance, the default is FALSE. |
|
This optional boolean directive instructs the module to truncate the file before each write, causing only the most recent log message to be saved.
The default is |
Functions
The following functions are exported by om_file.
- integer
file_size() -
Return the size of the current output file in bytes. Returns undef if the file doesn’t exist. This can happen if no event records have been written to this file yet.
Procedures
The following procedures are exported by om_file.
reopen();-
Reopen the current file. This procedure should be called if the file has been removed or renamed, for example with the file_cycle(), file_remove(), or file_rename(); procedures of the xm_fileop module. This does not need to be called after rotate_to() because that procedure reopens the file automatically.
rotate_to(string filename);-
Rotate the current file to the filename specified. The module will then open the original file specified with the File directive. Note that the rename(2) system call is used internally which does not support moving files across different devices on some platforms. If this is a problem, first rotate the file on the same device. Then use the xm_exec exec_async() procedure to copy it to another device or file system, or use the xm_fileop file_copy() procedure.
Examples
This configuration reads log messages from a socket and writes the messages to a file. No additional processing is done.
<Input uds>
Module im_uds
UDS /dev/log
</Input>
<Output file>
Module om_file
File "/var/log/messages"
</Output>
<Route uds_to_file>
Path uds => file
</Route>| The om_file module can exhaust space on a file system if left on its own without proper rotation. |
This configuration rotates the file if it is larger than 10 MB and no more than two past log files are retained.
<Input tcp>
Module im_tcp
ListenAddr 0.0.0.0:5125
</Input>
<Extension fileop>
Module xm_fileop
</Extension>
define LOCALFILE '/opt/nxlog/var/log/nxlog/local_output'
<Output file>
Module om_file
File %LOCALFILE%
<Schedule>
Every 60 sec
<Exec>
if (file_size() >= 10M)
{
fileop->file_cycle(%LOCALFILE%, 2);
reopen();
}
</Exec>
</Schedule>
</Output>
<Route tcp_to_file>
Path tcp => file
</Route>With this configuration, NXLog accepts log messages via TCP and parses them as BSD Syslog. A separate output file is used for log messages from each host. When the output file size exceeds 15 MB, it will be automatically rotated and compressed.
| Using unsanitized fields for filenames is risky as it exposes the agent to injection attacks. |
<Extension exec>
Module xm_exec
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input tcp>
Module im_tcp
ListenAddr 0.0.0.0:1514
Exec parse_syslog_bsd();
</Input>
<Output file>
Module om_file
File 'tmp/output_' + $SanitizedHostname + '_' + month(now())
<Exec>
# Remove filename illegal characters
$SanitizedHostname = $Hostname;
if $SanitizedHostname =~ s/[\\\/:*?"<>|]//g {};
if (file->file_size() > 15M)
{
$newfile = 'tmp/output_' + $SanitizedHostname + '_' +
strftime(now(), '%Y%m%d%H%M%S');
file->rotate_to($newfile);
exec_async("/bin/bzip2", $newfile);
}
</Exec>
</Output>
<Route tcp_to_file>
Path tcp => file
</Route>