Use dynamic output filenames

As an alternative to traditional log file rotation, you can define dynamic output filenames. To do this, specify the value of the om_file File directive as a string type expression built using any available fields and functions, such as log event attributes or the current date and time.

Writing to dynamic filenames could result in log events written to multiple files with partial arbitrary names. Because of this, such configurations are not suitable if a third-party application expects to read logs from a file with a static name. We recommend you use file rotation in such cases.

Use event attributes in filenames

You can build dynamic filenames with event attributes such as $EventTime or $Hostname. The following examples illustrate organizing events into separate log files by hostname or day.

Example 1. Using a string attribute in the filename

This configuration receives syslog messages over UDP and parses records into structured data using the parse_syslog() procedure of the xm_syslog module. This procedure sets the $Hostname field from the syslog header. The configuration also uses the $Hostname field to set the File directive of the om_file instance.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:514
    Exec          parse_syslog();
</Input>

<Output output_file>
    Module        om_file
    File          '/var/logs/' + $Hostname
</Output>

Example 2. Using a datetime attribute in the filename

This configuration receives syslog messages over UDP and parses records into structured data using the parse_syslog() procedure of the xm_syslog module. This procedure sets the $EventTime field from the syslog header. The configuration also uses the $EventTime field to set the File directive of the om_file instance.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:514
    Exec          parse_syslog();
</Input>

<Output output_file>
    Module        om_file
    File          '/var/log/nxlog-out-' + strftime($EventTime, '%Y-%m-%d')  (1)
</Output>

1	The strftime() function converts the datetime value to a string in the specified format.

Use timestamps in filenames

You can retrieve the current date and time with NXLog Agent using the now() function. You can then use the timestamp to organize output log files by a unit of time.

Example 3. Rotating log files to a nested directory structure

In this example, we’ll organize logs into folders by year and month, as shown below.

/var/logs/<YEAR>/<MONTH>/<YYYY-MM-DD>.log

The output instance in this configuration defines a Schedule to rotate files daily. When the schedule runs, it creates the necessary folder structure and rotates the current output log file accordingly.

nxlog.conf

<Extension fileop>
    Module    xm_fileop
</Extension>

<Output output_file>
    define OUTPUT_DIR /var/logs

    Module    om_file
    File      '%OUTPUT_DIR%/nxlog-out.log'
    <Schedule>
        When  @daily
        <Exec>
            # Create the directories if necessary
            dir_make('%OUTPUT_DIR%/' + strftime(now(), '%Y/%m')); (1) (2)

            # Rotate the current output file into the correct directory
            rotate_to('%OUTPUT_DIR%/' + strftime(now(), '%Y/%m/%Y-%m-%d.log'));  (3)
        </Exec>
    </Schedule>
</Output>

1	The xm_fileop dir_make() procedure creates the specified directory structure if it doesn’t exist.
2	The strftime() function converts the datetime value to a string in the specified format.
3	The om_file rotate_to() procedure renames the current output file to the specified filename. After, the module recreates the output file specified by the File directive and continues writing to it.