Redis (om_redis)

This module can store data in a Redis server. It issues RPUSH commands using the Redis Protocol to send data.

The input counterpart, im_redis, can be used to retrieve data from a Redis server.

Configuration

The om_redis module accepts the following directives in addition to the common module directives. The Host directive is required.

Required directives

The following directives are required for the module to start.

Host

This mandatory directive specifies the IP address or DNS hostname of the Redis server to connect to.

Optional directives

Channel

This directive is interpreted the same way as the Key directive, i.e., it can be an expression that evaluates to a string. The module will publish records to a Redis channel with this name. The default is nxlog. The Command directive must be set to PUBLISH when using this directive.

Command

This optional directive specifies the command to be used. The possible commands are LPUSH, RPUSH (the default), LPUSHX, RPUSHX, and PUBLISH.

Key

This specifies the Key used by the RPUSH command. It must be a string type expression. If the expression in the Key directive is not a constant string (it contains functions field names, or operators), it will be evaluated for each event to be inserted. The default is nxlog. The usage of this directive is mutually exclusive with the usage of the PUBLISH command in the Command directive.

LocalPort

This optional directive specifies the local port number of the connection. If this is not specified, a random high port number will be used, which is not always ideal in firewalled network environments.

Due to the required TIME-WAIT delay in closing connections, attempts to bind to LocalPort may fail. In such cases, the message Address already in use will be written to nxlog.log. If the situation persists, it could impede network performance.

OutputType

See the OutputType directive in the list of common module directives. If this directive is not specified, the default Dgram formatter function is used, which writes the value of $raw_event without a line terminator. You can use Binary to preserve structured data if the logs are intended to be read from Redis by another NXLog Agent instance.

Port

This specifies the port number of the Redis server. The default is port 6379.

Reconnect

This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and double with every attempt. If the duration of the successful connection is greater than the current reconnect interval, then the reconnect interval will be reset to 1 sec.

The Reconnect directive must be used with caution. If it is used on multiple systems, it can send reconnect requests simultaneously to the same destination, potentially overloading the destination system. It may also cause NXLog Agent to use unusually high system resources or cause NXLog Agent to become unresponsive.

ReconnectOnData

This optional directive defines the behavior when the connection with the remote host is lost. When set to TRUE, the module only attempts to reconnect when it has data to send. The default value is FALSE; it will always keep a connection open with the remote host.

Procedures

The following procedures are exported by om_redis.

reconnect();

Force a reconnection. This can be used from a Schedule block to periodically reconnect to the server.

The reconnect() procedure must be used with caution. If configured, it can attempt to reconnect after every event sent, potentially overloading the destination system.

Examples

Example 1. Sending logs to Redis with the default configuration

This configuration only sets the mandatory Host directive to the Redis server IP address. The module default will use the RPUSH command to write the logs to a key named nxlog.

nxlog.conf

<Input test_input>
    Module      im_testgen
    MaxCount    2
</Input>

<Output redis>
    Module      om_redis
    Host        192.168.1.213
</Output>

Output sample

127.0.0.1:6379> lrange nxlog 0 -1
1) "0@Wed Jul 13 16:47:59 2022"
2) "1@Wed Jul 13 16:47:59 2022"

Example 2. Sending logs to Redis with a custom command and key

The configuration below sets the mandatory Host directive to the Redis server IP address. It also specifies the Command and Key directives to use the LPUSH command for writing logs to a key named event_logs.

nxlog.conf

<Input test_input>
    Module      im_testgen
    MaxCount    2
</Input>

<Output redis>
    Module      om_redis
    Host        192.168.1.213
    Command     LPUSH
    Key         'event_logs'
</Output>

Output sample

127.0.0.1:6379> lrange event_logs 0 -1
1) "1@Wed Jul 13 16:52:39 2022"
2) "0@Wed Jul 13 16:52:39 2022"

Example 3. Publishing logs to a custom channel

The configuration below sets the mandatory Host directive to the Redis server IP address. It also specifies the Command and Channel directives for publishing logs to the event_logs Redis channel.

nxlog.conf

<Input test_input>
    Module      im_testgen
    MaxCount    2
</Input>

<Output redis>
    Module      om_redis
    Host        192.168.1.213
    Command     PUBLISH
    Channel     'event_logs'
</Output>

Output sample

127.0.0.1:6379> subscribe event_logs
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "event_logs"
3) (integer) 1
1) "message"
2) "event_logs"
3) "0@Wed Jul 13 17:00:56 2022"
1) "message"
2) "event_logs"
3) "1@Wed Jul 13 17:00:56 2022"