TCP (om_tcp)

This module initiates a TCP connection to a remote host and transfers log messages. Or, when using the ListenAddr directive, this module accepts client connections and multiplexes data to all connected clients. The TCP transfer protocol provides more reliable log transmission than UDP. If security is a concern, consider using the om_ssl module instead.

To examine the supported platforms, see the list of installation packages.

Configuration

The om_tcp module accepts the following directives in addition to the common module directives. The Host or ListenAddr directive is required.

Required directives

One of the following mutually exclusive directives is required for the module to start.

Host

The module connects to this IP address or hostname. If using a hostname, the module resolves the hostname to an IP address on each new connection.

You can define the port number by appending it to the IP address or hostname using a colon as a separator (host:port). The default port is 514.

IPv6 addresses must be enclosed in square brackets ([addr]:port). For example, [2001:0db8:85a3:0:0:8a2e:0370:7334]:514.

You can define this directive multiple times to connect to multiple hosts or ports in failover mode. Specify IPv4 and IPv6 addresses separately as needed. If you use a DNS name, you should keep the addresses or CNAMEs below 16 to avoid issues caused by DNS response size limits.

If the module is configured with multiple Host directives or a resolver, such as DNS, returns multiple addresses, the module will connect to the first reachable address. The module will remain connected to that address until it is stopped or the connection is severed. DNS changes are therefore not picked up by the module without intervention.

If the connection to an address fails, the module will attempt to connect to the next address until it reaches the end of the set of addresses for the same Host directive. Then, it performs a lookup on the next Host directive, if configured. Once all options are exhausted, the module will start over from the first address of the first Host directive until the connection can be re-established.

ListenAddr

Set this directive to the IP address or DNS hostname on which the module accepts connections. The default is localhost, which restricts access to the local machine and prevents connections from remote hosts.

Append the port number to the address using a colon separator (host:port). The default port is 514. For IPv6 addresses, enclose the address in square brackets ([addr]:port). Use 0.0.0.0:514 for IPv4 or [::]:514 for IPv6 to accept connections on all interfaces. NXLog Agent requires additional privileges to bind to ports below 1024 on Linux. See Granting additional Linux capabilities for more information.

You can specify this directive multiple times to accept connections on multiple addresses or ports within a single module instance. Set IPv4 and IPv6 addresses as separate entries. If you use a DNS name, keep the number of addresses or CNAMEs below 16 to avoid errors caused by DNS response size limits.

Listen-mode directives

The following directives are available when using the ListenAddr directive.

AllowIP

Set this directive to restrict incoming connections to specific IP addresses or networks. You can specify this directive multiple times to allow multiple IPs or networks. AllowIP has no effect unless ListenAddr is configured.

If AllowIP is not set, the module falls back to BlockIP to deny specific connections. If neither AllowIP nor BlockIP is set, the module does not restrict incoming connections.

The following IP address formats may be used:

0.0.0.0 (IPv4 address)
0.0.0.0/32 (IPv4 network with subnet bits)
0.0.0.0/0.0.0.0 (IPv4 network with subnet address)
aa::1 (IPv6 address)
aa::12/64 (IPv6 network with subnet bits)

BlockIP

Set this directive to deny incoming connections from specific IP addresses or networks. You can specify this directive multiple times to deny multiple IPs or networks. BlockIP has no effect unless ListenAddr is configured.

If BlockIP is not set, the module falls back to AllowIP to restrict incoming connections. If neither BlockIP nor AllowIP is set, the module does not restrict incoming connections.

The following IP address formats may be used:

0.0.0.0 (IPv4 address)
0.0.0.0/32 (IPv4 network with subnet bits)
0.0.0.0/0.0.0.0 (IPv4 network with subnet address)
aa::1 (IPv6 address)
aa::12/64 (IPv6 network with subnet bits)

MaxConnections

Set this directive to limit the number of concurrent active connections for a listening TCP socket. The default is 4294967295 (unlimited). When the limit is reached, the module rejects new connections and logs an error:

Number of allowed active connections(10) reached: 10. Refusing connection from 127.0.0.1

ConnectionIdleTimeout

Set this directive to close TCP connections that have been idle for longer than the specified number of seconds. The minimum value is 15 seconds. If this directive is not set, the module keeps idle TCP connections open indefinitely.

ExclusiveAddrUse

Set this directive to TRUE to prevent other processes or module instances from binding to the same port.

The default is FALSE; multiple module instances can bind to the same port.

This directive is only supported on Windows.

ReuseAddr

Set this directive to TRUE to allow the module instance to bind to a port that is already in use.

The default is TRUE; multiple module instances can listen on the same port and process data simultaneously.

ReusePort

Set this directive to TRUE to allow multiple module instances to listen on the same port. Each instance runs in a separate thread, allowing NXLog Agent to process incoming logs simultaneously.

The default is FALSE; multiple module instances cannot bind to the same port.

This directive is not supported on Windows.

Optional directives

LocalPort

This optional directive specifies the local port number of the connection. If this is not specified, a random high port number will be used, which is not always ideal in firewalled network environments. This directive only applies if Host is used instead of ListenAddr.

Due to the required TIME-WAIT delay in closing connections, attempts to bind to LocalPort may fail. In such cases, the message Address already in use will be written to nxlog.log. If the situation persists, it could impede network performance.

OutputType

See the OutputType directive in the list of common module directives. The default is LineBased_LF.

QueueInListenMode

If set to TRUE, this boolean directive specifies that events should be queued if no client is connected. If this module’s buffer becomes full, the preceding module in the route will be paused or events will be dropped, depending on whether FlowControl is enabled. This directive only applies if ListenAddr is used instead of Host. The default is FALSE: om_tcp will discard events if no client is connected.

Reconnect

This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and double with every attempt. In the latter case, when the system decides that the reconnection is successful, the reconnect interval is immediately reset to 1 sec.

The Reconnect directive must be used with caution. If it is used on multiple systems, it can send reconnect requests simultaneously to the same destination, potentially overloading the destination system. It may also cause NXLog Agent to use unusually high system resources or cause NXLog Agent to become unresponsive.

ReconnectOnData

This optional directive defines the behavior when the connection with the remote host is lost. When set to TRUE, the module only attempts to reconnect when it has data to send. The default value is FALSE; it will always keep a connection open with the remote host.

TCPNoDelay

This boolean directive is used to turn off the network optimization performed by Nagle’s algorithm. Nagle’s algorithm is a network optimization tweak that tries to reduce the number of small packets sent out to the network, by merging them into bigger frames, and by not sending them to the other side of the session before receiving the ACK. If this directive is unset, the TCP_NODELAY socket option will not be set.

TcpSendStallDetect

Use this directive to log a warning when the module cannot send data to the destination, allowing you to monitor pipeline backpressure. When set to TRUE, the module logs the following message to the NXLog Agent log file: TCP send buffer full for <instance_name> - write blocked. The default is FALSE.

TCP connections are kept alive by keep-alive packets. This feature is enabled by default and cannot be disabled.

Procedures

The following procedures are exported by om_tcp.

reconnect();

Force a reconnection. This can be used from a Schedule block to periodically reconnect to the server.

The reconnect() procedure must be used with caution. If configured, it can attempt to reconnect after every event sent, potentially overloading the destination system.

Examples

Example 1. Sending logs over TCP

With this configuration, NXLog Agent will read log messages from a socket and forward them via TCP.

nxlog.conf

<Input uds>
    Module  im_uds
    UDS     /dev/log
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.1.1:1514
</Output>

<Route uds_to_tcp>
    Path    uds => tcp
</Route>

Example 2. Sending logs over TCP with failover

This configuration sends logs via TCP in a failover configuration (multiple Hosts defined).

nxlog.conf

<Output tcp>
    Module  om_tcp
    Host    192.168.1.2:1514
    Host    192.168.1.3:1234
    Host    example.com:1234
</Output>