TCP (om_tcp)

This module initiates a TCP connection to a remote host and transfers log messages. Or, in Listen mode, this module accepts client connections and multiplexes data to all connected clients. The TCP transfer protocol provides more reliable log transmission than UDP. If security is a concern, consider using the om_ssl module instead.

To examine the supported platforms, see the list of installation packages.

Configuration

The om_tcp module accepts the following directives in addition to the common module directives. The Host or ListenAddr directive is required.

Required directives

The following directives are required for the module to start.

Use either Host for the connect or ListenAddr for the listen mode.

Host

The module connects to this IP address or hostname. If additional hosts are specified on new lines, the module works in a failover configuration. If a destination becomes unavailable, the module automatically fails over to the next one. If the last destination becomes unavailable, the module fails over to the first destination.

The port number can be defined by appending it to the hostname or IP address using a colon as a separator (host:port). The default port is 514.

IPv6 addresses must be enclosed in square brackets ([addr]:port). For example, [2001:0db8:85a3:0:0:8a2e:0370:7334]:514.

When the Host directive is used with a hostname instead of an IP address, the hostname will be resolved to an IP address for each new connection. If a resolver, e.g. DNS, returns multiple IP addresses, the module will connect to the first IP address. If a single output instance is configured with multiple Host directives or the resolver returns multiple addresses for a name, these hosts are accessed in failover mode. If a Host directive is configured with a hostname, the product performs a name lookup and establishes the connection to the first reachable address in the returned set of addresses. The module will remain connected to that address until it is stopped, or the connection is severed. DNS changes are therefore not picked up by the module without intervention. If the connection fails to the first address of the set, the module will attempt to connect to the next address, until it reaches the end of the set of addresses. Then it performs a lookup on the next Host directive, if so configured. Once all options are exhausted, the module will start over from the first Host directive, cycling through them again until the connection can be re-established.

ListenAddr

The module accepts connections on this IP address or DNS hostname. For security, the default listen address is localhost (the localhost loopback address is not accessible from the outside). To receive logs from remote hosts, the address specified here must be accessible.

The port number can be defined by appending it to the hostname or IP address using a colon as a separator (host:port). The default port is 514. IPv6 addresses must be enclosed in square brackets ([addr]:port).

Using the any address 0.0.0.0:514 for IPv4 and [::]:514 for IPv6 connections is common.

You can define this directive multiple times to listen on multiple addresses or ports in a single module instance. Specify IPv4 and IPv6 addresses separately as needed. If you use a DNS name, you should keep the addresses or CNAMEs below 16 to avoid issues caused by DNS response size limits.

Listen-mode directives

The following directives are available when using the ListenAddr directive.

AllowIP

This optional directive can be used to allow IP addresses and/or networks to connect. The directive can be set multiple times to add different IPs or networks to allow. This directive is only active when the ListenAddr directive is present. In the absence of this directive, the BlockIP directive is considered. If both AllowIP and BlockIP are absent, then hosts are not restricted from connecting to a listening module.

The following formats may be used for the AllowIP directive:

0.0.0.0 (IPv4 address)
0.0.0.0/32 (IPv4 network with subnet bits)
0.0.0.0/0.0.0.0 (IPv4 network with subnet address)
aa::1 (IPv6 address)
aa::12/64 (IPv6 network with subnet bits)

BlockIP

This optional directive can be used to deny IP addresses and/or networks to connect. The directive can be set multiple times to add different IPs or networks to deny. This directive is only active when the ListenAddr directive is present. In the absence of this directive, the AllowIP directive is considered. If both AllowIP and BlockIP are absent, then hosts are not restricted from connecting to a listening module.

The following formats may be used for the BlockIP directive:

0.0.0.0 (IPv4 address)
0.0.0.0/32 (IPv4 network with subnet bits)
0.0.0.0/0.0.0.0 (IPv4 network with subnet address)
aa::1 (IPv6 address)
aa::12/64 (IPv6 network with subnet bits)

MaxConnections

With this optional directive it is possible to set the maximum number of allowed concurrent/active connections for a listening TCP socket. If not specified, the default value is 4294967295, unlimited. When the limit is reached, the incoming connection will be rejected and an error message is shown in the selflog

2024-03-01 22:29:16 ERROR [im_tcp|in_tcp] Number of allowed active connections(10) reached: 10. Refusing connection from 127.0.0.1

ConnectionIdleTimeout

This optional directive defines the maximum time in seconds before NXLog Agent closes TCP connections without traffic. The minimum timeout value is 15 seconds. If this directive is not specified, NXLog Agent does not close idle TCP connections.

ExclusiveAddrUse

This optional boolean directive specifies whether the module instance should exclusively bind to the specified port. The default value is FALSE; multiple module instances can bind to the same port.

This directive is only supported on Windows platforms.

ReuseAddr

This optional boolean directive determines whether the module instance should forcibly bind to a port already in use. The default value is TRUE; multiple instances can listen on the same port and process data simultaneously.

ReusePort

This optional boolean directive specifies whether multiple om_tcp module instances can listen on the same port. When you enable this directive, multiple instances run in a separate thread, allowing NXLog Agent to process incoming logs simultaneously. See the examples below. The default value is FALSE; multiple instances cannot bind to the same port.

This directive is not supported on Windows platforms.

Optional directives

LocalPort

This optional directive specifies the local port number of the connection. This directive only applies if Listen is set to FALSE. If this is not specified, a random high port number will be used, which is not always ideal in firewalled network environments.

Due to the required TIME-WAIT delay in closing connections, attempts to bind to LocalPort may fail. In such cases, the message Address already in use will be written to nxlog.log. If the situation persists, it could impede network performance.

OutputType

See the OutputType directive in the list of common module directives. The default is LineBased_LF.

QueueInListenMode

If set to TRUE, this boolean directive specifies that events should be queued if no client is connected. If this module’s buffer becomes full, the preceding module in the route will be paused or events will be dropped, depending on whether FlowControl is enabled. This directive only applies if Listen is set to TRUE. The default is FALSE: om_tcp will discard events if no client is connected.

Reconnect

This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and double with every attempt. If the duration of the successful connection is greater than the current reconnect interval, then the reconnect interval will be reset to 1 sec.

The Reconnect directive must be used with caution. If it is used on multiple systems, it can send reconnect requests simultaneously to the same destination, potentially overloading the destination system. It may also cause NXLog Agent to use unusually high system resources or cause NXLog Agent to become unresponsive.

ReconnectOnData

This optional directive defines the behavior when the connection with the remote host is lost. When set to TRUE, the module only attempts to reconnect when it has data to send. The default value is FALSE; it will always keep a connection open with the remote host.

TCPNoDelay

This boolean directive is used to turn off the network optimization performed by Nagle’s algorithm. Nagle’s algorithm is a network optimization tweak that tries to reduce the number of small packets sent out to the network, by merging them into bigger frames, and by not sending them to the other side of the session before receiving the ACK. If this directive is unset, the TCP_NODELAY socket option will not be set.

TCP connections are kept alive by keep-alive packets. This feature is enabled by default and cannot be disabled.

Procedures

The following procedures are exported by om_tcp.

reconnect();

Force a reconnection. This can be used from a Schedule block to periodically reconnect to the server.

The reconnect() procedure must be used with caution. If configured, it can attempt to reconnect after every event sent, potentially overloading the destination system.

Examples

Example 1. Sending logs over TCP

With this configuration, NXLog Agent will read log messages from a socket and forward them via TCP.

nxlog.conf

<Input uds>
    Module  im_uds
    UDS     /dev/log
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.1.1:1514
</Output>

<Route uds_to_tcp>
    Path    uds => tcp
</Route>

Example 2. Sending logs over TCP with failover

This configuration sends logs via TCP in a failover configuration (multiple Hosts defined).

nxlog.conf

<Output tcp>
    Module  om_tcp
    Host    192.168.1.2:1514
    Host    192.168.1.3:1234
    Host    example.com:1234
</Output>