Configure load balancing

Several NXLog Agent modules include self-managed failover functionality. See NXLog Agent failover for a further explanation of failover.

Below, we provide examples to help you set up an NXLog Agent relay cluster with load balancing and automatic failover. Such a configuration distributes the log processing load between the cluster members, prevents idle nodes, and provides failover in case any nodes fail.

Create a load-balancing cluster with failover

The following diagram illustrates an NXLog Agent relay cluster receiving logs from different sources. Each log source sends logs to a different primary node in the cluster with failover to the other nodes.

With this setup, log sources communicate with the NXLog Agent relay cluster in an active/passive model, but the cluster operates as a static active/active load balancer. This technique minimizes the chance of idle cluster nodes, ensuring the efficient use of each node and boosting performance.

NXLog Agent cluster with load balancing and failover

Figure 1. NXLog Agent load balancing and automatic failover mode

The following steps guide you in configuring an NXLog Agent cluster and forwarding logs to it as described above. See Configure an NXLog Agent relay for more configuration examples.

Configure the NXLog Agent cluster nodes

The first step is configuring each NXLog Agent node in the relay cluster to receive logs from your log sources and forward them to their destination.

Example 1. Configuring cluster nodes

This configuration uses an im_batchcompress module instance to receive logs from other NXLog Agent instances. It then forwards the logs to a SIEM over TCP using the om_tcp output module.

nxlog.conf

<Input nxlog_agent>
    Module        im_batchcompress
    ListenAddr    0.0.0.0:1514
</Input>

<Output siem>
    Module        om_tcp
    Host          siem.example.com:1514
</Output>

<Route relay>
    Path          nxlog_agent =>  siem
</Route>

Forward logs to the cluster

Once your NXLog Agent cluster is up and running, you can configure your log sources to forward logs to it with the om_batchcompress output module.

Example 2. Forwarding logs to an NXLog Agent cluster with failover

Configure your first log source (the Linux and macOS nodes in the diagram above) to send logs to the first NXLog Agent cluster node and failover to the other two nodes.

nxlog.conf

<Output agent_relay>
    Module    om_batchcompress

    Host      192.168.1.51:1514 # Active
    Host      192.168.1.52:1514 # Passive
    Host      192.168.1.53:1514 # Passive
</Output>

Configure your second log source (the DNS Server nodes in the diagram above) to send logs to the second NXLog Agent cluster node and failover to the other two nodes.

nxlog.conf

<Output agent_relay>
    Module  om_batchcompress

    Host    192.168.1.52:1514 # Active
    Host    192.168.1.53:1514 # Passive
    Host    192.168.1.51:1514 # Passive
</Output>

Finally, configure your third log source (the Sysmon node in the diagram above) to send logs to the third NXLog Agent cluster node and failover to the other two nodes.

nxlog.conf

<Output agent_relay>
    Module  om_batchcompress

    Host    192.168.1.53:1514 # Active
    Host    192.168.1.51:1514 # Passive
    Host    192.168.1.52:1514 # Passive
</Output>