Configuring NXLog Manager for cluster mode

For load balancing, it is possible to run multiple instances of NXLog Manager so that a group of agents connect to a specific Manager instance, yet all agents can still be managed from any available NXLog Manager instance. This mode is referred to as distributed mode or cluster mode.

Cluster mode is not meant for ensuring high availability. Agents will not automatically reconnect to another instance if their original instance fails. In addition, all NXLog Manager instances connect to a single SQL database (DB) that is not highly available.

Prerequisites

Ensure that you have at least one NXLog Manager instance installed, running, and successfully managing agents before you configure cluster mode. This includes installing and initializing the SQL DB that all other instances will also connect to.

You can install the rest of the NXLog Manager instances before you begin or after the initial cluster configuration.

Reconfiguring the SQL database

For cluster mode, you need to reconfigure the SQL DB that you initialized during the first NXLog Manager instance installation process.

Follow these steps to allow the nxlog-manager5 user to connect from multiple hosts:

Rename the nxlog-manager5 user:

> RENAME USER 'nxlog-manager5'@'localhost' TO 'nxlog-manager5'@'%';
Query OK, 0 rows affected (0.002 sec)

Verify that the user is correctly renamed.

The Host column must show % for the nxlog-manager5 user:

> select Host,User from mysql.user;
+-----------+----------------+
| Host      | User           |
+-----------+----------------+
| %         | nxlog-manager5 |
| localhost | mariadb.sys    |
| localhost | mysql          |
| localhost | root           |
+-----------+----------------+
4 rows in set (0.001 sec)

Follow these steps to allow external connections:

Open /etc/my.cnf for editing.

Run whereis my.cnf to find the configuration file’s location if it is not at its default location. Your OS might also include other files into the main file instead of specifying the options directly. Follow the include directive if that is the case.
In the [mysqld] section, follow these steps:
- Ensure that skip-networking is either absent or prefixed with the hash sign (#) to disable the setting.

#skip-networking

+ ** Set skip-bind-address:

skip-bind-address

+ . Restart your SQL server: ** MariaDB:

# systemctl restart mariadb

+ ** MySQL:

# systemctl restart mysqld

+ The service name might differ on your OS.

These settings enable your SQL DB to accept remote client connections on the default port 3306. If you need to change the port number check the official documentation for your DB.

Configuring the NXLog Manager instances

NXLog Manager instances communicate with each other using the JMS (Java Message Service) API on port 20'000. They also need to connect to the same SQL DB as they rely on DB records to learn about each other.

On each NXLog Manager instance machine, follow these steps to enable cluster mode:

Stop the NXLog Manager service:
```
# systemctl stop nxlog-manager
```
Open /opt/nxlog-manager/conf/nxlog-manager.conf and set the following:
```
INSTANCE_MODE=distributed-manager
```
Open /opt/nxlog-manager/conf/jetty-env.xml and do the following updates:
- Substitute the public IP address of the interface used for communication with the other instances for the link-local address 127.0.0.1. For example:
  <Set name="jmsBrokerAddress">10.0.0.42</Set>
- Ensure that the JDBC driver points to your SQL DB:
  <Set name="jdbcUrl">jdbc:mysql://10.0.0.42:3306/nxlog-manager5?useUnicode=true&characterEncoding=UTF-8&characterSetResults=UTF-8&autoReconnect=true&useServerPrepStmts=false</Set>
  Replace 10.0.0.42 with the IP address of your SQL server and 3306 with the actual port number if your DB is listening on a non-standard port.
Start the NXLog Manager service:
```
# systemctl start nxlog-manager
```
Log in to the NXLog Manager UI and verify that it was able to connect to the database successfully.

Adding instances to a cluster

Follow these steps to add a new NXLog Manager instance to a running cluster:

Install the instance without running the DB initialization step.
Stop the NXLog Manager service:
```
# systemctl stop nxlog-manager
```
Update the NXLog Manager configuration for cluster mode as specified in Configuring the NXLog Manager instances.
Start the NXLog Manager service:
```
# systemctl start nxlog-manager
```
Log in to the NXLog Manager UI and check that you see the same data available as in the other instances.
Add agents to the new NXLog Manager instance as you normally would.

Making configuration changes in cluster mode

In cluster mode, all NXLog Manager instances use a single database and share the same settings. Changes that you make from the UI on one instance affect all other instances. When you update Agent Manager connection parameters, restart the NXLog Manager service on all nodes for the changes to be applied.

Because all NXLog Manager instances share the same configuration, the Agent Manager needs to listen on all available network interfaces. You can configure this setting from the NXLog Manager UI by navigating to Admin > Settings > Agent Manager and setting the Listen address field to 0.0.0.0.

When you change NXLog Manager settings directly in the configuration files, such as the heap size or the open file limit, you must apply them on each instance individually.