Configuring NXLog Manager for cluster mode

It is possible to run multiple instances of NXLog Manager so that a group of agents connect to a specific Manager instance, yet all agents can still be managed from any available NXLog Manager instance. This mode is referred to as distributed mode or cluster mode.

The following needs to be set in the /opt/nxlog-manager/conf/nxlog-manager.conf configuration file on each instance:

nxlog-manager.conf

INSTANCE_MODE=distributed-manager

NXLog Manager instances communicate with each other using the JMS (Java Message Service) API. Please set the public IP address of the interface in /opt/nxlog-manager/conf/jetty-env.xml and make sure to replace the value 127.0.0.1 set for JMSBrokerAddress with the public IP:

jetty-env.xml

<Set name="jmsBrokerAddress">10.0.0.42</Set>

To operate in clustered mode, NXLog Manager requires MariaDB Galera Cluster. For more information on Galera Cluster refer to the MariaDB Galera Cluster documentation and the Getting Started with MariaDB Galera Cluster guide. The following sections show how to migrate from a standalone instance to cluster mode and how to install MariaDB Galera Cluster.

Migrating from standalone mode to cluster mode

Follow these steps to migrate an existing NXLog Manager standalone installation to cluster mode and add more instances to the cluster.

Migrate the existing installation:

Stop the NXLog Manager service:
```
# systemctl stop nxlog-manager
```
Update the NXLog Manager configuration for cluster mode as specified in Migrating from standalone mode to cluster mode.
Stop the MariaDB service:
```
# systemctl stop mariadb
```
Install and configure MariaDB Galera Cluster according to your version of MariaDB. Refer to Installing MariaDB Galera Cluster for instructions.
Start the NXLog Manager service:
```
# systemctl start nxlog-manager
```
Log in to the NXLog Manager UI and verify that it was able to connect to the database successfully.

To add another instance to the cluster:

Install and configure MariaDB Galera Cluster. Refer to Installing MariaDB Galera Cluster for instructions. The same MariaDB version as the existing instance needs to be installed.
If the IP of the new instance has not already been added, update the MariaDB Galera Cluster configuration on the first instance and restart the MariaDB service.
Install NXLog Manager according to your OS. Refer to the instructions in the Migrating from standalone mode to cluster mode section.
Stop the NXLog Manager service:
```
# systemctl stop nxlog-manager
```
Update the NXLog Manager configuration for cluster mode as specified in Migrating from standalone mode to cluster mode.
Start the NXLog Manager service:
```
# systemctl start nxlog-manager
```
Log in to the NXLog Manager UI and you should see the same data available as in the first instance.
Add agents to the new NXLog Manager instance as you normally would. Refer to Migrating from standalone mode to cluster mode for instructions.

Further considerations for cluster mode:

In cluster mode all NXLog Manager instances use a single database and share the same settings. Changes made from the UI on one instance affect all other instances. When Agent Manager connection parameters are updated, it is advised to restart the NXLog Manager service on all nodes for the changes to be applied.
Since all NXLog Manager instances share the same configuration, the Agent Manager needs to be configured to listen on all available network interfaces. This setting can be configured from the NXLog Manager UI by navigating to Admin > Settings > Agent Manager and setting the Listen address field to 0.0.0.0.
NXLog Manager settings that can only be updated by editing configuration files, such as changing the heap size or open file limit, will need to be applied on each instance individually.

Installing MariaDB Galera Cluster

Install the MariaDB Galera Cluster package according to your MariaDB version and OS:
On each node in the cluster, edit the MariaDB option file to add the Galera Cluster settings. See the documentation on Configuring MariaDB with Option Files if you’re not sure where to find the file for your installation. The following is a sample configuration:
```
[mysqld]

# Galera Cluster Configuration
wsrep_on = ON
wsrep_provider = /usr/lib/galera/libgalera_smm.so
wsrep_cluster_address = "gcomm://<IP_OF_NODE_1>,<IP_OF_NODE_2>,<IP_OF_NODE_N>"
wsrep_cluster_name = "NXCluster"
default_storage_engine = InnoDB
binlog_format = row
innodb_autoinc_lock_mode = 2
innodb_force_primary_key = 1
innodb_doublewrite = 1
```
Refer to the MariaDB documentation on Configuring Galera Cluster for more information on the required settings.
Stop MariaDB on all nodes:
```
# systemctl stop mariadb
```
Start the cluster on the first node:
```
# galera_new_cluster
```
Start the MariaDB service on the rest of the nodes:
```
# systemctl start mariadb
```
Verify that all nodes are running by logging in to MariaDB and executing the following command:
```
# mysql -u root -e "show status like 'wsrep_cluster_size';"
```
The command should return a value corresponding to the number of nodes in the cluster.