Control NXLog Agent RAM and CPU usage

NXLog Agent is designed to collect and process logs as fast as possible. A side-effect of this may be high resource usage when you have many events to be processed. You can use operating system-level tools to limit resource consumption by NXLog Agent. However, this will also introduce a delay in log processing. Implementing a limit on resource usage is a good option when a log processing delay is not a problem or to deal with intermittent high loads since the delay will only occur for a short period. When dealing with a consistently high load and introducing a log processing delay is not desirable, you may need to consider other solutions, such as splitting the load between multiple NXLog Agent instances.

Below, we provide some options for setting up control methods that prevent NXLog Agent from consuming more than the designated amount of RAM and CPU cycles. Different approaches and tools apply according to your operating system.

Linux-based systems

Using cpulimit

cpulimit is a command-line tool that you can use to limit the CPU usage of a process. You can use a process manager like Supervisor to start cpulimit automatically and execute it as a long-running process. Both tools are available in most of the default repositories of Unix-like distributions.

Follow these steps to install and configure cpulimit and Supervisor:

Install both tools according to your platform.
- On RHEL/CentOS, enable the EPEL Repository:
  # yum install epel-release
  Then execute the following command to install cpulimit and Supervisor:
  # yum install cpulimit supervisor
- For Debian/Ubuntu, execute the following command:
  # apt install cpulimit supervisor
Open the Supervisor configuration file located at /etc/supervisor/supervisord.conf with a text editor.
Configure Supervisor to start cpulimit.
```
[program:cpulimit]
command=cpulimit --exe nxlog --limit 20
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/var/log/cpulimit.log
```
This configuration controls the resource usage of NXLog Agent by limiting its consumption to 20% of processor time.

The autostart=TRUE option automatically starts cpulimit when Supervisor starts.

The autorestart=TRUE option automatically restarts cpulimit when it exits, regardless of its exit code.

Messages or errors from stdout and stderr will be written to file in /var/log/cpulimit.log. This log file provides information about process state changes and can help debug the process if it’s not starting.

This configuration is only an example; you can modify it as needed.
Disable automatic startup of NXLog Agent from systemd to prevent it from colliding with the process started by cpulimit.
```
# systemctl disable nxlog
```
Execute the following commands to enable and start Supervisor.
```
# systemctl enable supervisord
# systemctl start supervisord
```

Using systemd

On operating systems that use systemd, you can control resource usage from the nxlog.service configuration file. Refer to your operating system’s documentation for more information on systemd usage.

The following instructions configure the NXLog Agent service with a CPU usage limit of 10% and a maximum memory allocation of 512MB:

Open the nxlog.service file located at /lib/systemd/system with a text editor.

Add the following lines at the end of the [Service] section:

CPUQuota=10%
MemoryMax=512M

The complete section should look like:

[Service]
Type=simple
User=root
Group=root
PIDFile=/opt/nxlog/var/run/nxlog/nxlog.pid
ExecStartPre=/opt/nxlog/bin/nxlog -v
ExecStart=/opt/nxlog/bin/nxlog -f -q
ExecStop=/opt/nxlog/bin/nxlog -s
ExecReload=/opt/nxlog/bin/nxlog -r
KillMode=process
CPUQuota=10%
MemoryMax=512M

Execute the following command to apply the configuration:
```
# systemctl daemon-reload
```
If NXLog Agent is already running, restart the service:
```
# systemctl restart nxlog
```

Using cgroups

Another way to limit RAM and CPU usage on Linux-based systems is through cgroups. cgroups allows you to allocate resources, such as CPU time, system memory, and network bandwidth, based on user-defined groups.

By default, NXLog Agent runs as the nxlog User and Group. Follow these steps to install cgroups and configure CPU and memory limits for the nxlog group:

Install cgroups according to your platform.
- On RHEL/CentOS, execute the following command:
  # yum install libcgroup libcgroup-tools
- On Debian/Ubuntu, execute the following command:
  # apt install cgroup-tools
Open the cgroups configuration file located at /etc/cgconfig.conf with a text editor.

Add the following configuration at the end of the file:

cgconfig.conf

group nxlog {
        cpu {
                cpu.cfs_quota_us = 10000;
                cpu.cfs_period_us = 100000;
        }
        memory {
                memory.limit_in_bytes="1G";
                memory.memsw.limit_in_bytes="2G";
        }
}

The first part of this configuration limits CPU usage to 10%. The second part caps RAM usage to 1GB and an additional 1GB of swap space.

To apply this configuration to NXLog Agent, add the following line to the /etc/cgrules.conf file:
cgrules.conf
```
@nxlog          cpu,memory      nxlog
```

With the configuration in place, you can now run the following commands to start the cgroups processes.

# sudo systemctl enable cgconfig
# sudo systemctl start cgconfig
# sudo systemctl enable cgred
# sudo systemctl start cgred

Windows systems

Windows systems do not provide a native method to control how much memory or CPU a process can use. However, you can use third-party tools to limit resource consumption. Below, we provide an example of how to limit NXLog Agent resource usage with Process Governor.

Using Process Governor

Process Governor, or procgov, is an open-source software that allows you to set constraints on a process on Windows operating systems.

Follow these steps to install and execute Process Governor:

Download procgov.zip, extract the archive to a folder, and run procgov.exe to install it.
Process Governor has no graphical user interface, so you must invoke it from the command line or a script. For example, this PowerShell script runs Process Governor with arguments that allow the NXLog Agent process a maximum of 200MB memory, 1 CPU, and 20% CPU usage. If NXLog Agent is already running, the script restarts it.
procgov.ps1
```
$workingdir = "C:\Program Files\procgov"
restart-service nxlog
$nxlog = (get-process nxlog | select -expand id)
& "$workingdir\procgov64.exe" --maxmem 200M --cpu 1 --cpurate 20 --pid $nxlog
```
Assigning insufficient memory for the NXLog Agent process may cause it to crash. We recommend observing the agent’s memory usage under normal operation and assigning a limit accordingly.

You may receive the following error message when attempting to manage resources for NXLog Agent using Process Governor:

ERROR: Access is denied. (0x80040005)

This error indicates that Process Governor cannot restart the NXLog Agent service. To resolve this issue, run Process Governor using the local SYSTEM account. You can do this with PsExec.

For example, execute the command as follows:

"$workingdir\psexec" -s -i "$workingdir\procgov64.exe" --maxmem 200M --cpu 1 --cpurate 20 --pid $nxlog

You may skip the -i (interactive) option if you do not need to see the Process Governor output.

Running Process Governor as a service

Running Process Governor from the command line or a script is adequate for troubleshooting. However, it may not be viable as a long-term solution. An alternative is to run Process Governor as a service using a service manager such as NSSM.

An example script for installing NSSM and configuring Process Governor to run as a service is available in our public git repository. Download the contents of the procgov-resource-handling directory and follow the README file to execute the script.

Disclaimer

While we endeavor to keep the information on this page up to date and correct, NXLog makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

Debian 11 (Bullseye)
CentOS 7
Windows 10
Windows Server 2012 R2

Last revision: 10 December 2021