Control NXLog Agent resource usage
NXLog Agent is designed to collect and process logs as fast as possible. A side-effect of this may be high resource usage when you have many events to be processed. You can use operating system-level tools to limit resource consumption by NXLog Agent. However, this will also introduce a delay in log processing. Implementing a limit on resource usage is a good option when a log processing delay is not a problem or to deal with intermittent high loads since the delay will only occur for a short period. When dealing with a consistently high load and introducing a log processing delay is not desirable, you may need to consider other solutions, such as splitting the load between multiple NXLog Agent instances.
Below, we provide some options for setting up control methods that prevent NXLog Agent from consuming more than the designated amount of RAM and CPU cycles. Different approaches and tools apply according to your operating system.
Linux-based systems
Using cpulimit
cpulimit is a command-line tool that you can use to limit the CPU usage of a process. You can use a process manager like Supervisor to start cpulimit automatically and execute it as a long-running process. Both tools are available in most of the default repositories of Unix-like distributions.
Follow these steps to install and configure cpulimit and Supervisor:
-
Install both tools according to your platform.
-
On RHEL/CentOS, enable the EPEL Repository:
# yum install epel-releaseThen execute the following command to install cpulimit and Supervisor:
# yum install cpulimit supervisor -
For Debian/Ubuntu, execute the following command:
# apt install cpulimit supervisor
-
-
Open the Supervisor configuration file located at
/etc/supervisor/supervisord.confwith a text editor. -
Configure Supervisor to start cpulimit.
[program:cpulimit] command=cpulimit --exe nxlog --limit 20 autostart=true autorestart=true redirect_stderr=true stdout_logfile=/var/log/cpulimit.logThis configuration controls the resource usage of NXLog Agent by limiting its consumption to 20% of processor time.
The
autostart=TRUEoption automatically starts cpulimit when Supervisor starts.The
autorestart=TRUEoption automatically restarts cpulimit when it exits, regardless of its exit code.Messages or errors from
stdoutandstderrwill be written to file in/var/log/cpulimit.log. This log file provides information about process state changes and can help debug the process if it’s not starting.This configuration is only an example; you can modify it as needed.
-
Disable automatic startup of NXLog Agent from systemd to prevent it from colliding with the process started by cpulimit.
# systemctl disable nxlog -
Execute the following commands to enable and start Supervisor.
# systemctl enable supervisord # systemctl start supervisord
Using systemd
On operating systems that use systemd, you can control resource usage from the nxlog.service configuration file.
Refer to your operating system’s documentation for more information on systemd usage.
The following instructions configure the NXLog Agent service with a CPU usage limit of 10% and a maximum memory allocation of 512MB:
-
Open the
nxlog.servicefile located at/lib/systemd/systemwith a text editor. -
Add the following lines at the end of the
[Service]section:CPUQuota=10% MemoryMax=512MThe complete section should look like:
[Service] Type=simple User=root Group=root PIDFile=/opt/nxlog/var/run/nxlog/nxlog.pid ExecStartPre=/opt/nxlog/bin/nxlog -v ExecStart=/opt/nxlog/bin/nxlog -f -q ExecStop=/opt/nxlog/bin/nxlog -s ExecReload=/opt/nxlog/bin/nxlog -r KillMode=process CPUQuota=10% MemoryMax=512M -
Execute the following command to apply the configuration:
# systemctl daemon-reload -
If NXLog Agent is already running, restart the service:
# systemctl restart nxlog
Using cgroups
Another way to limit RAM and CPU usage on Linux-based systems is through cgroups. cgroups allows you to allocate resources, such as CPU time, system memory, and network bandwidth, based on user-defined groups.
By default, NXLog Agent runs as the nxlog User and Group.
Follow these steps to install cgroups and configure CPU and memory limits for the nxlog group:
-
Install cgroups according to your platform.
-
On RHEL/CentOS, execute the following command:
# yum install libcgroup libcgroup-tools -
On Debian/Ubuntu, execute the following command:
# apt install cgroup-tools
-
-
Open the cgroups configuration file located at
/etc/cgconfig.confwith a text editor. -
Add the following configuration at the end of the file:
cgconfig.confgroup nxlog { cpu { cpu.cfs_quota_us = 10000; cpu.cfs_period_us = 100000; } memory { memory.limit_in_bytes="1G"; memory.memsw.limit_in_bytes="2G"; } }The first part of this configuration limits CPU usage to 10%. The second part caps RAM usage to 1GB and an additional 1GB of swap space.
-
To apply this configuration to NXLog Agent, add the following line to the
/etc/cgrules.conffile:cgrules.conf@nxlog cpu,memory nxlog -
With the configuration in place, you can now run the following commands to start the cgroups processes.
# sudo systemctl enable cgconfig # sudo systemctl start cgconfig # sudo systemctl enable cgred # sudo systemctl start cgred
Windows systems
Windows systems do not provide a native method to control how much memory or CPU a process can use. However, you can use third-party tools to limit resource consumption. Below, we provide an example of how to limit NXLog Agent resource usage with Process Governor.
Using Process Governor
Process Governor, or procgov, is an open-source software that allows you to set constraints on a process on Windows operating systems.
Follow these steps to install and execute Process Governor:
-
Download procgov.zip, extract the archive to a folder, and run
procgov.exeto install it. -
Process Governor has no graphical user interface, so you must invoke it from the command line or a script. For example, this PowerShell script runs Process Governor with arguments that allow the NXLog Agent process a maximum of 200MB memory, 1 CPU, and 20% CPU usage. If NXLog Agent is already running, the script restarts it.
procgov.ps1$workingdir = "C:\Program Files\procgov" restart-service nxlog $nxlog = (get-process nxlog | select -expand id) & "$workingdir\procgov64.exe" --maxmem 200M --cpu 1 --cpurate 20 --pid $nxlogAssigning insufficient memory for the NXLog Agent process may cause it to crash. We recommend observing the agent’s memory usage under normal operation and assigning a limit accordingly.
|
You may receive the following error message when attempting to manage resources for NXLog Agent using Process Governor: This error indicates that Process Governor cannot restart the NXLog Agent service. To resolve this issue, run Process Governor using the local SYSTEM account. You can do this with PsExec. For example, execute the command as follows: You may skip the |
Running Process Governor as a service
Running Process Governor from the command line or a script is adequate for troubleshooting. However, it may not be viable as a long-term solution. An alternative is to run Process Governor as a service using a service manager such as NSSM.
An example script for installing NSSM and configuring Process Governor to run as a service is available in our public git repository. Download the contents of the procgov-resource-handling directory and follow the README file to execute the script.