Installer packages are provided for multiple platforms, including Linux and Windows. You can use NXLog across your entire infrastructure, without resorting to different tools for different platforms.

Depending on the configuration, NXLog will run as a server, a client, or a combination of both. You have the freedom to choose the deployment architecture that best meets your needs. For example, NXLog can collect local log data and forward it, relay data without storing it locally, or save incoming log data to disk.

NXLog can accept data from many different sources, convert the data internally, and output it to other destinations. You can use NXLog as a single tool to process all of the different types of logs in your organization. For example, logs can be collected from files, databases, Unix domain sockets, network connections, and other sources. BSD Syslog, IETF Syslog, the Snare Agent format, Windows Event Log, JSON, and other formats are supported. NXLog can likely be configured to read or write logs in your custom application format, using the NXLog language and provided extension modules.

With an event-based architecture for processing tasks in parallel, non-blocking input and output where possible, and a worker thread pool for incoming log messages, NXLog is designed for high performance on modern multi-core and multi-processor systems. The input/output readiness notifications provided by most operating systems are used to efficiently handle large numbers of open files and network connections.

NXLog provides features throughout the application to maintain the security of your log data and systems. The core can be configured to run as an unprivileged user, and special privileges (such as binding to ports below 1024) are accessed through Linux capabilities rather than requiring the application to run as root. TLS/SSL is supported for encrypted, authenticated communications and to prevent data interception or alteration during transmission.

NXLog has a lightweight, modular architecture, providing a reduced memory footprint and increased flexibility for different uses. The core handles files, events, and sockets, and provides the configuration language; modules provide the input, output, and processing capabilities. Because modules use a common API, you can write new modules to extend the features of NXLog.

Log messages can be buffered in memory or on disk. This increases reliability by holding messages in a temporary cache when a network connectivity issue or dropout occurs. Conditional buffering can be configured by using the NXLog language to define relevant conditions. For example, UDP messages may arrive faster than they can be processed, and NXLog can buffer the messages to disk for processing when the system is under less load. Conditional buffering can be used to explicitly buffer log messages during certain hours of the day or when the system load is high.

NXLog can be configured to separate high-priority log processing from low-priority log processing, ensuring that it processes the most important data first. When the system is experiencing high load, NXLog will avoid dropping important incoming messages. For example, incoming UDP messages can be prioritized to prevent dropped logs if a high volume of TCP messages overloads the system.

Built-in flow control ensures that a blocked output does not cause dropped log messages when buffers are full. In combination with the previously mentioned parallel processing, buffering, and prioritization, the possibility of message loss is greatly reduced.

NXLog uses an Apache-style configuration syntax that is easy to read and can be parsed or generated with scripts. The NXLog language supports advanced scripting and processing capabilities that are usually only found in full-fledged scripting languages. The syntax is similar to Perl, so users familiar with that language can learn it easily. It supports polymorphic functions and procedures and regular expressions with captured sub-strings. Modules can register additional functions and procedures to further extend the capabilities of the language.

NXLog includes a scheduler service. A task can be scheduled from any module without requiring an external tool such as Cron. Log files can be rotated automatically, on a time-based schedule or according to file size. The file reader and writer modules in NXLog can detect when an input or output file moves or changes its name, and re-open the file automatically.

NXLog can perform advanced processing actions on log messages, in addition to the core features already mentioned. By using additional modules, NXLog can solve many related tasks such as message classification, event correlation, pattern matching, message filtering, message rewriting, and conditional alerting. You can use a single tool for all log processing functionality.

Sometimes log messages need to be processed in batches for conversion, filtering, or analysis. NXLog provides an offline mode in which it processes all input and then exits. Because NXLog does not assume that the event time and processing time are identical, time-based correlation features can be used even during offline log processing.

NXLog supports explicit character set conversion and automatic character set detection. Log messages received in different character sets can be automatically normalized to a common standard, allowing messages to be compared across different sources.

In addition to Linux and Windows, installer packages are provided for the BSDs and the major variants of Unix (AIX, Solaris, and macOS) operating systems.

Installer packages are digitally signed to ensure that the binaries are not corrupted or compromised. In many cases, this is a compliance mandate.

Log data can be transferred in compressed batches with the im_batchcompress and om_batchcompress input/output modules. This can help in limited bandwidth scenarios.

Due to vulnerabilities discovered in the SSL protocols, some protocols may need to be disabled. The various SSL/TLS networking modules in NXLog Enterprise Edition can be configured to allow only specific protocols via the SSLProtocol directive. On Windows, NXLog Enterprise Edition can utilize TLSv1.2 while NXLog Community Edition supports TLSv1.0 only.

The ODBC input and output modules, im_odbc and om_odbc, allow log data to be read from or inserted into any ODBC-compliant database. The primary purpose of the im_odbc module is native Windows MSSQL support to enable log collection from Windows applications that write logs to MSSQL. The om_odbc output module can be used to insert data into an ODBC database. These modules are available on Windows and Linux too.

The dedicated xm_admin extension module enables NXLog agents to be managed remotely over a secure SOAP/JSON SSL connection or to be integrated with existing monitoring and management tools. The configurations, correlation rules, patterns, and certificates can all be updated remotely from the NXLog Manager web interface or from scripts. In addition, the NXLog agents and the individual modules can be stopped/started and log collection statistics can be queried for real-time statistics.

Additional functionality is provided to guarantee a clean recovery in the case of a system crash, ensuring that no messages are lost or duplicated.

The pm_evcorr processor module can efficiently solve complex event correlation tasks, with capabilities similar to what the open-source SEC tool provides.

The im_http and om_http input and output modules make it possible to send or receive log message data over HTTP or HTTPS.

Accepting and initiating TCP and UDP connections to a remote server can be achieved using the dedicated protocol-specific im_tcp and om_tcp modules, as well as the im_udp and om_udp modules respectively.

Some SIEM and log collection systems use the IP address of the UDP Syslog packet sent by the client. As a server or relay, the om_udpspoof output module can be configured to retain the original IP address of the sender.

Several compliance standards mandate file integrity monitoring. With the im_fim input module, NXLog Enterprise Edition can be used to detect modifications to files or directories. This module is available on Windows as well as Linux.

The im_regmon module facilitates the monitoring of the Windows Registry and generates event records in case of changes in the monitored registry entries.

The passive monitoring of network traffic can be implemented via utilizing the im_pcap module.

The xm_xml extension module can parse nested XML and data stored in XML attributes.

Parsing of nested JSON is implemented in the xm_json module. UTF-8 character encoding validation can be enforced to avoid parser failures caused by invalid UTF-8 encoding from other tools.

Parsing and generation of key-value formatted data can be accomplished by using the xm_kvp module.

Messages which span multiple lines can be processed with the xm_multiline module. The flexible configuration of the module is reached through the specification of the header line, the footer line, and the number of lines in a message.

The W3C format is widely used in various Microsoft products and perhaps IIS is the most well-known producer. Parsing of W3C is possible with the xm_csv extension module, but that requires defining the fields in the configuration and adjustment when the IIS configuration is changed. The xm_w3c extension module can automatically parse the logs using the field information stored in the headers. It also supports automatic parsing of the data format produced by BRO.

The xm_cef and xm_leef modules provide parsing and generation of CEF and LEEF formatted data. CEF (Common Event Format) was introduced by HP ArcSight and LEEF (Log Event Extended Format) is used by IBM Security QRadar.

Two extension modules help simplify data processing. The xm_rewrite module allows fields to be renamed, kept (whitelisted), or deleted (blacklisted). It also supports the Exec directive so log processing logic can be localized to avoid duplicated statements. The xm_filelist module provides two functions, contains() and matches(), which can be invoked to check whether a string is present in a text file. This can be a username or an IP address for example. The files are cached in memory and any changes are automatically picked up without the need to reload NXLog.

The Enterprise Edition of NXLog supports custom modules which can be developed in the following programming languages:

The xm_resolver extension module provides cached DNS lookup functions for translating between IP addresses and host names. User and group names can also be mapped to/from user and group ids.

The om_elasticsearch output module allows log data to be loaded directly into an Elasticsearch server without requiring Logstash.

The im_checkpoint input module enables the remote collection of Check Point firewall logs over the OPSEC/LEA protocol. This feature is only available in the Linux version.

Redis is often used as an intermediate queue for log data. Two native modules, im_redis and om_redis, are available to push and pull data to and from Redis servers.

The xm_snmp extension module can be used to parse SNMP traps. The traps can then be handled like regular log messages: converted to Syslog, stored, forwarded, etc.

The im_wseventing input module can be used to collect forwarded events from Windows hosts. The Windows clients can be configured from Group Policy to send Windows Event Log using Windows Event Forwarding. While NXLog Enterprise Edition can collect Windows Event Log remotely over WMI and MSRPC, this module provides improved security for collecting from Windows machines in agentless mode, with support for both Kerberos and HTTPS data transfer. The im_wseventing module is platform independent and available on Linux as well as Windows.

The om_webhdfs output module is available to support the Hadoop ecosystem.

The im_winperfcount input module can collect metrics from Windows Performance Counters such as CPU, disk, and memory statistics.

The im_msvistalog module can read .evt, .evtx, and .etl event log files directly; this is particularly useful for forensics purposes.

The im_msvistalog module retrieves the EventData and UserData parts which can contain important data in some log sources. In addition, SID values in the Windows Event Log record can be resolved to account names to produce the same output that Windows Event Viewer gives.

The im_etw module provides direct reading of event tracing data of both kernel and user-mode applications.

The xm_netflow extension module can parse Netflow packets received over UDP. It supports Netflow v1, v5, v7, v9, and IPFIX.

ZeroMQ is a popular high performance messaging library. The im_zmq and om_zmq modules provide input and output support for the ZeroMQ protocol.

The Enterprise Edition supports communicating logs via:

Simple test events can be generated using the im_testgen module according to the specified counter. The im_null and om_null module can be used for testing purposes as well.

Via the im_mark module, NXLog can send mark messages confirming its correct operation.

NXLog Enterprise Edition supports conversion of message strings between various character sets. This may be useful when the encoding of accepted messages differs from UTF-8.
Compression and encryption operations with logs are available via the xm_zlib and xm_crypto modules.

NXLog Enterprise Edition provides retention and rotation policies which can be applied to files, such as log retention based on the file size and age, and cyclic rotation and retention of files. All these features are available via the xm_fileop module.

External scripts can be run on startup at the input, output, and extension levels using the im_exec, om_exec, and xm_exec modules respectively.

NXLog Enterprise Edition supports a wider variety of log sources and provides additional functionalities to the modules which exist in both the Enterprise and Community versions of NXLog.

Using the pm_buffer prevents from losing messages which arrive over UDP.

Unlike NXLog Community Edition which is a volunteer effort, NXLog Enterprise Edition receives regular hot fixes and enhancements.

For NXLog Enterprise Edition, a dedicated professionally trained support team is available and ready to act at request. They can be available 24/7 with a world-class, 4-hour SLA.

Our constantly updated, ever-growing documentation, already well above 1500 pages, is a stand-alone product in itself. It is complete with configuration samples, real-world examples, and integration guides offering much more than a generic manual.