Get started with NXLog
NXLog uses a modular, file-based configuration system that gives you the flexibility to tailor your configuration according to your requirements. To make the most out of your NXLog deployment, it is worth taking a few minutes to understand the log processing concept. Once you have a good foundation of how NXLog handles logs, it will be easy to configure for collecting and parsing logs from diverse sources.
Log processing overview
NXLog processes logs via input, processor, and output modules. In addition, extension modules provide support for processing specific log formats.
With this modular architecture, you can choose which modules to load and separate the input from the output so that a single input can be routed to multiple outputs and multiple inputs can be routed to a single output.
- Input
-
Collect events from your log sources. These may be local or remote file-based logs, Windows events, syslog messages received over TCP or UDP, or any other source supported by NXLog’s Input Modules.
NXLog treats each event as a log record consisting of fields. In most cases, it saves the collected log message in the
$raw_eventfield and creates the following core fields:-
$EventReceivedTime -
$SourceModuleName -
$SourceModuleType
Depending on the input module, other fields may be created automatically.
-
- Parse
-
Parse log messages into structured data to be able to process the record further. Various Extension Modules support parsing common log formats or provide functions to extend log processing capabilities via an external script. In addition, the NXLog language supports the use of Regular Expressions for parsing custom log formats. Some input modules automatically parse log messages into fields, so further parsing may not be necessary.
- Process / Normalize
-
Once a log message is parsed and additional fields are created, you can decide what to do with the log record. For example, you can drop records based on the event type or severity, truncate data, or enrich the record with additional data. This is the stage where you can normalize data according to your SIEM requirements. See Log normalization.
- Format
-
Convert the log record to the required output format. Most output modules use the value of
$raw_event, so you need to ensure that the data in this field is in the required format. Extension modules support converting records to common formats, such as JSON, CSV, and XML. - Output
-
Forward the data to the destination, such as a centralized file repository, database, SIEM, log analytics solution, or any destination supported by NXLog’s Output Modules.
The documentation of individual modules provides details on configuration settings, fields created by default parsing, as well as functions and procedures made available by the module.
Next steps
-
Read more about NXLog’s System architecture
