NXLog Platform terminology

Rules that define who can view which log types and how far back a user can view data. They act as a second-level filter on top of the user’s roles.

Agent refers to an NXLog Agent instance installed and configured on a computer to collect telemetry data from various sources and forward them to one or more destinations, including NXLog Platform. In addition, NXLog Agent can filter, normalize, and enrich data before sending it to its destination. An agent is used for agent-based log collection.

A log collection mode where you install an agent on a device to collect, parse, and forward logs to a central location.

NXLog Platform provides NXLog Agent as its log collector agent, which can be installed on a wide range of operating systems. It supports platform-specific sources, such as Windows Event Log, Linux system logs, macOS system logs, and syslog, among others. NXLog Agent can also filter, normalize, and convert log records into a different format before forwarding them.

A technique to collect logs remotely without installing an agent on the log source. It is used when agent-based log collection is not possible because of technical, administrative, or compliance restrictions.

NXLog Platform can operate in this mode, receiving logs over the network using transport protocols such as TCP, UDP, and HTTP. It then processes the logs and forwards them to a central location, such as NXLog Platform.

The NXLog Platform component for monitoring, configuring, and managing your NXLog Agent fleet. It also includes configuration management and automatic enrollment rules.

An audit log or an audit trail is a type of log that contains a non-repudiable record of system activities. They are an essential part of IT security, giving insight into what is happening in your environment and helping you meet compliance requirements and risk management. NXLog Platform automatically records audit logs. You can view and search the audit log from the NXLog Platform UI.

A way to enroll NXLog Agent instances to NXLog Platform and configure them according to predefined rules. Once you create auto-enrollment rules, all you need to do is install NXLog Agent and configure it for first contact, then let NXLog Platform do the rest.

Bandwidth refers to the network’s maximum data transfer capacity within a specific time frame. It is typically measured in terms of the amount of data that can be transmitted in one second, for example, megabits per second (Mbps).

When transferring telemetry data, the amount of data that can be sent over the network within a given time frame depends on the available bandwidth. For example, consider the bandwidth required to aggregate telemetry data from thousands of computers or transfer logs from high-volume sources over the network. Not to be confused with EPS (Events Per Second).

A standard log format developed by ArcSight to allow vendors and customers to integrate their product information with ArcSight ESM. The CEF standard defines a syntax for log records. It comprises a standard header and a variable extension of key-value pairs.

CEF is one of the many log formats NXLog Agent supports. NXLog Agent can parse and generate logs in Common Event Format (CEF) with the Common Event Format extension.

A predefined NXLog Agent configuration. You can create configurations for different requirements and apply each one to multiple NXLog Agent instances. Configurations allow you to deploy configuration updates to agents in bulk.

A data structuring format that uses commas as a delimiter to separate data values. It is most suitable for storing tabular data in plain text. CSV files are often used when data needs to be compatible with many different programs. You can open a CSV file in any text editor or a spreadsheet program like Microsoft Excel.

CSV is one of the many data formats NXLog Agent supports. Telemetry data formatted as values delimited by commas, spaces, or semicolons is very common. NXLog Agent can parse and generate data in CSV format using the Delimiter-Separated Values extension.

In NXLog Platform, a customer is synonymous with an organization. It is an entity subscribed to NXLog Platform. Each organization manages its own users and NXLog Agent fleet.

Web console where NXLog Platform customers manage their organization details, subscriptions, billing information, and invoices. The Customer Portal is hosted by NXLog at https://platform.nxlog.co.

An NXLog Platform installation. A deployment can be one of three types:

A service that translates host and domain names into IP addresses. Clients send requests to the local DNS server, which in turn communicates with root and top-level domain servers, to resolve the IP address for a given name.

DNS servers create detailed DNS traffic logging. It helps system administrators troubleshoot DNS errors or identify attempts to attack the DNS infrastructure. DNS clients also record client DNS queries to a server. However, DNS server logs are often of higher value than DNS client logs, since they can help identify malicious activity, such as Denial of Service (DoS) and Command and Control attacks.

Given the importance of DNS logging, we provide several guides, tutorials, and dedicated NXLog Agent modules to collect all types of DNS telemetry data.

An approach to securing end-user devices, such as laptops, desktop computers, and mobile phones. Protecting endpoints is essential because they can be entry points for malicious attacks and exploits. Endpoint security systems have evolved from conventional antivirus software to more comprehensive protection that relies on logs collected from the endpoints. Today, endpoint security systems can detect, analyze, block, and contain ongoing attacks.

NXLog Agent plays an important role in transferring telemetry data from devices to your endpoint security solution.

The act of subscribing an NXLog Agent instance to NXLog Platform. An enrolled agent has received a TLS certificate, a public key, and a UUID, and you can manage it from NXLog Platform.

EPS is the standard unit of measurement for log processing speed, i.e., how many events an application can process in one second. In other words, it is a measurement used in IT to define the number of events that go in or out of a system in a given time. NXLog Platform uses EPS to measure the log processing rate.

A Microsoft Windows feature that allows recording of kernel and application-defined events. You can configure different Windows services, such as the Windows Firewall and DNS Server, to log events through Windows Event Tracing.

Debug and Analytical channels rely on ETW, and they cannot be collected through regular Windows Event Log channels. NXLog Agent can read these event traces using the Event Tracing for Windows input module. This module does not require you to save logs to an intermediate file on disk. Instead, it reads the data directly from the ETW provider.

A method that uses patterns to find relationships between events from different sources, such as applications, devices, and operating systems. It allows you to take remedial action when correlation rules identify a pattern that poses a security threat.

Most SIEMs are capable of correlating events. However, performing event correlation at the log collection level can be more practical, as it minimizes the data sent to your SIEM, thereby reducing log noise, network traffic, and costs. NXLog Agent can correlate events using the Event Correlator processor module. Along with the NXLog language, it provides the tools you need to correlate events at the log collection stage.

The process that automatically transfers operations to an identical backup system when the primary system becomes unavailable due to failure or scheduled downtime. Failover can occur on a computer, network, hardware, or software component on standby if the initial system or component fails.

NXLog Agent supports failover mode. You can set up a cluster of agents, where if the first node fails, the next node in the cluster takes over. Network modules, such as the TCP and HTTP(s) modules, support failover configuration, where if the first destination is unavailable, the module tries to send logs to the next available destination.

A JSON-based, structured log format popularized by Graylog. It consists of a set of predefined event fields, including the event timestamp, hostname, severity, and long and short messages, and supports additional custom fields for application-specific information.

GELF is one of the many data formats NXLog Agent supports. NXLog Agent can parse and generate GELF logs with the GELF extension.

A network layer protocol utilized by network devices to diagnose communication problems. ICMP is primarily used to determine whether data is reaching its intended destination promptly. Additionally, network devices, such as routers, utilize the ICMP protocol for error reporting when network issues prevent the delivery of data packets.

Hackers have also found ways to use ICMP messages maliciously, such as Ping of Death (PoD) attacks, Smurf attacks, and ping flood attacks. While few networks are vulnerable to PoD and Smurf attacks today, most systems remain susceptible to ping flood attacks. In corporate networks, ICMP traffic commonly indicates ping requests, and a certain amount of ICMP traffic is expected. However, a sharp increase in ICMP traffic over a short period usually indicates malicious activity. Therefore, ICMP traffic is worth monitoring.

NXLog Agent provides the Packet Capture input module to capture network traffic. You can use it in conjunction with the Event Correlator processor module to define a threshold for expected ICMP traffic and generate an alert if exceeded.

A hardware device or software application that monitors network activity and generates alerts for security violations. IDS can be host-based or network-based, and apply a mixture of signature-based and anomaly-based detection techniques to identify threats. Any malicious activity or infringement is usually reported to an administrator or aggregated via a SIEM. Some intrusion detection systems are augmented with tools such as a honeypot to attract and categorize malicious traffic.

NXLog Agent can simplify log analysis for an IDS by providing it with filtered, trimmed, parsed, and normalized data. It can also collect logs from diverse sources and send them to an IDS and other destinations simultaneously. Finally, intrusion detection systems also generate logs, which NXLog Agent can collect and forward to a central repository.

A standard data-interchange text format consisting of key-value pairs and arrays. JSON is straightforward for software solutions to parse and generate. As a result, it is frequently used for serializing structured data when exchanging it over a network, such as between servers and web applications. Additionally, Unicode encoding ensures that this messaging format is universally accessible. JSON can be used with any programming language, as mapping domain objects is very straightforward. Because JSON is platform-independent, it is a standard log format.

JSON is one of the many data formats NXLog Agent supports. NXLog Agent can parse and generate JSON logs with the JSON extension.

Logs containing detailed information, including boot messages, hardware driver information, kernel status, and other kernel-related events within the operating system. Kernel logging is initialized from the system startup and is helpful for system auditing and troubleshooting hardware, driver, and startup issues.

NXLog Agent provides several input modules for collecting kernel logs:

A set of data elements consisting of a key, a constant defining the type of data set, and its associated variable value. KVP is an efficient way to store telemetry data because it is easy for software applications to process it. At the same time, humans can read it effortlessly.

KVP is one of the many data formats NXLog Agent supports. NXLog Agent can parse and generate KVP logs with the Key-Value Pairs extension.

A custom log format used by IBM Security QRadar products. It uses key-value pairs to describe an event and consists of predefined event attributes. It also supports additional custom attributes for application-specific information.

LEEF is one of the many log formats that NXLog Agent supports. NXLog Agent can parse and generate LEEF logs with the LEEF extension.

Refers to the process of consolidating logs from multiple sources into a central repository, such as NXLog Platform or your SIEM. Log aggregation agents, such as NXLog Agent, may offer extra functionality, such as log parsing, filtering, correlation, and data normalization.

Log centralization provides administrators with a comprehensive view of activities across their network, making it easier to identify and troubleshoot issues. Storing logs in a central location offers several benefits over logs on disparate systems and is a fundamental part of an effective log management strategy.

One of the most critical aspects determining whether your strategy is successful is the log collection agent you choose. NXLog Agent boasts industry-leading log processing features and an unmatched ability to integrate with any SIEM solution.

The process of formatting telemetry data according to the platform taxonomy when forwarding events to a SIEM or log analysis tool. Almost all SIEM solutions have taxonomies for different types of logs. Normalization enables SIEMs to efficiently interpret logs from diverse sources, facilitates event correlation, and makes it easier for you to work with the data in dashboards and reports.

NXLog Agent can translate logs from different sources into a unified taxonomy. It supports mapping event fields to the required schema, enriching log records with additional fields, and converting events into a different data format. Normalization may require log records to include standard metadata fields, such as labels describing the environment where the event originated and keywords to tag the event. Such data might not be part of the original event record but must be added from an external source.

The process of extracting information from log records into named fields or columns. It applies techniques to extract elements from log data and split them into individual fields for easier consumption by other systems. A common parsing method is using regular expressions with capturing groups.

NXLog Agent can parse incoming log records to allow further processing, such as applying filtering rules or enriching the data. It includes dedicated modules for parsing specific data formats, such as JSON and XML. Additionally, the NXLog language supports regular expressions with named capturing groups and provides several string manipulation functions.

A device or application producing logs, such as:

A log source sends data to an NXLog Agent instance or exposes logs so NXLog Agent can collect them. Log sources are the basis of licensing.

The NXLog Platform component for storing and managing logs. The database backend runs on ClickHouse and uses a schemaless data structure. It also includes data access rules and saved searches and filters.

NXLog Platform automatically creates log types based on the data it ingests. The log type corresponds to the NXLog Agent SourceModuleName core field, which is the name of the input module instance that collected the data.

Logs containing event records spanning multiple lines. An example of multi-line logs is application debug logs that contain stack traces. Such event records start with a timestamp, followed by an error message and the stack trace. Multi-line logs require special treatment because the log collector must be aware of the components that comprise a single event record.

NXLog Agent supports multi-line logs via the dedicated Multiline Parser extension. This module allows you to define how to parse log records, including specifying header and footer lines, or setting a fixed line count.

A Cisco network protocol designed to collect active network traffic. NetFlow provides information such as the source, destination, and traffic volume. IT administrators use NetFlow data for network monitoring and capacity planning. IT professionals use the NetFlow protocol as a network traffic analyzer to determine the point of origin, destination, volume, and network traffic paths.

NetFlow is one of the many log formats NXLog Agent supports. NXLog Agent can parse NetFlow logs with the NetFlow extension.

An entity subscribed to NXLog Platform. Each organization manages its users and NXLog Agent fleet.

Customer and Organization might be used interchangeably in NXLog Platform, representing the NXLog Platform’s view of the world: each organization has its own set of users, licenses, invoices, etc.

On the NXLog Platform side, we do not require an Organization to be equal to a business entity so that a single company can set up multiple organizations on NXLog Platform, for example, for different departments.

In this sense, using Organization instead of Customer is more descriptive.

A security compliance standard to guarantee that all businesses, regardless of size or type, that accept, process, or store credit card data maintain a protected environment. The PCI-DSS compliance standard was developed by major credit card companies, including Visa, Mastercard, American Express, Discover, and JCB, to protect sensitive customer data and lessen the risk of data breaches and fraud. It protects against various threats, including hacking, identity theft, and credit card fraud.

Compliance is one of the essential security measures for companies. PCI-DSS is important because it helps safeguard sensitive financial and personal information from unauthorized access.

NXLog Platform can help meet compliance mandates in numerous ways, including protecting cardholder data. NXLog Agent supports data encryption and compression to help ensure that cardholder data has not been tampered with.

A set of rules defining how to format and exchange data over a computer network. It allows different devices to communicate with each other regardless of their underlying software and hardware differences. For example, some of the most used network protocols are TCP/IP, HTTPS, and SMTP.

NXLog Agent supports all the necessary network protocols for collecting and forwarding logs. It can also collect protocol-based network traffic using the Packet Capture input module, including a set of industrial network protocols. NXLog Agent also implements a custom NXLog Transport protocol, which transfers compressed, structured data between agent instances and NXLog Platform.

An NXLog Agent instance receiving data from multiple sources over the network and forwarding it to a centralized location, such as NXLog Platform or a SIEM.

A type of security software that collects and analyzes telemetry data from various sources, including endpoint devices, firewalls, intrusion detection/prevention systems, and servers. The goal of a SIEM is to provide real-time analysis of security events generated by these devices and report security-related information.

SIEM systems rely on log collection agents, such as NXLog Agent, to ingest data from heterogeneous sources. Besides collecting telemetry data, an adequate agent filters, enriches, and transforms the data into the SIEM’s taxonomy before sending it to the SIEM. This initial processing at the log source ensures the SIEM receives clean data that it can analyze efficiently.

An application-layer protocol for managing and monitoring network devices. It provides a method for collecting and organizing security logs from managed devices across an IT network.

NXLog Agent supports parsing SNMP v1, v2c, and v3 trap messages using the SNMP extension. For SNMP v3 messages, it also supports the user-based security model (USM), providing authentication and encryption functionality.

A consistent, predetermined format for telemetry data that facilitates searching and analysis. Some standard log formats include JSON, CSV, XML, and KVP.

Most logs already contain structured or at least semi-structured data. For unstructured or semi-structured data that contains known fields, it is more efficient to parse the records sooner rather than later. NXLog Agent contains data parsing and formatting capabilities that simplify this process at the log collection stage.

A set of features sold together, such as the Free, Basic, and Premium subscription plans. You can change or add more features to your subscription over time. Subscriptions are valid for a set period and expire at the end of their term if not renewed.

An IT standard for logging messages widely used by Linux/Unix operating systems and network devices such as routers and switches. It is used for logging status events and reporting incidents that aid in system monitoring and troubleshooting. Syslog’s roots date back to the 1980s, and several iterations have followed, including BSD syslog (RFC 3164) and IETF syslog (RFC 5424).

Syslog remains one of the most widely used log formats today. NXLog Agent fully supports syslog and can parse and generate syslog messages with the Syslog extension.

A communication protocol used to transmit data over the network. It provides ordered and error-checked delivery of a stream of octets (bytes) between applications and is considered a reliable protocol. In the context of telemetry data, TCP is most likely to be used when transferring data between computers, for example, via a relay or sending it to a SIEM.

NXLog Agent fully supports TCP and can receive and forward data using the TCP input and output modules.

An NXLog Platform instance belonging to a specific organization. An NXLog Platform cloud deployment may run multiple tenants, while an on-premises deployment runs a single tenant.

TLS, formerly SSL, is a cryptographic protocol that provides secure communication over a network by encrypting the transmitted data. TLS/SSL offers end-to-end protection for data sent over the network. It is used for safe web browsing, secure instant messaging, email, file transfers, video conferencing, voice-over-IP, and other network services.

Encryption plays a key role in safeguarding and protecting confidential data during tranmission. NXLog Agent can securely receive and forward data over TCP using the TLS/SSL input and output modules.

A communication protocol used to transmit data over the network. It does not provide data delivery acknowledgment and is, therefore, not a reliable protocol.

UDP is the transport protocol used by the legacy BSD syslog (RFC 3164). Therefore, this protocol can be advantageous to receive log messages from older devices that do not support newer protocols. UDP is also the preferred protocol for high-volume, real-time data transfer, when data loss is not a concern.

NXLog Agent fully supports UDP and can receive and forward data using the UDP input and output modules.

NXLog Platform supports Role-Based Access Control (RBAC) to define which parts of the NXLog Platform UI each user can access and what operations they can perform. Every NXLog Platform user is assigned one or more roles representing the user’s permissions.

A text-based log format similar to CEF, but provides an extended list of fields. It can be customized to log only relevant data and omit unwanted fields. Since it is a standard format, most log analysis tools can parse it out of the box.

The W3C format is one of the many log formats that NXLog Agent supports. NXLog Agent can parse the W3C Extended Log File Format, including Microsot IIS and Zeek logs, using the W3C extension.

A server configured to collect Windows events from remote computers. It receives events via WEF (Windows Event Forwarding) without requiring an agent to be installed on the clients.

NXLog Agent provides the Windows Event Collector input module, which allows it to serve as a WEC server on both Windows and Linux platforms. In addition to collecting events from WEF clients, it provides functionality to trim, filter, and normalize events before sending them to a central repository, such as NXLog Platform.

A native Windows service that forwards Windows events to a WEC (Windows Event Collector). It is used for agentless log collection, when installing a log collection agent is not possible, for example, due to operational or compliance requirements.

While WEF is a valuable technology, it has its limitations. Some of these limitations include being unable to forward logs from Event Tracing Providers, scaling and managing many WEF clients being difficult, and not being able to forward events directly to a SIEM.

NXLog Agent supports collecting Windows events from WEF clients, either by taking the role of a WEC server or by collecting forwarded events from a Windows WEC server.

A unique identifier associated with each event in Windows Event Log. The event ID determines the reason for every event logged. Event IDs are unique per source but are not globally unique; therefore, different sources may use the same event ID to identify unrelated operations.

You can view the logs in Windows Event Log and their associated Event IDs in the Event Viewer MMC snap-in, which is included in Windows.

NXLog Agent’s Event Log for Windows input module supports collecting Windows events using the Windows Event Log API, meaning it does not need any intermediate software or writing logs to a file. In addition, it supports collecting Windows events based on event IDs using XPath queries, so you can choose which event IDs to collect from the relevant sources.

Microsoft’s implementation of the Web-Based Enterprise Management (WBEM) and Common Information Model (CIM) standards. WMI provides an infrastructure for overseeing remote systems and delivering management data. While it’s a critical component of Windows operating systems, malicious actors can exploit it, posing a security risk.

You can collect WMI logs via Windows Event Log or Event Tracing for Windows (ETW). Before Windows Vista and Windows Server 2008, WMI also logged events to a file. NXLog Agent supports collecting WMI events using the Event Log for Windows and Event Tracing for Windows input modules.

Short for XML Path Language, it uses a "path-like" syntax to navigate and identify nodes in an XML document. You can also use XPath queries to test addressed nodes within a document to determine whether they match a specific pattern.

Windows Event Log supports a subset of XPath 1.0. You can use XPath queries to subscribe to events that match specific criteria in Windows Event Viewer. NXLog Agent’s Event Log for Windows input module also supports collecting Windows events based on XPath queries.

Event Viewer offers the most practical way to write and test queries. For example, you can right-click a log in Event Viewer and select Filter Current Log. From there, you can configure the filter from the user interface and switch to the XML tab to view the XPath query.