NXLog search
Let's Talk Start free
Let's Talk
6.10

OpenTelemetry Exporter (om_otel)

This module establishes HTTP(S) and GRPC connections to an OpenTelemetry collector or backend using the OpenTelemetry Protocol (OTLP) Specification. It supports traces, logs and metrics.

OpenTelemetry, or OTel, is an open-source observability framework that provides APIs and other tools for collecting and exporting telemetry data, such as traces, metrics, and logs. OTel is vendor-agnostic, and many observability platforms, libraries, services, and applications integrate it. It supports HTTP/1.1 and HTTP/2 connections.

To examine the supported platforms, see the list of installation packages.

Output data format

om_otel forwards data according to the Format directive. Each record includes the fields listed for im_otel, except the $raw_event, $RecordType, and $Severity fields.

Additionally, om_otel automatically sets the following fields if they are not set:

Record type Empty field Replacement value

Logs

$TimeUnixNano

$EventTime if the field is available, otherwise the current time.

$ObservedTimeUnixNano

$EventReceivedTime if the field is available, otherwise the current time.

$SeverityText

$Severity

$Body

$Message

$Resource

$SourceName will populate $Resource['Attributes'].

Traces

$Name

$Message if the field is set, otherwise $raw_event.

$TraceId

Randomly generated ID

$SpanId

Randomly generated ID

$StartTimeUnixNano

$EventTime, $EventReceivedTime, or the current time, depending on the first available field.

$EndTimeUnixNano

$EventTime, $EventReceivedTime, or the current time, depending on the first available field.

Metrics

om_otel can act as an OTLP metrics exporter. By default, it flattens nested objects to a single key-value pair level and outputs metrics of type gauge. Optionally, you can use a mapping file to define the field type, i.e., sum or gauge, labels, resource attributes, and global labels. Records consisting of a pre-encoded OTLP metric from im_otel are forwarded as-is without further processing.

To ensure the module processes records as metrics, set the $RecordType field to Metric explicitly in the corresponding input module instance.

Mapping file format

Metrics require explicit mapping to determine how to interpret fields. Fields can be assigned as:

  • metrics — Can be of type sum or gauge.

  • labels — Attributes attached to each metric.

  • resource or scope — Additional attributes that attach context to the data.

  • global labels — Fields shared across all metrics.

The mapping is specified as a JSON schema file using the MappingFile directive. For example:

{
  "metrics": {
    "sum": [
      {
        "name": "http.requests.total",
        "labels": [
          "http.method",
          "http.status.code",
          "env"
        ]
      }
    ],
    "gauge": [
      {
        "name": "cpu.usage.percent",
        "labels": [
          "host.name",
          "host.region",
          "env"
        ]
      }
    ]
  },
  "resource": [
    "service.name",
    "host.name"
  ],
  "scope": [
    "telemetry.sdk.name",
    "telemetry.sdk.version"
  ],
  "global_labels": [
    "sdk.version"
  ]
}

This mapping defines:

  • A metric of type sum named http.requests.total, with labels http.method, http.status.code, and env.

  • A metric of type gauge named cpu.usage.percent, with labels host.name, host.region, and env.

  • Two resource attributes, service.name and host.name.

  • Two scope attributes, telemetry.sdk.name and telemetry.sdk.version.

  • One global label, sdk.version, that is added to every emitted metric datapoint.

Name sanitization

The module sanitizes metric names according to the OutputMode.

If the OutputMode directive is set to prometheus (default), it sanitizes names to ensure metrics are valid for any Prometheus-compatible backend as follows:

  • The first charater must match [a-zA-Z_:]. If it’s not, the metric name is prefixed with nx_.

  • Subsequent characters may be [a-zA-Z0-9_:]. Any other characters are replaced with _.

If the OutputMode directive is set to otel, it sanitizes names to ensure metrics are valid for any OTLP-compatible backend as follows:

  • The first character must be a letter [A-Za-z]. If it’s not, the metric name is prefixed with nx_.

  • Subsequent characters may be [A-Za-z0-9_.-/]. Any other characters are replaced with _.

  • The maximum length is 255 characters. If exceeded, the name is truncated and appended with …​.

  • Empty names are replaced with a safe default (nx_default).

Configuration

The om_otel module accepts the following directives in addition to the common module directives.

Required directives

URL

Specifies the URL for the module to send the event data.

HTTP(S) directives

AddHeader

This optional directive can be specified multiple times to add custom headers to each HTTP request.

Compression

This optional directive can be used to enable HTTP compression for outgoing HTTP messages. The possible values are none, gzip, and deflate (not supported together with the grpc format). By default, compression is disabled. Please note that some HTTP servers may not accept compressed HTTP requests. If a server doesn’t support a specific compression method, it may return 415 Unsupported Media Type errors in response to compressed requests.

HTTPBasicAuthUser

HTTP basic authentication username. You must also set the HTTPBasicAuthPassword directive to use HTTP authentication. This directive is not supported together with the grpc format.

HTTPBasicAuthPassword

HTTP basic authentication password. You must also set the HTTPBasicAuthUser directive to use HTTP authentication. This directive is not supported together with the grpc format.

HTTPSAllowExpired

This boolean directive specifies whether the connection should be allowed with an expired certificate. If set to TRUE, the remote host will be able to connect with an expired certificate. The default is FALSE: the certificate must not be expired.

This directive is not supported when using the grpc format.

HTTPSAllowUntrusted

This boolean directive specifies that the connection should be allowed without certificate verification. If set to TRUE, the connection will be allowed even if the remote host presents an unknown or self-signed certificate. The default value is FALSE: the remote host must present a trusted certificate.

This directive is not supported when using the grpc format.

HTTPSCADir

This directive specifies a path to a directory containing certificate authority (CA) certificates. These certificates will be used to verify the certificate presented by the remote host. The certificate files must be named using the OpenSSL hashed format, i.e. the hash of the certificate followed by .0, .1 etc. To find the hash of a certificate using OpenSSL:

$ openssl x509 -hash -noout -in ca.crt

For example, if the certificate hash is e2f14e4a, then the certificate filename should be e2f14e4a.0. If there is another certificate with the same hash then it should be named e2f14e4a.1 and so on.

A remote host’s self-signed certificate (which is not signed by a CA) can also be trusted by including a copy of the certificate in this directory.

The default operating system root certificate store will be used if this directive is not specified. Unix-like operating systems commonly store root certificates in /etc/ssl/certs. Windows operating systems use the Windows Certificate Store, while macOS uses the Keychain Access Application as the default certificate store. See Certification Authority (CA) certificates in the NXLog Platform User Guide for more information on using this directive.

This directive is not supported when using the grpc format.

HTTPSCAFile

This specifies the path of the certificate authority (CA) certificate that will be used to verify the certificate presented by the remote host. A remote host’s self-signed certificate (which is not signed by a CA) can be trusted by specifying the remote host certificate itself. In the case of certificates signed by an intermediate CA, the certificate specified must contain the complete certificate chain (certificate bundle).

HTTPSCertFile

This specifies the path of the certificate file that will be presented to the remote host during the HTTPS handshake.

HTTPSCertKeyFile

This specifies the path of the private key file that was used to generate the certificate specified by the HTTPSCertFile directive. This is used for the HTTPS handshake.

Optional directives

Format

Specifies the format used to send the events. The possible values are grpc, json, and protobuf. The default is grpc.

MappingFile

The path to a JSON schema file specifying how to interpret fields. By default, the module treats fields in records without a corresponding mapping as gauges and flattens nested objects using the dot notation.

See Metrics above for more information about the schema file.

OutputMode

Applies to metric records only. The accepted values are prometheus (default) or otel. See Name sanitization above for more information.

Proxy

This optional directive is used to specify the protocol, IP address (or hostname) and port number of the HTTP or SOCKS proxy server to be used. The format is protocol://hostname:port. This directive is not supported together with the grpc format.

Reconnect

This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and doubles on every attempt. If the duration of the successful connection is greater than the current reconnect interval, then the reconnect interval will be reset to 1 sec.

Examples

Example 1. Sending syslog messages over OTLP/HTTP

This configuration collects syslog messages from a file and parses them into structured data using the parse_syslog() procedure of the Syslog extension. It then forwards the records to an OpenTelemetry collector listening on the OTLP/HTTP logs endpoint.

nxlog.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input system_logs>
    Module    im_file
    File      '/var/log/syslog'
    <Exec>
        parse_syslog();
        $RecordType = 'Log'; (1)
    </Exec>
</Input>

<Output otel>
    Module    om_otel
    URL       http://192.168.1.123:4318/
    Format    json
</Output>

<Route r1>
    Path      system_logs => otel
</Route>
1 Sets the $RecordType field to Log, ensuring om_otel emits the record through the OTLP log pipeline.
Example 2. Sending JSON spans as traces

This configuration collects network traces in JSON format from a file and parses them into structured data using the parse_json() procedure of the JSON extension. It then forwards the records to an OpenTelemetry collector listening on the OTLP/HTTP traces endpoint.

nxlog.conf
<Extension json>
    Module    xm_json
</Extension>

<Input network_trace>
    Module    im_file
    File      '/var/log/network_trace.json'
    <Exec>
        parse_json();
        $RecordType = 'Trace'; (1)
    </Exec>
</Input>

<Output otel>
    Module    om_otel
    URL       http://192.168.1.123:4318/
    Format    json
    <Exec>
        if defined($operation) {
            $Name = $operation;
        }
        if defined($status) {
            if not defined($Status) { (2)
                $Status = ();
            }
            if integer($status) >= 400 {
                $Status('Code') = 2;
            }
            else {
                $Status('Code') = 1;
            }
        }
    </Exec>
</Output>

<Route r1>
    Path      network_trace => otel
</Route>
1 Sets the $RecordType field to Trace, ensuring om_otel serializes each record as an OTLP span.
2 Ensures the required $Status field exists with type Hash.
Example 3. Exporting metrics with a mapping file

This configuration collects metrics in JSON format from a file and parses them into structured data using the parse_json() procedure of the JSON extension. It then forwards the records to an OpenTelemetry collector as OTLP metrics.

nxlog.conf
<Extension json>
    Module       xm_json
</Extension>

<Input metrics>
    Module       im_file
    File         '/var/log/metrics.json'
    <Exec>
        parse_json();
        $RecordType = 'Metric'; (1)
    </Exec>
</Input>

<Output otel>
    Module       om_otel
    URL          http://192.168.1.123:4318/
    Format       json
    MappingFile  /opt/nxlog/etc/otlp_mapping.json (2)
</Output>

<Route r1>
    Path         metrics => otel
</Route>
1 Sets the $RecordType field to Metric, ensuring om_otel applies OTLP metric serialization.
2 The MappingFile specifies the path to the JSON schema file. Refer to Mapping file format above for an example.

© Copyright NXLog Ltd.

PRIVACY POLICY GENERAL TERMS OF BUSINESS