OpenTelemetry Collector (im_otel)

This module accepts HTTP(S) and GRPC connections using the OpenTelemetry Protocol (OTLP) Specification. It supports traces and logs and can be a drop-in replacement for other receivers.

OpenTelemetry, or OTel, is an open-source observability framework that provides APIs and other tools for collecting and exporting telemetry data, such as traces, metrics, and logs. OTel is vendor-agnostic, and many observability platforms, libraries, services, and applications integrate it. It supports HTTP/1.1 and HTTP/2 connections.

This module may have limited log ingestion capacity (EPS), depending on the environment. If this limit is exceeded, errors such as StatusCode.UNAVAILABLE may appear in the agent logs.

This issue can occur if the buffer becomes full, preventing the module from accepting new log events. In such cases, it may be necessary to increase the value of LogqueueSize and BacklogCount instead of using the default settings, especially in high-throughput environments.

To examine the supported platforms, see the list of installation packages.

Configuration

The im_otel module accepts the following directives in addition to the common module directives.

Optional directives

HTTPSCertFile

Specifies the path of the certificate file that will be presented to the remote client during the HTTPS handshake. If this directive is not specified, the module uses HTTP.

HTTPSCertKeyFile

Specifies the path of the private key file that was used to generate the certificate specified by the HTTPSCertFile directive. This is used for the HTTPS handshake. If this directive is not specified, the module uses HTTP.

ListenAddrGRPC

The module will accept GRPC connections on this IP address and port. The default value is 0.0.0.0:4317.

IPv6 addresses must be enclosed in square brackets, i.e., [host]:port.

ListenAddrHTTP

The module will accept HTTP connections on this IP address and port. The default value is 0.0.0.0:4318.

IPv6 addresses must be enclosed in square brackets, i.e., [host]:port.

BacklogCount

Specifies the number of received messages that will be buffered. Once the buffer is full, the module stops accepting new messages. The default value is 100.

Fields

The following fields are used by im_otel.

$raw_event (type: string): A list of event fields in key-value pairs.

$Attributes (type: hash): Additional event attributes that describe the specific occurrence. See Attribute Limits for more information on attribute restrictions. Present in Log and Trace records.

$Body (type: unknown): [Optional] A value containing the body of the log record, such as a user-friendly message (including multi-line) describing the event or event attributes in a structured form (e.g., arrays and maps). Only present in Log records.

$DroppedAttributesCount (type: integer): The number of attributes that were discarded. Attributes may be discarded if their keys are too long or there are too many attributes. See Configurable Parameters for more information on the default attribute limits. The value is 0 if no attributes are discarded. Present in Log and Trace records.

$DroppedEventsCount (type: integer): The number of discarded events. The value is 0 if no events are discarded. Only present in Trace records.

$DroppedLinksCount (type: integer): The number of discarded links. Links may be discarded when the maximum number is exceeded. The value is 0 if no links were discarded. Present in Log and Trace records.

$EndTimeUnixNano (type: integer): The span end time. On the client side, this is the time on the local machine when the span execution ends. On the server side, this is the time when the server application handler stops running. The value is in Unix epoch time (nanoseconds since 00:00:00 UTC on 1 January 1970). It is only present in Trace records.

$Events (type: array): A list of events. An event is a time-stamped annotation of the span consisting of a user-supplied text message and key-value pairs. Only present in Trace records.

$Events[i]('Attributes') (type: hash): A collection of key/value pairs of event attributes. See Attribute Limits for more information on attribute restrictions. Present in Log and Trace records.

$Events[i]('DroppedAttributesCount') (type: integer): The number of discarded attributes. The value is 0 if no attributes are discarded. Present in Log and Trace records.

$Events[i]('Name') (type: string): The name of the event. Only present in Trace records.

$Events[i]('TimeUnixNano') (type: integer): The time the event occurred. Only present in Trace records.

$Flags (type: integer)

[Optional] This is a bit field present in Log and Trace records.

Log: The 8 least significant bits are the trace flags according to the W3C Trace Context specification. The 24 most significant bits are reserved and must be set to 0. Readers must not assume that the 24 most significant bits will be zero and must correctly mask the bits when reading the 8-bit trace flag (use flags & LOG_RECORD_FLAGS_TRACE_FLAGS_MASK).
Trace: Bits 0-7 (8 least significant bits) are the trace flags as defined in W3C Trace Context specification. To read the 8-bit W3C trace flag, use flags & SPAN_FLAGS_TRACE_FLAGS_MASK. See trace-flags for the flag definitions.

$Kind (type: integer): A value to distinguish between spans generated in a particular context. For example, two spans with the same name may be distinguished using CLIENT (caller) and SERVER (callee) to identify queueing latency associated with the span. The possible values are 1 (INTERNAL), 2 (SERVER), 3 (CLIENT), 4 (PRODUCER), 5 (CONSUMER). Only present in Trace records.

$Links (type: array): A collection of links. Links are references from this span to a span in the same or a different trace. Only present in Trace records.

$Links[i]('Attributes') (type: Hash): A collection of key-value pairs of link attributes. See Attribute Limits for more information on attribute restrictions. Present in Log and Trace records.

$Links[i]('DroppedAttributesCount') (type: integer): The number of discarded attributes. The value is 0 if no attributes were discarded. Present in Log and Trace records.

$Links[i]('SpanId') (type: array): The unique identifier of the linked span. The ID is an 8-byte array. Only present in Trace records.

$Links[i]('TraceId') (type: array): A unique identifier of a trace that this linked span is part of. The ID is a 16-byte array. Only present in Trace records.

$Links[i]('TraceState') (type: string): The TraceState associated with the link. Only present in Trace records.

$Name (type: string): A description of the span’s operation. Only present in Trace records.

$ObservedTimeUnixNano (type: integer): The time when the collection system observed the event. Only present in Log records.

$ParentSpanId (type: array): The ID of this span’s parent span. If this is a root span, this field must be empty. The ID is an 8-byte array. Only present in Trace records.

$RecordType (type: string): The type of record. It can be 'Trace' or 'Log'.

$Resource (type: hash): The resource for the logs in this message. If this field is not set, the resource info is unknown. Present in Log and Trace records.

$Resource('Attributes') (type: hash): A set of attributes that describe the resource. See Attribute Limits for more information on attribute restrictions. Present in Log and Trace records.

$Resource('DroppedAttributesCount') (type: integer): The number of discarded attributes. The value is 0 if no attributes are discarded. Present in Log and Trace records.

$ResourceSchemaUrl (type: string): The schema URL, if known. The URL identifies the schema in which the resource data is recorded. See Schema URL for more information. This URL applies to the data in the &Resource field. Present in Log and Trace records.

$Scope (type: hash): [Optional] A message representing the instrumentation scope, such as the fully qualified name and version. Present in Log and Trace records.

$Scope('Attributes') (type: hash): Additional attributes that describe the scope. See Attribute Limits for more information on attribute restrictions. Present in Log and Trace records.

$Scope('DroppedAttributesCount') (type: integer): The number of attributes that were discarded. Attributes may be discarded if their keys are too long or there are too many attributes. The value is 0 if no attributes are discarded. Present in Log and Trace records.

$Scope('Name') (type: string): The name of the instrumentation scope. The value is empty if the name is unknown. Present in Log and Trace records.

$Scope('Version') (type: string): The scope version. Present in Log and Trace records.

$ScopeSchemaUrl (type: string): The Schema URL, if known. The URL identifies the schema in which the log data is recorded. See Schema URL for more information. Present in Log and Trace records.

$Severity (type: string): [Optional] An alias to $SeverityText. Only present in Log records.

$SeverityNumber (type: integer): [Optional] The numerical value of the severity. It is normalized to values described in the Log Data Model. Only present in Log records.

$SeverityText (type: string): [Optional] The severity text, or log level, is the original string representation as logged from the source. Only present in Log records.

$SpanId (type: array): [Optional] The unique identifier of a span within a trace. The ID is an 8-byte array assigned when the span is created. An ID with all zeroes OR of length other than 8 bytes is considered invalid (an empty string in OTLP/JSON is zero-length and thus is also invalid). Only present in Log records.

$StartTimeUnixNano (type: integer): The span start time. On the client side, this is the time on the local machine when the span execution starts. On the server side, this is the time when the server’s application handler starts running. The value is in Unix epoch time (nanoseconds since 00:00:00 UTC on 1 January 1970). It is only present in Trace records.

$Status (type: Hash): An optional final status for this span. The status is unavailable when the span’s status code is unset, i.e. assume STATUS_CODE_UNSET (code = 0). Present in Log and Trace records.

$Status('Code') (type: integer): The status code. Present in Log and Trace records.

$Status('Message') (type: String): A human readable error message intended for developers. Present in Log and Trace records.

$TimeUnixNano (type: integer): The time of the event in Unix epoch time (nanoseconds since 00:00:00 UTC on 1 January 1970). A value of 0 indicates an unknown or missing timestamp. Only present in Log records.

$TraceId (type: array): [Optional] The unique identifier of a trace. The ID is a 16-byte array, and all logs from the same trace share the same ID. An ID with all zeroes OR of length other than 16 bytes is considered invalid (an empty string in OTLP/JSON is zero-length and thus is also invalid). Only present in Log records.

$TraceState (type: string): Conveys information about the request position in multiple distributed tracing graphs. The trace state is in the W3C Trace Context format. See Tracestate Header and Trace Context Specification for more information about this field. Only present in Trace records.