Prometheus (om_prometheus)

This module exposes Prometheus-compatible metrics via an HTTP endpoint. It supports counters and gauges extracted from structured records.

Output data format

The module converts incoming records into a set of Prometheus-compatible key-value pairs. It processes events according to the specified mapping. It supports:

Counters — Values that accumulate over time.
Gauges — Values that represent the current state.

The module treats any numeric field without a corresponding mapping as a Gauge.

The module exposes metrics over HTTP in standard Prometheus exposition format. The following endpoints are available:

/metrics: Returns the current set of metrics in Prometheus format
/healthy: Always returns HTTP 200. Used for checking the probe health.
/ready: Always returns HTTP 200. Used for checking the probe readiness.

Nested structures in incoming records are automatically flattened using _ as the delimiter. For example, { "system": { "cpu": { "usage": 42.5 }}} becomes system_cpu_usage. The name and labels fields in the MappingFile are sanitized as well and must match the fully flattened keys.

The user must take into account the behavior of floating-point number representation: Incoming integer numbers greater than 2^53 will lose precision, and their output will be in scientific format.

Configuration

The om_prometheus module accepts the following directives in addition to the common module directives.

HTTPS directives

HTTPSCACertFile

The path of the certificate authority (CA) certificate that will be used to verify the certificate presented by the remote host. A remote host’s self-signed certificate (which is not signed by a CA) can be trusted by specifying the remote host certificate itself. In the case of certificates signed by an intermediate CA, the certificate specified must contain the complete certificate chain (certificate bundle).

HTTPSCertFile

The path of the certificate file that will be presented to the remote host during the HTTPS handshake.

HTTPSCertKeyFile

The path of the private key file that was used to generate the certificate specified by the HTTPSCertFile directive. This is used for the HTTPS handshake.

The key must be in PKCS#8 format. Outdated PKCS#1 RSA keys (e.g. starting with, -----BEGIN RSA PRIVATE KEY-----) are not supported and will cause initialization to fail.

Optional directives

ListenAddr

The IP address and port that the module will bind to and accept connections on. The default is 0.0.0.0:9400.

MappingFile

The path to a JSON schema file specifying how to interpret fields. This file should define a metrics object containing counter and gauge arrays as follows:

Each field in the incoming records can either be used as a metric or as a label, but not both.

If a field name appears as a label key for any metric in the MappingFile, that same field will be ignored as a metric—even if it holds a numeric value in some records.

{
  "metrics": {
    "counter": [
      {
        "name": "field_1",
        "labels": ["label_1", "label_2"]
      }
    ],
    "gauge": [
      {
        "name": "field_2",
        "labels": ["label_1", "label_3"]
      }
    ]
  }
}

The module treats fields in log records without a corresponding mapping as gauges.

Sanitize

Prometheus allows only [A-Za-z0-9:_] characters in metric and label names. When this boolean directive is enabled, the module automatically sanitizes all keys - including those defined in the mapping file. This ensures compliance with Prometheus naming rules.

When enabled, the module rewrites all keys in incoming structured data (e.g., from JSON records) to comply with Prometheus naming rules. This includes:

Replacing invalid characters with underscores (_). Please note that this also applies to the . character, so the mapping file must be constructed accordingly.
Prepending nx_ to keys that start with a digit or non-alphabetic character.

This affects both metric names and label keys derived from JSON field names.

The default is TRUE. When FALSE, invalid metric names may cause Prometheus to reject the data.

A user of om_prometheus should consider this behavior and ensure that no duplicate metrics are unintentionally created due to sanitization.

For example, the original keys 1metric and nx_1metric will both be sanitized to nx_1metric, potentially causing metric collisions.

For more information about Prometheus naming requirements, refer to the official Prometheus naming guidelines.

Examples

Example 1. Exposing metrics for Prometheus scraping

This configuration processes records obtained from an input module (which converts the input example json to nxlog record via xm_json→to_json) and exports them as metrics for Prometheus scraping. See the Prometheus <scrape_config> documentation for information on scraping targets.

nxlog.conf

<Output prometheus_export>
    Module         om_prometheus
    ListenAddr     0.0.0.0:9464
    MappingFile    /opt/nxlog/etc/prometheus-mapping.json
</Output>

The following mapping defines a counter and several gauge fields and their labels.

prometheus-mapping.json

{
  "metrics": {
    "counter": [
      {
        "name": "http_request_count",
        "labels": ["method", "region"]
      }
    ],
    "gauge": [
      {
        "name": "cpu_usage",
        "labels": ["host", "region"]
      },
      {
        "name": "memory_usage",
        "labels": ["host", "region"]
      },
      {
        "name": "disk_io",
        "labels": ["host", "region"]
      }
    ]
  }
}

Input sample

{"cpu_usage": 38.6, "memory_usage": 65.0, "disk_io": 78, "host": "server3", "region": "eu-central"}
{"cpu_usage": 38.7, "http_request_count": 33, "disk_io": 110, "new_parameter": 333.33, "method": "SET", "region": "eu-central"}

The following is a sample response to a request to http://localhost:9464/metrics.

Output sample

# TYPE cpu_usage gauge
cpu_usage{region="eu-central"} 38.7
# TYPE disk_io gauge
disk_io{region="eu-central"} 110
# TYPE http_request_count counter
http_request_count{method="SET",region="eu-central"} 33
# TYPE memory_usage gauge
memory_usage{host="server3",region="eu-central"} 65
# TYPE new_parameter gauge
new_parameter{region="eu-central"} 333.33