JSON (xm_json)

This module parses JSON-formatted telemetry data, such as logs, into structured fields. It also converts structured records into JSON output.

You can parse JSON data in one of two ways:

Call the parse_json() procedure to parse the $raw_event field or a string parameter as a JSON object. Use this method if your input module reads line-based data with one JSON object per line.
Set the InputType directive of your input module to the name of an xm_json module instance to parse the raw data directly. This method works with JSON that spans multiple lines, JSON arrays, and newline-delimited JSON. For an example, see Parsing multiple JSON objects and arrays.

You can convert records to JSON in one of two ways:

Call the to_json() function or procedure.
Set the OutputType directive of your output module to the name of an xm_json module instance to output records as elements in a JSON array. For an example, see Output a JSON array.

Be aware of these limitations:

JSON does not have a datetime type, so NXLog Agent saves these values as strings. The parser detects datetime values automatically, so you do not need to call parsedate().
parse_json() and to_json() do not support floating-point or integer values that are bigger than the 32-bit or 64-bit limits of your computer. They convert these values to strings to preserve the original data.
JSON keys are limited to 499 characters. A longer key causes a failure and can result in data loss.

To examine the supported platforms, see the list of installation packages.

Creating and populating fields

The parse_json() procedure parses a string containing a JSON object into structured data. It expects the $raw_event field or the string passed as a parameter to be in the following format:

{"Key1":"Value1","Key2":"Value2"}

When this procedure parses a record, it creates fields according to the keys and values in the JSON object. You can use these fields for further processing, or to convert the record into a different output format. For examples of how to parse JSON records and manipulate fields, see Parsing JSON logs below.

Input modules can create additional fields containing information about the log source. When you convert a record to a different format, these fields carry over to the output, which can increase memory and bandwidth usage.

Check the Fields section of your input module’s documentation to see which fields it adds, and test your configuration before deploying it. Use the delete() procedure or the xm_rewrite extension to remove any fields you do not need.

Extracting values from JSON

The extract_json() function and procedure extract a node’s value from a JSON tree. Both accept a JSONPath expression to select a node and an optional string containing the JSON data to parse. If you omit this string, they parse the content of the $raw_event core field.

Accepted JSONPath expressions

extract_json() implements a subset of JSONPath, expressed as $.a.['b']['c'].d[1].['e'][1]..f notation:

Node selection by name. For example, $.a or $['a'].
Array element selection by index, including multi-dimensional arrays. For example, $.a[1] or $.a[1][1][1] or $['a'][1][1][1].
Search for a named node in one nesting layer. For example, $..a.
Any sequence of the above expressions.

When specifying a $..a-type expression, extract_json() does not perform a recursive search for nodes but limits the scope to only child nodes of the current level. It returns the first match.

Returned data

The extract_json() function returns a string containing the node’s value or JSON if the matched node contains a JSON object.

The extract_json() procedure sets the value of the $raw_event field instead of returning a value.

Since extract_json() parses and rewrites data as a string, floating-point values always include 6 digits after the decimal point.

For an example, see Extracting fields from JSON objects below.

Configuration

The xm_json module accepts the following directives in addition to the common module directives.

Optional directives

DateFormat

Set this directive to specify the format of date and time strings in the output. It is similar to the global DateFormat but is independent of it. The default is YYYY-MM-DDThh:mm:ss.sTZ.

DetectNestedJSON

Set this directive to FALSE to deactivate autodetection of nested JSON strings when calling the to_json() function or procedure.

For example, if a field $key contains the string value {"subkey":42}, with DetectNestedJSON set to FALSE, to_json() produces {"key":"{\"subkey\":42}"}.

The default is TRUE, which produces valid nested JSON instead: {"key":{"subkey":42}}.

Flatten

Set this directive to TRUE to flatten nested JSON when calling the parse_json() procedure, creating field names with dot notation.

For example, with Flatten set to TRUE, the following JSON creates the fields $event.time and $event.severity:

{"event":{"time":"2026-08-22T00:00:00.000Z","severity":"ERROR"}}

The default is FALSE, which keeps nested JSON as a string field.

ForceUTF8

Set this directive to TRUE to ensure output JSON is valid UTF-8. The JSON specification requires JSON records to be UTF-8 encoded, and some tools fail to parse JSON if it is not valid UTF-8. When activated, the module replaces each invalid character in the output JSON with a question mark (?). The default is FALSE.

IncludeHiddenFields

Set this directive to FALSE to exclude fields starting with a dot (.) or underscore (_) in their names from JSON generated by the to_json() function or procedure. The default is TRUE; all fields are included.

ParseDate

Set this directive to FALSE to prevent the module from attempting to parse strings that begin with a 4-digit year (regular expression ^[12][0-9]{3}-) as timestamps.

The default is TRUE; the module attempts to parse strings that look like timestamps.

PrettyPrint

Set this directive to TRUE to pretty-print JSON output, writing each key-value pair on a new line with indentation. This adds line breaks to the JSON, which can cause parser errors in some tools that expect single-line JSON. The default is FALSE.

UnFlatten

Set this directive to TRUE to generate nested JSON when field names contain a dot (.) when calling the to_json() procedure.

For example, with UnFlatten set to TRUE, the two fields $event.time and $event.severity are converted to JSON as follows:

{"event":{"time":"2026-08-22T00:00:00.000Z","severity":"ERROR"}}

The default is FALSE, which outputs the following JSON:

{"event.time":"2026-08-22T00:00:00.000Z","event.severity":"ERROR"}

Functions

The following functions are exported by xm_json.

type: string extract_json(type: string jsonpath): Search the $raw_event field using the specified JSONPath expression. If successfully matched, it returns the value of the matched node. If no match is found, it returns an empty string.
type: string extract_json(type: string jsonpath, type: string json_data): Search json_data using the specified JSONPath expression. If successfully matched, it returns the value of the matched node. If no match is found, it returns an empty string.
type: unknown parse_json(type: string source): Parse the parameter string as JSON format into a simple or compound returning value.
type: string to_json(): Convert the fields to JSON and return this as a string value. Any field having a leading dot (.) or underscore (_) will be automatically excluded unless IncludeHiddenFields directive is set to TRUE. The existing $raw_event field is never included in the generated JSON.
type: string to_json(type: unknown value): Converts the value parameter of any simple or compound type to JSON and returns this as a string value.

Procedures

The following procedures are exported by xm_json.

extract_json(type: string jsonpath);: Search the $raw_event field using the specified JSONPath expression. If successfully matched, it rewrites the $raw_event field with the value of the matched node. If no match is found, $raw_event will be set to an empty string.
extract_json(type: string jsonpath, type: string json_data);: Search json_data using the specified JSONPath expression. If successfully matched, it rewrites the $raw_event field with the value of the matched node. If no match is found, $raw_event will be set to an empty string.
parse_json();: Parse the $raw_event field as JSON input.
parse_json(type: string source);: Parse the given string as JSON format.
to_json();: Convert the fields to JSON and write it to the $raw_event field. Any field starting with a dot (.) or underscore (_) will be automatically excluded unless the IncludeHiddenFields directive is set to TRUE. The $raw_event core field is not included when converting fields to JSON.

Examples

Converting logs to JSON

Example 1. Converting syslog to JSON format

This configuration receives logs over TCP and parses records into structured data with the parse_syslog() procedure. It then converts records to JSON using the to_json() procedure.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input tcp_listen>
    Module        im_tcp
    ListenAddr    0.0.0.0:1514
    <Exec>
        parse_syslog();
        to_json();
    </Exec>
</Input>

The following is a syslog message in BSD format.

Input sample

<30>Aug 24 15:45:43 SRV01.example.com acpid: 1 client rule loaded

The following JSON object shows the same event after NXLog Agent processed it.

Output sample

{
  "MessageSourceAddress": "127.0.0.1",
  "EventReceivedTime": "2026-08-24T15:46:24.386367+02:00",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "Hostname": "SRV01.example.com",
  "SyslogFacilityValue": 3,
  "SyslogFacility": "DAEMON",
  "SyslogSeverityValue": 6,
  "SyslogSeverity": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventTime": "2026-08-24T15:45:43.000000+02:00",
  "SourceName": "acpid",
  "Message": "1 client rule loaded"
}

Example 2. Converting Windows events to syslog-encapsulated JSON

This configuration collects Windows events and converts them to JSON format using the to_json() procedure. It then converts records to BSD syslog format using the to_syslog_bsd() procedure.

nxlog.conf

<Extension syslog>
    Module    xm_syslog
</Extension>

<Extension json>
    Module    xm_json
</Extension>

<Input windows_events>
    Module    im_msvistalog
    <Exec>
        $Message = to_json(); (1)
        to_syslog_bsd(); (2)
    </Exec>
</Input>

1	Converts the event to JSON format and writes the value to the `$Message` field.
2	Converts the record to BSD syslog, which uses the $Message field to populate the message part.

Output sample

<14>Aug  24 14:40:11 SRV01 Service_Control_Manager[1332]: {"EventTime":"2026-08-24T14:40:11.770503+02:00","Hostname":"SRV01","EventType":"INFO","SeverityValue":2,"Severity":"INFO","SourceName":"Service Control Manager","FileName":"System","EventID":7036,"CategoryNumber":0,"RecordNumber":6788,"Message":"The nxlog service entered the running state. ","EventReceivedTime":"2026-08-24T14:40:12.770211+02:00"}

Parsing JSON logs

Example 3. Parsing simple JSON

This configuration collects events in JSON format from a file and parses records into structured data using the parse_json() procedure. It processes the events and converts them back to JSON using the to_json() procedure.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Input server_logs>
    Module      im_file
    File        '/tmp/server_logs'
    <Exec>
        parse_json();

        delete($SourceModuleType); (1)
        delete($SourceModuleName);

        to_json();
    </Exec>
</Input>

1	Deletes unnecessary core fields.

The following is an event in JSON format. Although it is pretty printed for readability, the configuration expects one record per line.

Input sample

{
  "Timestamp": "2026-08-24T22:27:14+02:00",
  "Hostname": "SRV01",
  "Message": "The server has started.",
  "Severity": "INFO"
}

The following JSON object shows the same event after NXLog Agent processed it.

Output sample

{
  "EventReceivedTime":"2026-08-24T22:27:25.902492+02:00",
  "Hostname":"SRV01",
  "Timestamp":"2026-08-24T22:27:14.000000+02:00",
  "Message":"The server has started.",
  "Severity":"INFO"
}

Example 4. Parsing multiple JSON objects and arrays

The following input samples consist of the same events in different JSON objects.

JSON objects spanning multiple lines

{
  "Date": "2026-08-24", "Time": "22:50:12",
  "Message": "The process has started"
},
{
  "Message": "The process has stopped!"
}

Array of JSON objects

[{"Date": "2026-08-24", "Time": "22:50:12", "Message": "The process has started"}, {"Message": "The process has stopped!"}]

Newline-delimited JSON objects

{"Date": "2026-08-24", "Time": "22:50:12", "Message": "The process has started"}
{"Message": "The process has stopped!"}

NXLog Agent can parse all of these inputs by setting the InputType of the input module instance to an xm_json instance.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Input myapp>
    Module      im_file
    File        'C:\MyApp\app.log'
    InputType   json (1)
    <Exec>
        if defined $Date { (2)
            $EventTime = strptime($Date + $Time, "%Y-%m-%d %T");
        }
        else {
            $EventTime = $EventReceivedTime;
        }

        delete($Date); (3)
        delete($Time);
        delete($EventReceivedTime);
        delete($SourceModuleType);

        to_json(); (4)
    </Exec>
</Input>

1	Sets the InputType to the xm_json module instance.
2	If the `$Date` field exists, it combines it with the `$Time` field to create the `$EventTime` field. Otherwise, it sets the `$EventTime` field to the value of the `$EventReceivedTime` core field.
3	Deletes unnecessary fields.
4	Converts the event back to JSON using the to_json() procedure.

The following JSON objects show the same events after NXLog Agent processed them. The output is the same for all three input samples above.

Output sample

{"Hostname":"SRV01","Message":"The process has started","EventTime":"2026-08-24T22:50:12.000000+02:00","SourceModuleName":"myapp"}
{"Hostname":"SRV01","Message":"The process has stopped!","EventTime":"2026-08-24T22:51:26.979726+02:00","SourceModuleName":"myapp"}

Example 5. Extracting fields from JSON objects

This configuration collects JSON-formatted events using the File input module, which populates the $raw_event core field with the log line read from the file. It then uses the extract_json() procedure to extract the EventData node from the original event and rewrite the $raw_event field.

nxlog.conf

<Extension json>
    Module    xm_json
</Extension>

<Input myapp>
    Module    im_file
    File      '/var/log/myapp'
    Exec      extract_json("$.EventData");
</Input>

The following is an event in JSON format. Although it is pretty printed for readability, the configuration expects one record per line.

Input sample

{
  "Stream": "stdout",
  "EventData": {
    "EventTime": "2026-08-24 09:35:12",
    "Message": "The process has started",
    "Severity": "INFO"
  }
}

The following JSON object shows the same event after NXLog Agent processed it.

Output sample

{
  "EventTime": "2026-08-24 09:35:12",
  "Message": "The process has started",
  "Severity": "INFO"
}

Example 6. Parsing compound data

This example demonstrates how to use the parse_json() function to process JSON logs containing composite data. The function converts the raw JSON event into an object, allowing you to access individual fields.

Input sample

{"message": "Session opened for user jdoe","metadata":{"event_type":"SSH Login","log_level":3}}
{"message": "Session closed"}

This configuration collects logs from a file and parses log records using the parse_json() function. Variables prefixed with `$$`are module variables, which means they are only accessible by that module instance and not propagated through the data processing pipeline.

nxlog.conf

<Extension json>
    Module    xm_json
</Extension>

<Input input_file>
    Module    im_file
    File      '/path/to/log/file'
    <Exec>
        $$parsed = json->parse_json($raw_event);             (1)
        log_info("Parsed type: " + type($$parsed));          (2)
        log_info("Message value: " + $$parsed("message"));   (3)
        log_info("Metadata value: " + $$parsed("metadata")); (4)
    </Exec>
</Input>

1	Parses the raw JSON event and stores the structured result in the `$$parsed` variable. The `parse_json()` function automatically detects whether the input is a JSON object or an array.
2	Logs the data type of the `$$parsed` variable, which should be `hash` for objects and `array` for lists. This verifies that the data was parsed correctly.
3	Extracts the value of the `Message` key from the parsed JSON. If the key is missing, it logs an empty value.
4	Retrieves the `Attributes` field from the JSON object. If this field contains a nested object, the structure is preserved. If the key is missing, it logs an empty value.

Notice that the second log entry in the output sample below has an empty Attributes value. If a field is missing from the JSON input, the parse_json() function sets its value to undef.

Output sample

2026-08-24 17:58:42 INFO [im_file|system_logs] Parsed type: hash
2026-08-24 17:58:42 INFO [im_file|system_logs] Message value: Session opened for user jdoe
2026-08-24 17:58:42 INFO [im_file|system_logs] Metadata value: ( 'event_type' => 'SSH Login', 'log_level' => 3 )
2026-08-24 17:58:42 INFO [im_file|system_logs] Parsed type: hash
2026-08-24 17:58:42 INFO [im_file|system_logs] Message value: Session closed
2026-08-24 17:58:42 INFO [im_file|system_logs] Metadata value:

Example 7. Parsing JSON arrays

The parse_json() function can automatically detect JSON arrays.

Input sample

[[101, "jdoe", "ssh_login"], [102, "asmith", "failed_ssh"], [103, "mjane", "session_closed"]]

nxlog.conf

<Extension json>
    Module    xm_json
</Extension>

<Input input_file>
    Module    im_file
    File      '/path/to/log/file'
    <Exec>
        $$parsed = json->parse_json($raw_event);                        (1)
        log_info("Parsed type: " + type($$parsed));                     (2)
        log_info("Second array item: " + $$parsed[1]);                  (3)
        log_info("First value of third array item: " + $$parsed[2][0]); (4)
    </Exec>
</Input>

1	Parses the raw JSON event and stores the structured result in the `$$parsed` variable. The `parse_json()` function automatically detects whether the input is a JSON object or an array.
2	Logs the data type of the `$$parsed` variable, which should be `array` in this case. This verifies that the data was parsed correctly.
3	Extracts the second element from the top-level array. Since JSON arrays use zero-based indexing, `$$parsed[1]` refers to the second item in the list.
4	Outputs the first element of the third nested array. If the data structure is different or the index is out of range, it will result in an error or an empty value.

Output sample

2026-08-24 18:14:40 INFO [im_file|in1] Parsed type: array
2026-08-24 18:14:40 INFO [im_file|in1] Second array item: [ 102, "asmith", "failed_ssh" ]
2026-08-24 18:14:40 INFO [im_file|in1] First value of third array item: 103

Output logs in JSON format

Example 8. Output a JSON array

This configuration collects events in JSON format from a file and parses records into structured data by specifying an xm_json module instance as the InputType. It processes the events and outputs them as a JSON array by specifying an xm_json module instance as the OutputType.

nxlog.conf

<Extension json>
    Module       xm_json
</Extension>

<Input myapp>
    Module       im_file
    File         'C:\MyApp\app.log'
    InputType    json (1)
</Input>

<Output file>
    Module       om_file

    <Exec>
        delete($EventReceivedTime); (2)
        delete($SourceModuleType);
    </Exec>

    OutputType   json (3)
    File         'C:\temp\nxlog-output.log'
</Output>

<Route r1>
    Path         myapp => file
</Route>

1	Sets the InputType to the xm_json module instance.
2	Deletes unnecessary core fields.
3	Sets the OutputType to the xm_json module instance.

The following are sample events in JSON format.

Input sample

{ "uid": 175, "user": "terry", "os": "linux", "active": false, "ssh": null }
[ { "uid": 235, "user": "alex", "os": "linux", "active": true, "ssh": null }, { "uid": 333, "user": "roger", "os": "windows", "active": false, "ssh": null } ]
{ "uid": 853, "user": "ben", "os": "windows", "active": false, "ssh": null }

The following JSON array shows the same events after NXLog Agent processed them. Note that the closing bracket of the array is added to the stream only when NXLog Agent closes the file.

Output sample

[ {"uid":175,"user":"terry","os":"linux","active":false,"ssh":null,"SourceModuleName":"myapp"}, {"uid":235,"user":"alex","os":"linux","active":true,"ssh":null,"SourceModuleName":"myapp"}, {"uid":333,"user":"roger","os":"windows","active":false,"ssh":null,"SourceModuleName":"myapp"}, {"uid":853,"user":"ben","os":"windows","active":false,"ssh":null,"SourceModuleName":"myapp"} ]

Example 9. Output compound data

Some SIEMs require sending data using a specific schema, which may include compound values. You can output compound values with the xm_json module in multiple ways. The configuration in these examples uses the Test Generator module to generate events in the following format.

Input sample

0@Mon Aug 24 13:17:46 2025

The first method is to use the to_json() function. However, this method may not process complex structures correctly, such as JSON objects inside arrays or nested arrays in JSON objects.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Input test>
    Module      im_testgen
    MaxCount    1
    <Exec>
        $generated = ( 'Message' => 'This is a test', (1)
                       'Attributes' => ( 'Type' => 'debug',
                                         'Value' => 10
                                       )
                     );

        log_info("Generated type: " + type($generated)); (2)
    </Exec>
</Input>

<Output file>
    Module      om_file
    File        '/var/log/nxlog-output.json'
    Exec        to_json(); (3)
</Output>

<Route r1>
    Path        test => file
</Route>

1	Creates a Hash field called `$generated` comprising two keys: `Message` and `Attributes`.
2	Logs the data type of the `$generated` field. This verifies the data is stored as a hash.
3	Converts the record, including the `$generated` field, into a JSON-formatted string and writes it to the `$raw_event` core field.

The following JSON object shows the event after NXLog Agent processed it.

Output sample

{
  "SeverityValue": 2,
  "EventTime": "2026-08-24T13:17:46.014673+02:00",
  "SourceName": "nxlog",
  "ProcessID": 26368,
  "EventReceivedTime": "2026-08-24T13:17:46.014673+02:00",
  "SourceModuleName": "test",
  "SourceModuleType": "im_testgen",
  "Hostname": "PC01",
  "generated": {
    "Message": "This is a test",
    "Attributes": {
      "Type": "debug",
      "Value": 10
    }
  }
}

An alternative approach is to use the OutputType directive. This method structures the output differently but does not allow for formatting adjustments.

nxlog.conf

<Extension json>
    Module       xm_json
</Extension>

<Input test>
    Module       im_testgen
    MaxCount     1
    <Exec>
        $generated = ( 'Message' => 'This is a test', (1)
                       'Attributes' => ( 'Type' => 'debug',
                                         'Value' => 10
                                       )
                     );

        log_info("Generated type:" + type($generated)); (2)
    </Exec>
</Input>

<Output file>
    Module        om_file
    File          '/var/log/nxlog-output.json'
    OutputType    json (3)
</Output>

<Route r1>
    Path        test => file
</Route>

1	Creates a Hash field called `$generated` comprising two keys: `Message` and `Attributes`.
2	Logs the data type of the `$generated` field. This verifies the data is stored as a hash.
3	Specifies that the output format should match the structure of events from the `json` (xm_json) module instance. This method maintains the event’s original structure, including any automatically added event fields, but does not allow for custom formatting or pretty-printing.

The following JSON array shows the event after NXLog Agent processed it. Note that the closing bracket of the array is added to the stream only when NXLog Agent closes the file.

Output sample

[
  {
    "SeverityValue": 2,
    "EventTime": "2026-08-24 13:17:46",
    "SourceName": "nxlog",
    "ProcessID": 26368,
    "EventReceivedTime": "2026-08-24 13:17:46",
    "SourceModuleName": "test",
    "SourceModuleType": "im_testgen",
    "Hostname": "PC01",
    "generated": {
      "Message": "This is a test",
      "Attributes": {
        "Type": "debug",
        "Value": 10
      }
    }
  }
]