Google SecOps (om_chronicle)
Google SecOps is a cloud service built as a specialized layer on top of the core Google infrastructure. It is designed for enterprises to privately retain, analyze, and search the massive amounts of security and network telemetry they generate. Google SecOps normalizes, indexes, correlates, and analyzes the data to provide instant analysis and context on risky activity.
The om_chronicle module uses Google Security Operations Ingestion API to forward logs to Google SecOps through the udmevents and unstructuredlogentries endpoints. Event data is sent in batches, thus reducing HTTP response latency and improving data throughput.
| To examine the supported platforms, see the list of installation packages. |
Google Security Operations Ingestion API authentication
NXLog Agent supports two ways of authenticating with Google SecOps, depending on the Google Security Operations Ingestion API version:
- API v2 with OAuth 2.0 (recommended)
-
Authenticates with a Google Developer Service Account. Download the service account’s JSON private key file (which contains, among other fields, the RSA
private_keyandclient_email) and set its path with the CredentialsFile directive.See Configuring a Google Cloud service account and Google’s documentation on obtaining API authentication credentials.
- API v1 with access key (legacy)
-
Authenticates with a Google Security Operations Ingestion API access key set with the ChronicleKey directive. This method is deprecated but still supported for backward compatibility.
Output log format
Unstructured logs
Google SecOps has different data format requirements for each log type. See Supported log types and default parsers in the Google SecOps documentation for more information.
om_chronicle forwards unstructured log records over HTTP(S) as a JSON payload in the following format:
{
"log_type": "<log_type>",
"customer_id": "<customer_id>",
"entries": [
{
"log_text": "<json_escaped_raw_event>",
"ts_rfc3339": "<event_time>"
}
]
}
For example:
{
"log_type": "BIND_DNS",
"customer_id": "c8c65bfa-5f2c-42d4-9189-64bb7b939f2c",
"entries": [
{
"log_text": "<13>Oct 9 17:33:15 SERVER-1 09-Oct-2023 17:33:15.187 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)",
"ts_rfc3339": "2023-10-09T17:51:27.122616+02:00"
}
]
}
Required fields:
-
log_typeis the value specified by the LogType directive. -
log_textis the JSON-escaped value of the$raw_eventfield. The value of this field must be in the data format required by Google SecOps for the specified log type. For example, forBIND_DNSlogs, the data must be in syslog format, while forWINDOWS_DNS, it can be in JSON, XML, or syslog + key-value pairs format.
Optional fields:
-
customer_idis present if the API v1 CustomerId or API v2 CustomerId directive is specified. -
ts_rfc3339is present if the$EventReceivedTimecore field is present. If you do not want this field in the output, use the delete() procedure to remove$EventReceivedTime. This can be useful if the log text already contains a received date and time.
Additional metadata, including the NXLog Agent-specific fields $SourceModuleName and $SourceModuleType, is not included in the output unless these values have been written to the $raw_event field.
The processing required to achieve this depends on the format of the input data.
For example, if the input data is in JSON format, you need to:
-
Use parse_json() to parse
$raw_eventinto fields. -
Populate any additional custom fields as needed.
-
Use to_json() to convert the fields to JSON format and update the value of
$raw_event.
Structured logs
om_chronicle forwards structured log records over HTTP(S) as a JSON payload in the following format:
{
"metadata": {
"event_type": "NETWORK_CONNECTION",
"event_timestamp": "<event_time>",
"description": "<json_escaped_raw_event>",
"ingestion_labels": [
{
"key": "source_module_name",
"value": "<source_module_name>"
},
{
"key": "source_module_type",
"value": "<source_module_type>"
}
]
},
"network": {
"ip_protocol": "TCP"
},
"principal": {
"ip": "<message_source_address>"
},
"target": {
"application": "nxlog",
"ip": "127.0.0.1"
}
}
For example:
{
"metadata": {
"description": "16-May-2023 20:11:15.187 client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)",
"event_timestamp": "2023-05-16T20:11:27.421707+05:45",
"event_type": "NETWORK_CONNECTION",
"ingestion_labels": [
{
"key": "source_module_name",
"value": "tcp"
},
{
"key": "source_module_type",
"value": "im_tcp"
}
]
},
"network": {
"ip_protocol": "TCP"
},
"principal": {
"ip": "127.0.0.1"
},
"target": {
"application": "nxlog",
"ip": "127.0.0.1"
}
}
Required fields:
-
descriptionis the JSON-escaped value of the$raw_eventfield. -
event_timestampis the value of$EventReceivedTime. -
ingestion_labels.source_module_nameis the value of$SourceModuleName, i.e., the name of the NXLog Agent input module instance. -
ingestion_labels.source_module_typeis the value of$SourceModuleType, i.e., the NXLog Agent input module.
If you remove any of these fields from the record, for example, with the delete() procedure, the field value is set to an empty string.
Configuration
|
Data sent with this module must be encoded in UTF-8. Otherwise, the destination will refuse and drop it. If the source does not guarantee that data is UTF-8 encoded, use the xm_charconv extension module to convert it to UTF-8. See Auto-detect and convert character sets in the NXLog Platform User Guide for an example. |
The om_chronicle module accepts the following directives in addition to the common module directives. The module can use the Google Security Operations Ingestion API v2 or the legacy API v1, and works in either unstructured or structured mode.
Required directives
The following directives are required for the module to start, depending on the Google Security Operations Ingestion API version in use.
Google Security Operations Ingestion API v2 authentication directives
Set this directive to specify the Google Security Operations Ingestion API version to use.
The accepted values for API v2 are |
|
Set this directive to the path of the private key file of the Google Developer Service Account required for authenticating with Google Security Operations Ingestion API v2. The API responds with HTTP error 403 if this key is not valid. See Configuring a Google Cloud service account for more information. |
|
Set this directive to the unique identifier (UUID) corresponding to your Google SecOps instance. The ID is provided by your Google SecOps representative. The API responds with HTTP error 400 or 403 if the UUID is not valid. This directive is required for API v2. |
Google Security Operations Ingestion API v1 authentication directives
| The Google Security Operations Ingestion API v1 is a legacy version. Use API v2 for new deployments. |
Set this directive to specify the Google Security Operations Ingestion API version to use.
The accepted values for API v1 are |
|
Set this directive to a Google Security Operations Ingestion API access key required for API v1. The API responds with HTTP error 403 if this key is not valid. |
Unstructured output format directives
The following directives define how the module formats the log records it sends to Google SecOps in unstructured mode.
Set this directive to specify the Google Security Operations Ingestion API endpoint to use.
The accepted values for unstructured mode are This mode supports forwarding unstructured logs to the unstructuredlogentries endpoint. Log data normalization is then handled by Google SecOps. |
|
Set this directive to a valid log type.
To retrieve the list of possible log types, visit This directive is required in unstructured mode. |
Structured output format directives
The following directives define how the module formats the log records it sends to Google SecOps in structured mode.
Set this directive to specify the Google Security Operations Ingestion API endpoint to use.
The accepted values for structured mode are This mode supports forwarding logs to the udmevents endpoint. Log data must be in the UDM format supported by Google SecOps. |
|
Set this directive to the name of a custom schema file to override the default mapping used to convert NXLog Agent fields to the Google SecOps UDM format. |
|
Set this block directive to map schema names to schema files, one pair per line ( |
|
Set this directive to the path of the directory containing the custom schema files defined by the ChronicleSchema and SchemaMap directives. The path can be relative to the configuration directory. The default is the configuration directory. |
HTTPS directives
The following directives configure secure data transfer via HTTPS.
Set this directive to |
|
Set this directive to |
|
Set this directive to the path of a directory containing Certificate Authority (CA) certificates.
The module uses these certificates to verify the certificate presented by the remote host.
Name the certificate files using the OpenSSL hashed format: the hash of the certificate followed by
For example, if the certificate hash is To trust a remote host’s self-signed certificate, include a copy of it in this directory. If this directive isn’t specified, NXLog Agent falls back to the default operating system root certificate store.
Unix-like operating systems commonly store root certificates in In addition, Microsoft’s PKI repository contains root certificates for Microsoft services. |
|
Set this directive to the path of the Certificate Authority (CA) certificate used to verify the certificate presented by the remote host. To trust a remote host’s self-signed certificate, specify the remote host’s certificate itself. For certificates signed by an intermediate CA, the specified certificate must contain the complete certificate chain (certificate bundle). |
|
Set this directive to the path of the certificate file to present to the remote host during the HTTPS handshake. |
|
Set this directive to the path of the private key file corresponding to the certificate specified by HTTPSCertFile. |
Optional directives
Set this directive to a custom header to add to each HTTP request. Specify this directive multiple times to add more than one header. |
|||
Set this directive to enable HTTP compression for outgoing HTTP messages.
The accepted values are Some HTTP servers may not accept compressed HTTP requests.
If the server doesn’t support the specified compression method, it may return |
|||
Set this directive to the HTTP basic authentication password. This directive requires you to also set the HTTPBasicAuthUser directive to use HTTP authentication. |
|||
Set this directive to the HTTP basic authentication username. If the HTTPBasicAuthPassword directive is not set, the module attempts HTTP Basic authentication using an empty password. |
|||
Set this directive to the maximum request size, in bytes. The default is 64 MB. |
|||
Set this directive to route outgoing requests through an HTTP or SOCKS proxy host.
The format is |
|||
This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and double with every attempt. In the latter case, when the system decides that the reconnection is successful, the reconnect interval is immediately reset to 1 sec.
|
|||
This optional directive defines the behavior when the connection with the remote host is lost.
When set to |
|||
Set this directive to a regional API endpoint to use.
The value is used verbatim as the hostname prefix.
For example, specifying The default is the United States Multi-Region endpoint ( |
|||
Set this directive to the unique identifier (UUID) corresponding to your Google SecOps instance. The ID is provided by your Google SecOps representative. The API responds with HTTP error 400 or 403 if the UUID is not valid. This directive is optional for Google Security Operations Ingestion API v1 in unstructured mode, but rejected in structured mode. |
|||
Set this directive to the Google Security Operations Ingestion API endpoint to use. Alternatively, omit this directive and let the module build the default endpoint from the optional ChronicleRegion, API v2
API v1 (legacy)
Depending on the URL provided, the module operates in plain HTTP or HTTPS mode. If you do not specify the port number in the URL, it uses port 80 for HTTP and port 443 for HTTPS.
|
Examples
This configuration collects Okta logs and forwards them to Google SecOps.
Since the OKTA LogType requires events in JSON format, the configuration converts log records to JSON using the to_json() procedure of the xm_json module.
# Google SecOps instance UUID, required.
define CUSTOMER_ID <YOUR_SECOPS_UUID>
<Extension json>
Module xm_json
</Extension>
<Output secops>
Module om_chronicle
ChronicleVersion 2
CustomerId %CUSTOMER_ID%
CredentialsFile '/opt/nxlog/etc/credentials.json'
ChronicleMode unstructured
LogType OKTA
<Exec>
$metadata.namespace = "OKTA_System_Logs"; (1)
to_json();
</Exec>
</Output>
| 1 | Adds a metadata label specifying the asset namespace. |
The following is a JSON-formatted log sample that will be sent to Google SecOps.
The ts_rfc3339 field is added by the module and corresponds to the $EventReceivedTime core field.
{
"customer_id": "c8c65bfa-5f2c-42d4-9189-64bb7b939f2c",
"log_type": "OKTA",
"entries": [
{
"ts_rfc3339": "2025-05-05T08:30:23.600622Z",
"log_text": "{\"legacyEventType\":null,\"severity\":\"WARN\",\"published\":\"2025-04-30T07:59:07.615Z\",\"actor\":{\"alternateId\":\"jsmith@example.com\",\"detailEntry\":null,\"displayName\":\"John Smith\",\"id\":\"00un5wtfa5WndtP4D5d7\",\"type\":\"User\"},\"debugContext\":{\"debugData\":{\"dtHash\":\"efa3b476edf8fde3ac23a45d411f8d6b458d4babbe1b1d3749b122583165298c\",\"operationRateLimitScopeType\":\"token\",\"operationRateLimitSecondsToReset\":\"14\",\"operationRateLimitSubtype\":\"ssws_token\",\"operationRateLimitThreshold\":\"10\",\"operationRateLimitTimeSpan\":\"1\",\"operationRateLimitTimeUnit\":\"MINUTES\",\"operationRateLimitType\":\"web_request\",\"requestId\":\"4a4df281828074beb313651fc0522576\",\"requestUri\":\"\/api\/v1\/logs\",\"url\":\"\/api\/v1\/logs?since=2025-04-05T09%3A46%3A29.126Z&limit=100&sortOrder=ASCENDING&after=1745999826225_1\"}},\"uuid\":\"fe0f2feb-2598-11f0-a0c6-09922710ae23\",\"transaction\":{\"detail\":{\"requestApiTokenId\":\"00T2xa8ze5IzSGp4q5d7\",\"rootApiTokenId\":\"00T2xa8ze5IzSGp4q5d7\"},\"id\":\"4a4df281828074beb313651fc0522576\",\"type\":\"WEB\"},\"version\":\"0\",\"authenticationContext\":{\"authenticationProvider\":null,\"authenticationStep\":0,\"credentialProvider\":null,\"credentialType\":null,\"externalSessionId\":\"trsJYq2Zrr5Sd-WS0bLerq9-A\",\"interface\":null,\"issuer\":null,\"rootSessionId\":\"trsJYq2Zrr5Sd-WS0bLerq9-A\"},\"client\":{\"device\":\"Unknown\",\"geographicalContext\":null,\"id\":null,\"ipAddress\":\"45.243.71.123\",\"userAgent\":{\"browser\":\"UNKNOWN\",\"os\":\"Unknown\",\"rawUserAgent\":\"nxlog\/6.7.9957\"},\"zone\":\"null\"},\"eventType\":\"system.operation.rate_limit.violation\",\"target\":[{\"alternateId\":null,\"detailEntry\":null,\"displayName\":null,\"id\":\"b192d91c-b242-36da-9332-d97a5579f865\",\"type\":\"Bucket UUID\"},{\"alternateId\":\"unknown\",\"detailEntry\":null,\"displayName\":\"test1\",\"id\":\"00T2xa8ze5IzSGp4q5d7\",\"type\":\"Token\"}],\"outcome\":null,\"device\":null,\"securityContext\":{\"asNumber\":null,\"asOrg\":null,\"domain\":null,\"isProxy\":null,\"isp\":null},\"displayMessage\":\"This API token has made too many requests\",\"request\":{\"ipChain\":[{\"geographicalContext\":null,\"ip\":\"45.243.71.123\",\"source\":null,\"version\":\"V4\"}]},\"EventReceivedTime\":\"2025-05-05T11:30:23.600622+03:00\",\"SourceModuleName\":\"okta\",\"SourceModuleType\":\"im_okta\",\"Hostname\":\"nxp\",\"metadata.namespace\":\"OKTA_System_Logs\"}"
}
]
}
This configuration collects logs from a BIND 9 DNS server and forwards them to Google SecOps in UDM format. It parses log records into structured data with a regular expression and uses a custom ChronicleSchema file in the same folder as the NXLog Agent configuration.
# Google SecOps instance UUID, required.
define CUSTOMER_ID <YOUR_SECOPS_UUID>
<Input dns>
Module im_file
File '/var/log/bind.log'
<Exec>
if $raw_event =~ /(?x)^(?<EventTime>\S+\s\S+)\s(?<category>.*):\s
(?<severity>.*):\s.*\s(?<source>.*)\#
(?<port>\d+):\s+(?<message>.+)$/
{
$EventTime = parsedate($EventTime);
}
</Exec>
</Input>
<Output secops_udm>
Module om_chronicle
ChronicleVersion 2
CustomerId %CUSTOMER_ID%
CredentialsFile '/opt/nxlog/etc/credentials.json'
ChronicleMode structured
ChronicleSchema 'bind_dns.json'
</Output>
{
"metadata": {
"event_type": "NETWORK_DNS",
"event_timestamp": "$EventTime",
"ingested_timestamp": "$EventReceivedTime",
"description": "$message",
"product_name": "BIND 9",
"vendor_name": "ISC",
"product_event_type": "$category",
"severity_details": "$severity",
"ingestion_labels": [
{
"key": "source_module_name",
"value": "$SourceModuleName"
},
{
"key": "source_module_type",
"value": "$SourceModuleType"
}
]
},
"src": {
"ip": "$source",
"port": "$port"
},
"principal": {
"hostname": "$Hostname"
}
}
09-Oct-2023 17:33:15.187 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)
The following is the JSON-formatted log record that will be sent to Google SecOps.
{
"customer_id": "c8c65bfa-5f2c-42d4-9189-64bb7b939f2c",
"events": [
{
"metadata": {
"description": "query: example.com IN A + (100.90.80.102)",
"event_timestamp": "2023-10-09T17:35:17.165000+02:00",
"event_type": "NETWORK_DNS",
"ingested_timestamp": "2023-10-09T17:39:17.901029+02:00",
"ingestion_labels": [
{
"key": "source_module_name",
"value": "dns"
},
{
"key": "source_module_type",
"value": "im_file"
}
],
"product_event_type": "queries",
"product_name": "BIND 9",
"severity_details": "info",
"vendor_name": "ISC"
},
"principal": {
"hostname": "SERVER-1"
},
"src": {
"ip": "10.120.20.20",
"port": "4238"
}
}
]
}
This configuration sends logs to Google SecOps in UDM format.
The output instance implements schema-selection logic based on the $EventType field.
It configures SchemaDir to a folder relative to the NXLog Agent configuration file, and uses SchemaMap with set_schema() to dynamically select the UDM schema.
# Google SecOps instance UUID, required.
define CUSTOMER_ID <YOUR_SECOPS_UUID>
<Output secops_udm>
Module om_chronicle
ChronicleVersion 2
CustomerId %CUSTOMER_ID%
CredentialsFile '/opt/nxlog/etc/credentials.json'
ChronicleMode structured
SchemaDir 'secops-schemas'
<SchemaMap>
copy file-copy.json
delete file-delete.json
spawn process-create.json
default default.json
</SchemaMap>
<Exec>
if ($EventType == "File copied") {
set_schema("copy");
}
else if ($EventType == "File deleted") {
set_schema("delete");
}
else if ($EventType == "Process created") {
set_schema("spawn");
}
else {
set_schema("default");
}
</Exec>
</Output>
This configuration collects logs from a BIND 9 DNS server and forwards them to Google SecOps in unstructured format.
Since the BIND_DNS LogType requires events in syslog format, the configuration converts log records to syslog using the to_syslog_bsd() procedure of the xm_syslog module.
# Google SecOps instance UUID, required.
define CUSTOMER_ID <YOUR_SECOPS_UUID>
<Extension syslog>
Module xm_syslog
</Extension>
<Output secops>
Module om_chronicle
ChronicleVersion 2
CustomerId %CUSTOMER_ID%
CredentialsFile '/opt/nxlog/etc/credentials.json'
ChronicleMode unstructured
LogType BIND_DNS
<Exec>
$Message = $raw_event;
to_syslog_bsd();
</Exec>
</Output>
This configuration collects logs from a BIND 9 DNS server and forwards them to Google SecOps.
Since the BIND_DNS LogType requires events in syslog format, the configuration converts log records to syslog using the to_syslog_bsd() procedure of the xm_syslog module.
| This example uses the legacy Google Security Operations Ingestion API v1 and is provided for backward-compatibility reference only. Use API v2 for new deployments. |
# Google SecOps Ingestion API access key, required.
define API_KEY <YOUR_API_KEY>
<Extension syslog>
Module xm_syslog
</Extension>
<Output secops>
Module om_chronicle
ChronicleKey %API_KEY%
# Unstructured events
ChronicleMode unstructured
# Identifies the type of logs in the batch.
LogType BIND_DNS
<Exec>
$Message = $raw_event;
to_syslog_bsd();
</Exec>
</Output>
09-Oct-2023 17:33:15.187 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)
09-Oct-2023 17:34:16.120 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)
09-Oct-2023 17:35:17.165 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)
The following is the JSON-formatted log record that will be sent to Google SecOps.
The ts_rfc3339 field is added by the module and corresponds to the $EventReceivedTime core field.
{
"log_type": "BIND_DNS",
"entries": [
{
"log_text": "<13>Oct 9 17:33:15 SERVER-1 09-Oct-2023 17:33:15.187 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)",
"ts_rfc3339": "2023-10-09T17:51:27.122616+02:00"
},
{
"log_text": "<13>Oct 9 17:34:16 SERVER-1 09-Oct-2023 17:34:16.120 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)",
"ts_rfc3339": "2023-10-09T17:51:27.126157+02:00"
},
{
"log_text": "<13>Oct 9 17:35:17 SERVER-1 09-Oct-2023 17:35:17.165 queries: info: client 10.120.20.20#4238: query: example.com IN A + (100.90.80.102)",
"ts_rfc3339": "2023-10-09T17:51:27.126176+02:00"
}
]
}