Microsoft Azure Monitor (im_azuremonitor)
This module fetches logs from Azure services using the Azure Monitor Log Analytics API.
Prerequisites
To use this module, you need a Microsoft Entra ID application and the following details:
-
The Application (client) ID.
-
The Directory (tenant) ID.
-
A client secret or a certificate and private key pair for authentication.
Create a Microsoft Entra ID application
Follow the instructions to Register an application in Microsoft Entra ID. Once you have created the new application, take note of the Application (client) ID and the Directory (tenant) ID from the application Overview page.

Generate a client secret
To use a client secret to authenticate with Microsoft’s API, follow the instructions to Add a client secret. Once you create the new client secret, save it in a safe location because you will not be able to retrieve it again once you navigate away from the page.

Generate a certificate and private key
To use certificate-based authentication to authenticate with Microsoft’s APIs, you need an X.509 certificate and the private key used to generate the certificate. The following instructions guide you through creating the private key pair using OpenSSL.
-
Save the following script in a file named
gencertkey.sh
#!/bin/sh openssl req -new -nodes -newkey rsa:2048 -x509 -days 730 -keyout certkey.pem -out cert.pem #openssl pkcs12 -export -out cert.pfx -inkey certkey.pem -in cert.pem openssl x509 -outform der -in cert.pem -out cert.der cat cert.der |openssl dgst -binary -sha1 | openssl base64 >cert.base64thumb THUMBPRINT=`cat cert.base64thumb` echo "ThumbPrint:$THUMBPRINT" UUID=`uuidgen` CERT=`cat cert.pem | grep -v "\-\-\-\-"| tr -d '\n'` cat <<EOF "keyCredentials": [ { "customKeyIdentifier":"$THUMBPRINT", "keyId":"$UUID", "type":"AsymmetricX509Cert", "usage":"Verify", "value":"$CERT" } ], EOF
-
The script depends on the
openssl
toolkit and theuuidgen
program. Install the corresponding packages if necessary.On Debian-based platforms:
# apt install openssl uuid-runtime
On Centos/Red Hat platforms:
# yum install openssl util-linux
-
Execute
gencertkey.sh
on the computer where NXLog Agent is installed to generate the certificate.$ ./gencertkey.sh
The output:
Generating a RSA private key ............+++++ ................................................+++++ writing new private key to 'certkey.pem' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]: State or Province Name (full name) [Some-State]: Locality Name (eg, city) []: Organization Name (eg, company) [Internet Widgits Pty Ltd]: Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []: Email Address []: ThumbPrint:0nFt3fB0JP7zuSmHaRQtmsFNYqo= "keyCredentials": [ { "customKeyIdentifier":"0nFt3fB0JP7zuSmHaRQtmsFNYqo=", "keyId":"629ab88d-1059-454b-b258-4ca05b46dee4", "type":"AsymmetricX509Cert", "usage":"Verify", "value":"MIIDXTCCAkWgAwIBAgIJAP+XrnwhAxjOMA0GCSqGSIb3DQEBCwUAMEUxCzAJB..." } ], --------
-
Take note of the following:
-
The path of the
certkey.pem
file and the value ofThumbPrint
. You will need these to configure NXLog Agent. -
The value of
KeyCredentials
. You will need this to configure the Microsoft Entra ID application.
-
-
Go to the Microsoft Entra ID application registration page, select Manifest on the left, and click Download.
-
Edit the downloaded manifest file and replace the KeyCredentials value with the value from the previous step.
"keyCredentials": [ { "customKeyIdentifier":"0nFt3fB0JP7zuSmHaRQtmsFNYqo=", "keyId":"629ab88d-1059-454b-b258-4ca05b46dee4", "type":"AsymmetricX509Cert", "usage":"Verify", "value":"MIIDXTCCAkWgAwIBAgIJAP+XrnwhAxjOMA0GCSqGSIb3DQEBCwUAMEUxCzAJB..." } ],
-
Save the changes and upload the manifest to Microsoft Azure.
Pagination
The module polls the API every PollInterval seconds and fetches all new records, using the TimeGenerated
field to detect new records.
Because there is usually a delay between when the event is sent to the server and when it’s available through the API, the module polls for events between now() - PollInterval*2
seconds and now() - PollInterval
seconds.
At startup, the module waits for PollInterval*2
seconds before it polls the API unless InitialPollDelay is specified.
Configuration
The im_azuremonitor module accepts the following directives in addition to the common module directives.
Required directives
The following directives are required for the module to start.
The path of the private key file that was used to generate the certificate specified by the CertThumbprint directive. It is only required if you’re using certificates to authenticate to the Azure Monitor Log Analytics API. This directive is mutually exclusive with the ClientSecret directive. |
|
The thumbprint of the X.509 certificate. It is only required if you’re using certificates to authenticate to the Azure Monitor Log Analytics API. This directive is mutually exclusive with the ClientSecret directive. |
|
The |
|
The Microsoft Entra ID application client secret. It is only required if you’re using a client secret to authenticate to the Azure Monitor Log Analytics API. This directive is mutually exclusive with the CertThumbprint and CertKeyFile directives. |
|
The name of the Azure Monitor table to collect logs from. |
|
The ID of the Microsoft Entra ID tenant to connect to. |
|
The ID of the workspace that owns the table. |
HTTP(S) directives
The following directives are for configuring HTTP(S) connection settings.
This optional directive can be specified multiple times to add custom headers to each HTTP request. |
|||
This optional directive can be used to enable HTTP compression for outgoing HTTP messages.
The possible values are |
|||
HTTP basic authentication password. You must also set the HTTPBasicAuthUser directive to use HTTP authentication. |
|||
HTTP basic authentication username. You must also set the HTTPBasicAuthPassword directive to use HTTP authentication. |
|||
Specifies if the connection should be allowed with an expired certificate.
If set to |
|||
Specifies if the connection should be allowed without certificate verification.
If set to |
|||
The path to a directory containing certificate authority (CA) certificates. These certificates will be used to verify the certificate presented by the remote host. The certificate files must be named using the OpenSSL hashed format, i.e. the hash of the certificate followed by .0, .1 etc. To find the hash of a certificate using OpenSSL:
For example, if the certificate hash is A remote host’s self-signed certificate (which is not signed by a CA) can also be trusted by including a copy of the certificate in this directory. The default operating system root certificate store will be used if this directive is not specified.
Unix-like operating systems commonly store root certificates in In addition, Microsoft’s PKI repository contains root certificates for Microsoft services. |
|||
The path of the certificate authority (CA) certificate that will be used to verify the certificate presented by the remote host. A remote host’s self-signed certificate (which is not signed by a CA) can be trusted by specifying the remote host certificate itself. In case of certificates signed by an intermediate CA, the certificate specified must contain the complete certificate chain (certificate bundle). |
|||
The path of the certificate file that will be presented to the remote host during the HTTPS handshake. This directive is only required when using a Proxy that uses TLS/SSL. |
|||
The path of the private key file that was used to generate the certificate specified by the HTTPSCertFile directive. This is used for the HTTPS handshake. This directive is only required when using a Proxy that uses TLS/SSL. |
|||
This optional directive is used to specify the protocol, IP address (or hostname) and port number of the HTTP or SOCKS proxy host to be used.
The format is |
|||
This optional directive sets the reconnect interval in seconds. If it is set, the module attempts to reconnect in every defined second. If it is not set, the reconnect interval will start at 1 second and double with every attempt. If the duration of the successful connection is greater than the current reconnect interval, then the reconnect interval will be reset to 1 sec.
|
|||
This optional directive defines the behavior when the connection with the remote host is lost.
When set to |
Polling directives
This optional directive is used to enable or disable the polling interval adjustment mechanism.
|
|
This optional directive specifies the initial delay in seconds before the module starts polling at startup. The default value is PollInterval*2 seconds. |
|
This optional directive specifies the polling interval, in seconds. The polling interval defines how frequently the module will check for new events. This value must be a positive integer greater than 0. The default value is 60 seconds.
|
Optional directives
The URL for authenticating with the API.
The default value is |
|||||||||||||||||||||||||||||||||||||
Specify a comma-separated list of table columns to retrieve. Column names are case-sensitive. If not specified, the module retrieves all columns. |
|||||||||||||||||||||||||||||||||||||
One or more
The queries must comply with the Kusto Query Language (KQL).
Any queries not starting with the |
|||||||||||||||||||||||||||||||||||||
Use this boolean directive to turn on connection logging.
Since connections can be very frequent, it could generate a lot of noise.
However, it helps troubleshoot connection issues.
The default value is |
|||||||||||||||||||||||||||||||||||||
This optional boolean directive instructs the module on where to start reading events from the log source. Reading all events can result in a lot of messages and is usually not the expected behavior. When The following matrix shows the outcome of this directive in conjunction with the SavePos directive:
|
|||||||||||||||||||||||||||||||||||||
This optional boolean directive instructs the module whether to save the position of the last read event before NXLog Agent exits. On the next startup, NXLog Agent will try to read the saved position from the cache file. This directive in conjunction with the ReadFromLast directive allows for resuming reading events directly from the saved position. When This directive can be overridden by the global NoCache directive.
If NoCache is |
|||||||||||||||||||||||||||||||||||||
This optional directive specifies the time of the first event to pull in RFC3339 format. If this directive is not set, the module reads events according to the ReadFromLast directive. |
|||||||||||||||||||||||||||||||||||||
The URL of the Azure Monitor Log Analytics API.
The default value is |
Creating and populating fields
When the im_azuremonitor module reads log records from the Azure Monitor Log Analytics API, it creates and populates fields according to the returned Columns.
You can use the fields for further processing or to convert the log record to a different output format, such as JSON or XML.
Additionally, the $raw_event
core field is populated as follows:
TimeGenerated Hostname Severity field1=value1 field2=value2 EventType=TableName
The EventType
field is populated with the TableName.
If the EventTime
, Hostname
, and Severity
fields are not returned as part of the log record, the following default values are used:
- Hostname
-
The value returned by the hostname() function.
- Severity
-
INFO
Examples
This configuration collects logs from a table named MyLogs.
im_azuremonitor receives records from the Log Analytics API in JSON format, which it then parses into structured data and writes each record as a list of key-value pairs to the $raw_event
field.
<Input azure_table> Module im_azuremonitor ClientId azure_ad_app_id TenantId azure_ad_tenant_id ClientSecret azure_app_client_secret WorkspaceId workspace_id TableName MyLogs </Input>
The following is a log record after it was processed by NXLog Agent.
2024-11-15 12:23:15 nxlog-server INFO MessageSourceAddress="192.168.0.123" SourceModuleName="tcp_listen" SourceModule="im_tcp" Message:"Connection accepted from 192.168.1.200" TenantId="c1580b88-581a-4cd0-a5c8-a3dd46241740" Type="MyLogs" _ResourceId="" EventType="MyLogs"
This configuration collects logs from a table named MyLogs.
It specifies a Filter to retrieve only logs containing "test" in the Message
field.
im_azuremonitor receives records from the Log Analytics API in JSON format, which it then parses into structured data and writes each record as a list of key-value pairs to the $raw_event
field.
<Input azure_table> Module im_azuremonitor ClientId azure_ad_app_id TenantId azure_ad_tenant_id ClientSecret azure_app_client_secret WorkspaceId workspace_id TableName MyLogs Filter 'Where Message has "test"' </Input>
The following is a matching log record after it was processed by NXLog Agent.
2024-11-15 12:23:15 nxlog-server INFO MessageSourceAddress="192.168.0.123" SourceModuleName="tcp_listen" SourceModule="im_tcp" Message:"This is a test message" TenantId="c1580b88-581a-4cd0-a5c8-a3dd46241740" Type="MyLogs" _ResourceId="" EventType="MyLogs"
This configuration uses certificate-based authentication to connect to the Log Analytics API. See Generate a certificate and private key above.
<Input azure_table> Module im_azuremonitor ClientId azure_ad_app_id TenantId azure_ad_tenant_id WorkspaceId workspace_id TableName MyLogs CertKeyFile /path/to/certkey.pem (1) CertThumbprint certificate_thumbprint (2) </Input>
1 | The CertKeyFile directive specifies the path of the certificate’s private key file. |
2 | The CertThumbprint directive specifies the certificate’s thumprint. |