Nginx

This topic explains how to collect Nginx logs with NXLog Agent.

The Nginx web server supports error and access logging. Both types of logs can be written to a file, forwarded as syslog via UDP, or written as syslog to a Unix domain socket. The sections below provide a brief overview; see the Logging section of the Nginx documentation for more detailed information.

NXLog Agent can be configured to collect Nginx logs.

Nginx error log

The error_log directive configures the destination and log level for the error log. This directive can be given in the main (top-level) configuration context to override the default. It can also be specified at the http, stream, server, and location levels, where it will override the inherited setting from the higher levels.

Example 1. Collecting Nginx error logs from file with NXLog Agent

With the following directive, Nginx will log all messages of warn severity or higher to the specified log file.

nginx.conf

error_log /var/log/nginx/error.log warn;

Following is a log message generated by Nginx, an NXLog Agent configuration for parsing it, and the output result in JSON format.

Log sample

2022/10/25 13:48:09 [emerg] 19827#19827: bind() to [::]:80 failed (98: Address already in use)

nxlog.conf

<Extension json>
    Module    xm_json
</Extension>

<Input nginx_error>
    Module    im_file
    File      '/var/log/nginx/error.log'
    <Exec>
        if $raw_event =~ /^(\S+ \S+) \[(\S+)\] (\d+)\#(\d+): (\*(\d+) )?(.+)$/
        {
            $EventTime = strptime($1, '%Y/%m/%d %H:%M:%S');
            $NginxLogLevel = $2;
            $NginxPID = $3;
            $NginxTID = $4;
            if $6 != '' $NginxCID = $6;
            $Message = $7;
            to_json();
        }
    </Exec>
</Input>

Output sample

{
  "EventReceivedTime": "2022-10-25T13:48:10.360819+03:00",
  "SourceModuleName": "nginx_error",
  "SourceModuleType": "im_file",
  "EventTime": "2022-10-25T13:48:09.000000+03:00",
  "NginxLogLevel": "emerg",
  "NginxPID": "19827",
  "NginxTID": "19827",
  "Message": "bind() to [::]:80 failed (98: Address already in use)"
}

Example 2. Collecting Nginx error logs via syslog with NXLog Agent

With this directive, Nginx will forward all messages of warn severity or higher to the specified syslog server. The messages will be generated with the local7 facility.

nginx.conf

error_log syslog:server=127.0.0.1:514,facility=local7 warn;

This NXLog Agent configuration can be used to parse the logs.

nxlog.conf

<Extension json>
    Module        xm_json
</Extension>

<Extension syslog>
    Module        xm_syslog
</Extension>

<Input nginx_error>
    Module        im_udp
    ListenAddr    127.0.0.1:514
    <Exec>
        parse_syslog();
        if $Message =~ /^\S+ \S+ \[\S+\] (\d+)\#(\d+): (\*(\d+) )?(.+)$/
        {
            $NginxPID = $1;
            $NginxTID = $2;
            if $4 != '' $NginxCID = $4;
            $Message = $5;
            to_json();
        }
    </Exec>
</Input>

Output sample

{
  "MessageSourceAddress": "127.0.0.1",
  "EventReceivedTime": "2022-10-25T17:16:57.272942+03:00",
  "SourceModuleName": "nginx_error",
  "SourceModuleType": "im_udp",
  "SyslogFacilityValue": 23,
  "SyslogFacility": "LOCAL7",
  "SyslogSeverityValue": 0,
  "SyslogSeverity": "EMERG",
  "SeverityValue": 5,
  "Severity": "CRITICAL",
  "Hostname": "127.0.0.1",
  "EventTime": "2022-10-25T17:16:55.000000+03:00",
  "SourceName": "nginx",
  "Message": "bind() to [::]:80 failed (98: Address already in use)",
  "NginxPID": "19827",
  "NginxTID": "19827"
}

Example 3. Collecting Nginx error logs via Unix Domain Socket with NXLog Agent

With this directive, Nginx will forward all messages of warn severity or higher to the specified Unix domain socket. The messages will be sent in syslog format with the local7 syslog facility.

nginx.conf

error_log syslog:server=unix:/var/log/nginx/error.sock,facility=local7 warn;

nxlog.conf

<Extension syslog>
    Module    xm_syslog
</Extension>

<Input nginx_error>
    Module    im_uds
    UDS       /var/log/nginx/error.sock
    <Exec>
        parse_syslog();
        if $Message =~ /^\S+ \S+ \[\S+\] (\d+)\#(\d+): (\*(\d+) )?(.+)$/
        {
            $NginxPID = $1;
            $NginxTID = $2;
            if $4 != '' $NginxCID = $4;
            $Message = $5;
        }
    </Exec>
</Input>

Nginx access log

By default, Nginx access logs are written to logs/access.log in the Nginx Combined Log Format. An NXLog Agent configuration example for parsing these logs can be found in the NXLog Platform User Guide Parse logs in Common and Combined Log Format. Access logs can also be forwarded in syslog format via UDP or a Unix domain socket, as shown below.

The Nginx log format can be customized by setting the log_format directive; see the Nginx documentation for more information.

Example 4. Collecting Nginx access logs via syslog with NXLog Agent

With this directive, Nginx will forward access logs to the specified syslog server. The messages will be generated with the local7 facility and the info severity.

Log sample

127.0.0.1 - - [25/Oct/2022:18:50:34 +0300] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:105.0) Gecko/20100101 Firefox/105.0"

nginx.conf

access_log syslog:server=192.168.1.1:514,facility=local7,severity=info;

This NXLog Agent configuration can be used to parse the logs.

nxlog.conf

<Extension json>
    Module        xm_json
</Extension>

<Extension syslog>
    Module        xm_syslog
</Extension>

<Input nginx_access>
    Module        im_udp
    ListenAddr    127.0.0.1:514
    <Exec>
        parse_syslog();
        if $Message =~ /(?x)^(\S+)\ \S+\ (\S+)\ \[([^\]]+)\]\ \"(\S+)\ (.+)
                          \ HTTP\/\d\.\d\"\ (\S+)\ (\S+)\ \"([^\"]+)\"
                          \ \"([^\"]+)\"/
        {
            $Hostname = $1;
            if $2 != '-' $AccountName = $2;
            $EventTime = parsedate($3);
            $HTTPMethod = $4;
            $HTTPURL = $5;
            $HTTPResponseStatus = $6;
            if $7 != '-' $FileSize = $7;
            if $8 != '-' $HTTPReferer = $8;
            if $9 != '-' $HTTPUserAgent = $9;
            delete($Message);
            to_json();
        }
    </Exec>
</Input>

Output sample

{
  "MessageSourceAddress": "127.0.0.1",
  "EventReceivedTime": "2022-10-25T18:54:22.265432+03:00",
  "SourceModuleName": "nginx_access",
  "SourceModuleType": "im_udp",
  "SyslogFacilityValue": 23,
  "SyslogFacility": "LOCAL7",
  "SyslogSeverityValue": 6,
  "SyslogSeverity": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "Hostname": "127.0.0.1",
  "EventTime": "2022-10-25T18:54:22.000000+03:00",
  "SourceName": "nginx",
  "HTTPMethod": "GET",
  "HTTPURL": "/",
  "HTTPResponseStatus": "304",
  "FileSize": "0",
  "HTTPUserAgent": "Mozilla/5.0 (X11; Ubuntu; Linux aarch64; rv:105.0) Gecko/20100101 Firefox/105.0"
}

Example 5. Collecting Nginx access logs via Unix Domain Socket with NXLog Agent

With this directive, Nginx will forward all logs of warn severity or higher to the specified Unix domain socket. The messages will be sent in syslog format with the local7 syslog facility.

nginx.conf

access_log syslog:server=unix:/var/log/nginx/access.sock,facility=local7,severity=info;

nxlog.conf

<Extension syslog>
    Module    xm_syslog
</Extension>

<Input nginx_access>
    Module    im_uds
    UDS       /var/log/nginx/access.sock
    <Exec>
        parse_syslog();
        if $Message =~ /(?x)^(\S+)\ \S+\ (\S+)\ \[([^\]]+)\]\ \"(\S+)\ (.+)
                          \ HTTP\/\d\.\d\"\ (\S+)\ (\S+)\ \"([^\"]+)\"
                          \ \"([^\"]+)\"/
        {
            $Hostname = $1;
            if $2 != '-' $AccountName = $2;
            $EventTime = parsedate($3);
            $HTTPMethod = $4;
            $HTTPURL = $5;
            $HTTPResponseStatus = $6;
            if $7 != '-' $FileSize = $7;
            if $8 != '-' $HTTPReferer = $8;
            if $9 != '-' $HTTPUserAgent = $9;
            delete($Message);
        }
    </Exec>
</Input>

Disclaimer

While we endeavor to keep the information in our guides up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

NXLog Agent version 5.6.7727
Ubuntu 20.04.4 LTS
Nginx version 1.18.0 (Ubuntu)

Last revision: 25 October 2022