Postfix

NXLog can be configured to collect logs from the Postfix mail server. Postfix logs its actions to the standard system logger with the mail facility type.

Syslog/Postfix log format

Oct 10 01:23:45 hostname postfix/component[pid]: message

The component indicates the Postfix process that produced the log message. Most log entries, those relevant to particular email messages, also include the queue ID of the email message as the first part of the message.

Log sample

Oct 10 01:23:45 mailhost postfix/smtpd[2534]: 4F9D195432C: client=localhost[127.0.0.1]
Oct 10 01:23:45 mailhost postfix/cleanup[2536]: 4F9D195432C: message-id=<20161001103311.4F9D195432C@mail.example.com>
Oct 10 01:23:46 mailhost postfix/qmgr[2531]: 4F9D195432C: from=<origin@other.com>, size=344, nrcpt=1 (queue active)
Oct 10 01:23:46 mailhost postfix/smtp[2538]: 4F9D195432C: to=<destination@example.com>, relay=mail.example.com[216.150.150.131], delay=11, status=sent (250 Ok: queued as 8BDCA22DA71)

Configuring Postfix logging

Several configuration directives, set in main.cf, can be used to adjust Postfix’s logging behavior.

lmtp_tls_loglevel
smtp_tls_loglevel
smtpd_tls_loglevel: The loglevel directives should be set to 0 (disabled, the default) or 1 during normal operation. Values of 2 or 3 can be used for troubleshooting.
debug_peer_level: Specify the increment in logging level when a remote client or server matches a pattern in the debug_peer_list parameter (default 2).
debug_peer_list: Provide a list of remote client or server hostnames or network address patterns for which to increase the logging level.

See the Postfix Debugging Howto and the postconf(5) man page for more information.

Collecting and processing Postfix logs

The local syslogd configuration determines where and how the mail facility logs are written, but normally the logs can be found in /var/log/maillog or /var/log/mail.log. See Collecting and Parsing Syslog and Linux system logs for more information about collecting Syslog logs.

Example 1. Reading from Syslog log file

This configuration reads the Postfix logs from file and forwards them via TCP to a remote host.

nxlog.conf

<Input postfix>
    Module  im_file
    File    "/var/log/mail.log"
</Input>

<Output out>
    Module  om_tcp
    Host    192.168.1.1
    Port    1514
</Output>

It is also possible to parse individual Postfix messages into fields, providing access to more fine-grained filtering and analysis of log data. The NXLog Exec directive can be used to apply regular expressions for this purpose.

Example 2. Extracting additional fields and filtering

Here is the Input module instance again, extended to parse the Postfix messages in the example above. Various fields are added to the event record, depending on the particular message received. Then in the Output module instance, only those log entries that are from Postfix’s smtp component and are being relayed through mail.example.com are logged to the output file.

nxlog.conf

<Extension _json>
    Module  xm_json
</Extension>

<Input postfix>
    Module  im_file
    File    "/var/log/mail.log"
    <Exec>
        if $raw_event =~ /(?x)^(\S+\ +\d+\ \d+:\d+:\d+)\ (\S+)
                          \ postfix\/(\S+)\[(\d+)\]:\ (.+)$/
        {
            $EventTime = parsedate($1);
            $HostName = $2;
            $SourceName = "postfix";
            $Component = $3;
            $ProcessID = $4;
            $Message = $5;
            if $Component == "smtpd" and
               $Message =~ /(\w+): client=(\S+)\[([\d.]+)\]/
            {
                $QueueID = $1;
                $ClientHostname = $2;
                $ClientIP = $3;
            }
            if $Component == "cleanup" and
               $Message =~ /(\w+): message-id=(<\S+@\S+>)/
            {
                $QueueID = $1;
                $MessageID = $2;
            }
            if $Component == "qmgr" and
               $Message =~/(\w+): from=(<\S+@\S+>), size=(\d+), nrcpt=(\w+)/
            {
                $QueueID = $1;
                $Sender = $2;
                $Size = $3;
                $Nrcpt = $4;
            }
            if $Component == "smtp" and
               $Message =~ /(?x)(\w+):\ to=(<\S+@\S+>),\ relay=([\w.]+)\[([\d.]+)\],
                            \ delay=(\d+),\ status=(\w+)\ \((\d+)\ \w+:\ queued\ as
                            \ (\w+)\)/
            {
                $QueueID = $1;
                $Recipient = $2;
                $RelayHostname = $3;
                $RelayIP = $4;
                $Delay = $5;
                $Status = $6;
                $SMTPCode = $7;
                $QueueIDDelivered = $8;
            }
        }
    </Exec>
</Input>

<Output out>
    Module  om_file
    File    "/var/log/smtp.log"
    <Exec>
        if $Component != "smtp" drop();
        if $RelayHostname != "mail.example.com" drop();
        to_json();
    </Exec>
</Output>

Using the example log entries above, this configuration results in a single JSON entry written to the log file.

Output sample

{
  "EventReceivedTime": "2016-10-05 16:38:57",
  "SourceModuleName": "postfix",
  "SourceModuleType": "im_file",
  "EventTime": "2016-10-10 01:23:46",
  "HostName": "mail",
  "SourceName": "postfix",
  "Component": "smtp",
  "ProcessID": "2538",
  "Message": "4F9D195432C: to=<destination@example.com>, relay=mail.example.com[216.150.150.131], delay=11, status=sent (250 Ok: queued as 8BDCA22DA71)",
  "QueueID": "4F9D195432C",
  "Recipient": "<destination@example.com>",
  "RelayHostname": "mail.example.com",
  "RelayIP": "216.150.150.131",
  "Delay": "11",
  "Status": "sent",
  "SMTPCode": "250",
  "QueueIDDelivered": "8BDCA22DA71"
}

Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

Last revision: 17 September 2018