Docker

Docker is a containerization technology that enables the creation and use of Linux containers. Containers allow a developer to package an application with all of its dependencies and distribute it as a single package. The Docker container technology is widely used in modern, micro-service architectures.

By concept, Docker images should be lightweight; usually only one application is present and running in the container. Therefore, Docker container logs are written to the standard out and standard error streams and logging must be performed from outside the image.

Configuring logging in Docker

By default, Docker writes logs from each container to a separate JSON file, stored under the container’s directory on the host machine. The logging of containers can be configured in two ways: by modifying the default logging configuration of the Docker daemon, or by changing it in the runtime options for a specific container. For more details about Docker’s logging drivers, see Configure logging drivers on Docker.com.

The default logging driver can be set in the daemon.json configuration file. This file is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\ on Windows Server hosts. The default logging driver is json-file.
The default logging driver can be overridden at the container level. To accomplish this, the log driver and its configuration options must be provided as parameters at container startup with the help of the docker run command. The configuration options are the same as setting up logging options for the Docker daemon. See the docker run command reference on Docker.com for more information.

Receiving logs from Docker

Collecting logs from a Docker daemon or container is supported in four ways depending on the log driver in use.

To find the current logging driver for a running container, run the following docker inspect command, substituting the container name or ID for <CONTAINER>.

$ docker inspect -f '{{.HostConfig.LogConfig.Type}}' <CONTAINER>

Receiving Docker logs in JSON

With the json-file log driver, Docker produces a line-based log file in JSON format for each container. See the JSON File logging driver guide on Docker.com for more information.

Because im_file recursively watches for log files in the containers directory, this may cause reduced performance in very large installations.

Example 1. Collecting Docker logs in JSON format

This example configuration reads from the JSON log files of all containers. The JSON fields are parsed and added to the event record with the xm_json parse_json() procedure. A $HostID field, with the container ID, is also added.

nxlog.conf

<Extension _fileop>
    Module  xm_fileop
</Extension>

<Extension _json>
    Module  xm_json
</Extension>

<Input in>
    Module  im_file
    File    '/var/lib/docker/containers/*/*-json.log'
    <Exec>
        parse_json();
        $HostID = file_basename(file_name());
        $HostID =~ s/-json.log//;
    </Exec>
</Input>

Receiving Docker logs in GELF format

The gelf logging driver is a convenient format that is understood by a number of tools such as NXLog Agent. In GELF, every log message is a dictionary with fields such as version, host, timestamp, short and long version of the message, and any custom fields that have been configured. See the Graylog Extended Format logging driver guide on Docker.com for more information.

Example 2. Collecting Docker logs in GELF format

In this example, NXLog Agent accepts and parses logs in GELF format on TCP port 12201 with the im_tcp and xm_gelf modules.

nxlog.conf

<Extension _gelf>
    Module      xm_gelf
</Extension>

<Input in>
    Module      im_tcp
    Host        0.0.0.0
    Port        12201
    InputType   GELF_TCP
</Input>

Collecting Docker logs in the Syslog format

The syslog logging driver routes logs to a Syslog server, such as NXLog Agent, via UDP, TCP, SSL/TLS, or a Unix domain socket. See the Syslog logging driver guide on Docker.com for more information.

Example 3. Collecting Docker Syslog logs

Here, NXLog Agent accepts logs on TCP port 1514 with the im_tcp module and parses the logs with the xm_syslog parse_syslog() procedure.

nxlog.conf

<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input in>
    Module  im_tcp
    Host    0.0.0.0
    Port    1514
    Exec    parse_syslog();
</Input>

Collecting Docker logs from Event Tracing for Windows (ETW)

On Windows-based systems, the etwlogs logging driver forwards container logs to the Event Tracing for Windows (ETW) system. Each ETW event contains a message with both the log and its context information. See the ETW logging driver guide on Docker.com for more information.

Example 4. Collecting Docker logs in ETW format

This example collects logs from the DockerContainerLogs Event Tracing provider using the im_etw module.

nxlog.conf

<Input in>
    Module      im_etw
    Provider    DockerContainerLogs
</Input>

Collecting Docker logs via Systemd

Systemd is the init system which is used by many modern Linux distributions. This system includes the journal component for handling log messages. All messages generated by Systemd-controlled processes are sent to journal. Each message contains a number of key-value pairs which can be used for filtering the data.

For more information about configuring Docker to use Systemd for logging, see the Journald logging driver guide on the Docker.com website.

Example 5. Collecting Docker Systemd logs

This example collects logs from the Systemd journal component using the im_systemd module.

nxlog.conf

<Input from_systemd>
    Module   im_systemd
    <Exec>
        if not(defined($ProcessName)) drop();
        else if (defined($ProcessName) and $ProcessName != 'dockerd') drop();
    </Exec>
</Input>

Disclaimer

While we endeavor to keep the information in our guides up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

Last revision: 03 June 2019