Packet capture (im_pcap)

This module provides support to passively monitor network traffic by generating logs for various protocols. It uses the libpcap and WinPcap libraries to capture network traffic.

When running on Windows, im_pcap uses the WinPcap library to capture network traffic. Due to licensing restrictions, the NXLog installer does not include WinPcap. The WinPcap library has to be obtained and installed separately before using im_pcap. You can get a copy from the project’s website. If WinPcap is not installed, attempting to use the im_pcap module will fail with an error.

Configuration

The im_pcap module accepts the following directives in addition to the common module directives.

Optional directives

Dev

This optional directive can only occur once. It specifies the name of a network device/interface on which im_pcap will capture packets. This directive is mutually exclusive with the File directive. On Windows systems, network device names must follow the Netgroup Packet Filter (NPF) name format recognized by the WinPcap library: \Device\NPF_{<GUID>}. The GUID portion of the name can be found using the command getmac -v.

im_pcap will listen on the first available interface if the Dev directive is omitted.

File

This optional directive can only occur once. It specifies the path to the file which contains captured packet data. The file path does not need to be enclosed in quotation marks, although both single-quoted and double-quoted paths are accepted. This directive is mutually exclusive with the Dev directive.

LogqueueSize

This optional directive specifies the maximum number of captured packets that this module can queue for parallel processing. When the queue becomes full, new packets will be dropped. The default value for this directive is inherited from the global-level LogqueueSize directive.

Protocol

This is an optional group directive. It specifies the protocol, port number and protocol-specific fields which should be captured. May be used multiple times in the module definition, to specify multiple protocols. If no Protocol directive is specified, then all protocols will be captured. It has the following sub-directives:

Type: Defines the name of a protocol to capture. Allowed types are; ethernet, ipv4, ipv6, ip, tcp, udp, http (httprequest, httpresponse), arp, vlan, icmp, pppoe (pppoesession, pppoediscovery), dns, mpls, gre (grev0, grev1), ppp_pptp, ssl, sll, dhcp, null_loopback, igmp (igmpv1, igmpv2, igmpv3), vxlan, sip (siprequest, sipresponse), sdp, radius, modbus, profinet (pn_dcp, pn_mrp, pn_io), dnp3, bacnet, iec104 (iec104apci, iec104asdu), iec61850 (iec61850iso, iec61850goose, iec61850gsem, iec61850sv, iec61850mms), s7 (s7comm), cotp, tpkt, rpc_cl, packettrailer, genericpayload, all.

IEC 61850 is a complex standard that includes a collection of protocols. Not all protocols from this standard can be filtered, e.g., iso8327, iso8823 and iso8650. This is because these protocols are dependent on the dissection of other layers. To support filtering, these protocols are wrapped with iec61850iso. Additionally, see the example below for how you can apply field-based filtering of IEC61850 traffic.

Port

A list of custom port numbers to capture for the protocol specified in the Type directive. If omitted, standard port number(s) corresponding to the protocol will be used. Refer to the JSON fields and Fields sections for details of the default ports for each protocol.

The list of port numbers must be separated by a comma. Spaces and tabs between the values are accepted.

Field

Optional directive to specify the fields to be included in the output. If this directive is not specified in the protocol configuration, then all available fields will be present in the output. If one or more Field directives are present in the configuration for a protocol, then only those fields will be present in the output. Refer to the JSON fields and Protocols and fields sections for details of the fields supported for each protocol.

Filter

Optional directive that defines a filter, which can be used to further limit the packets that should be captured and handled by the module. Filters do not need to be enclosed in quotation marks, although both single-quoted and double-quoted filters are accepted. If this directive is not used, then no filtering will be done.

Filtering is done by the libpcap library. See the Manpage of PCAP-FILTER in the libpcap documentation for the syntax. Not all protocols are supported for libpcap filtering.

JSON fields

See the list of JSON fields.

Protocols and fields

See the list of Protocols and fields.

Examples

Example 1. Reading from a PCAP file while applying a packet filter

In this example, the File directive defines the path and filename of a .pcap file containing packets saved by Wireshark. The Filter directive defines a filter that selects only TCP packets targeted for port 443. The output is formatted as JSON while written to file.

nxlog.conf

<Extension _json>
    Module  xm_json
</Extension>

<Input pcap>
    Module    im_pcap
    File      "/tmp/example.pcap"
    Filter    tcp dst port 443
</Input>

<Output file>
    Module  om_file
    File    "/tmp/output.json"
    Exec    to_json();
</Output>

Example 2. Capturing TCP, Ethernet, and HTTP traffic to a single file

In this example, the configuration illustrates how the Protocol group directive can be defined multiple times within the same module instance. Three types of network packets are to be captured: HTTP requests; TCP for the source and destination ports of all visible TCP traffic; and Ethernet to log the MAC addresses of packet sources and their destinations. The events are formatted to JSON while writing to a file.

This approach has two distinct advantages. It produces events that include all fields of all three protocols, which enables correlation between protocols that yield source and destination information with those protocols that do not provide such fields. Additionally, it achieves this goal using a single module instance instead of multiple instances, which reduces system resource consumption.

nxlog.conf

<Extension _json>
    Module  xm_json
</Extension>

<Input pcap>
    Module  im_pcap
    <Protocol>
        Type    http
        Field   http.request.uri
        Field   http.request.method
        Field   http.response.code
        Field   http.response.phrase
    </Protocol>
    <Protocol>
        Type    tcp
        Field   tcp.src_port
        Field   tcp.dst_port
        Field   tcp.flag
    </Protocol>
    <Protocol>
        Type    ethernet
        Field   eth.src_mac
        Field   eth.dest.mac
    </Protocol>
</Input>

<Output file>
    Module  om_file
    File    "tmp/output"
    Exec    to_json();
</Output>

Example 3. Capturing TCP, Ethernet, and HTTP traffic to separate files

In this example, each of the three protocols is managed by a separate module instance. The events are formatted to JSON while being written to each of their respective files. This approach can be used when there is a need to analyze each protocol in isolation from each other. Because three input instances are used, more system resources will be consumed when compared to the multi-protocol, single-instance approach.

nxlog.conf

<Extension _json>
    Module  xm_json
</Extension>

<Input pcap_tcp>
    Module  im_pcap
    <Protocol>
        Type    tcp
        Field   tcp.src_port
        Field   tcp.dst_port
        Field   tcp.flag
    </Protocol>
</Input>

<Input pcap_http>
    Module  im_pcap
    <Protocol>
        Type    http
        Field   http.request.uri
        Field   http.request.method
        Field   http.response.code
        Field   http.response.phrase
    </Protocol>
</Input>

<Input pcap_eth>
    Module  im_pcap
    <Protocol>
        Type    Ethernet
        Field   eth.src_mac
        Field   eth.dest.mac
    </Protocol>
</Input>

<Output tcp_file>
    Module  om_file
    File    "tmp/tcp_output"
    Exec    to_json();
</Output>

<Output http_file>
    Module  om_file
    File    "tmp/http_output"
    Exec    to_json();
</Output>

<Output eth_file>
    Module  om_file
    File    "tmp/eth_output"
    Exec    to_json();
</Output>

Example 4. Field-based filtering of IEC61850 traffic

In this example, the configuration illustrates how to use an Exec block to filter events based on protocol fields. The Protocol directive of the im_pcap module instance captures iec61850iso traffic. The output module instance checks if the iso8650.aare.aso_context_name field exists. If it does, and its value equals 28ca220203, the event is dropped. Events that are not dropped are formatted to JSON and written to file.

nxlog.conf

<Extension _json>
    Module    xm_json
</Extension>

<Input pcap>
    Module    im_pcap
    File      "/tmp/example.pcap"
    <Protocol>
        Type    iec61850iso
    </Protocol>
</Input>

<Output file>
    Module    om_file
    File      "/tmp/output.json"
    <Exec>
        if defined(${iso8650.aare.aso_context_name})
            and ${iso8650.aare.aso_context_name} == "28ca220203"
        {
            drop();
        }
        to_json();
    </Exec>
</Output>

Example 5. Collecting Modbus logs from a specific network interface

This configuration collects Modbus packets from a network interface. Captured messages are then converted to JSON.

nxlog.conf

<Extension json>
    Module          xm_json
</Extension>

<Input modbus>
    Module          im_pcap
    Dev             \Device\NPF_{30CA6F95-396D-45FE-B392-BFA2B3AF4AAD} (1)
    <Protocol>
        Type        modbus (2)
    </Protocol>
    Exec            to_json(); (3)
</Input>

1	Specifies the network inteface/device
2	Specifies the protocol type
3	Formats the output to JSON

Example 6. Collecting BACnet logs from a specific network interface

This configuration collects BACnet packets from a network interface. Captured messages are then converted to JSON.

nxlog.conf

<Extension json>
    Module          xm_json
</Extension>

<Input bacnet>
    Module          im_pcap
    Dev             \Device\NPF_{403FC877-F019-4013-9387-A78FC58403CB} (1)
    Filter          port 47808 (2)
    <Protocol>
        Type        bacnet (3)
    </Protocol>
    Exec            to_json(); (4)
</Input>

1	Specifies the network inteface/device
2	Specifies the port filter
3	Specifies the protocol type
4	Formats the output to JSON

Example 7. Collecting DNS logs from multiple PCAP files on Linux

This configuration collects DNS packets from multiple PCAP files using nxlog-processor(8). It utilizes an external bash script that lists the files and makes them available to nxlog-processor. Once the nxlog-processor command runs, it reads the oldest file and deletes it. When reading multiple files, nxlog-processor has to be triggered for each PCAP file to be processed. This can be achieved by a cron job on Linux.

This example uses the /tmp directory for the files involved. Change your directory according to your environment.

bash script to include as a stdout

#!/bin/sh -e
echo FILE `ls -tr /tmp/*.pcap | head -1`

nxlog.conf

<Input pcap>
    Module  im_pcap
    include_stdout /tmp/script.sh (1)
    <Protocol> (2)
        Type    dns 
        Field   dns.query
    </Protocol>
</Input>

1	Specifies the script to run
2	Specifies the protocol

Below is the command with the parameters to run nxlog-processor

nxlog-processor command

# ./nxlog-processor -c /opt/nxlog/etc/nxlog.conf; rm -f `ls -tr /tmp/*.pcap | head -1`

Example 8. Collecting DNS logs from multiple PCAP files on Windows

This configuration collects DNS packets from multiple PCAP files using nxlog-processor(8). It utilizes an external cmdlet with PowerShell code that lists the files and makes them available to nxlog-processor. Once the nxlog-processor command runs, it reads the oldest file and deletes it. When reading multiple files, nxlog-processor has to be triggered for each PCAP file to be processed. This can be achieved by a scheduled task in Windows.

This example uses the C:\testpcap\ directory for the files involved. Change your directory according to your environment.

powershell (cmd) script to include as a stdout

@( Set "_= (
REM " ) <#
)
@Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
set powershell=powershell.exe

REM Use this if you need 64-bit PowerShell (has no effect on 32-bit systems).
REM if defined PROCESSOR_ARCHITEW6432 (
REM set powershell=%SystemRoot%\SysNative\WindowsPowerShell\v1.0\powershell.exe
REM )

REM Use this if you need 32-bit PowerShell.
REM if NOT %PROCESSOR_ARCHITECTURE% == x86 (
REM set powershell=%SystemRoot%\SysWOW64\WindowsPowerShell\v1.0\powershell.exe
REM )

%powershell% -ExecutionPolicy Bypass -NoProfile ^
-Command "iex ((gc '%~f0') -join [char]10)"
EndLocal & Exit /B %ErrorLevel%
#>

# PowerShell code starts here.

# To get the pcap file with the earliest date
Get-ChildItem -Path "C:\testpcap" -Filter "*.pcap" | Sort-Object -Property LastWriteTime | Select-Object -First 1 | ForEach-Object {
    Write-Output "FILE '$($_.FullName)'"
}

nxlog.conf

<Input pcap>
    Module  im_pcap
    include_stdout C:\testpcap\powershell.cmd (1)
    <Protocol> (2)
        Type    dns 
        Field   dns.query
    </Protocol>
</Input>

1	Specifies the script to run
2	Specifies the protocol

Below is the command with the parameters to run nxlog-processor

nxlog-processor command

 nxlog-processor.exe -c "C:\Program Files\nxlog\conf\nxlog.conf" & cmd /C "FOR /F "delims=" %I IN ('DIR /B /O:D "C:\testpcap\*.pcap"') DO (DEL /Q /F "C:\testpcap\%I" & EXIT /B)"