Packet capture (im_pcap)
This module provides support to passively monitor network traffic by generating logs for various protocols. It uses the libpcap and WinPcap libraries to capture network traffic.
| When running on Windows, im_pcap uses the WinPcap library to capture network traffic. Due to licensing restrictions, the NXLog installer does not include WinPcap. The WinPcap library has to be obtained and installed separately before using im_pcap. You can get a copy from the project’s website. If WinPcap is not installed, attempting to use the im_pcap module will fail with an error. |
Configuration
The im_pcap module accepts the following directives in addition to the common module directives.
Optional directives
This optional directive can only occur once.
It specifies the name of a network device/interface on which im_pcap will capture packets.
This directive is mutually exclusive with the File directive.
On Windows systems, network device names must follow the Netgroup Packet Filter (NPF) name format recognized by the WinPcap library:
|
|||||||
This optional directive can only occur once. It specifies the path to the file which contains captured packet data. The file path does not need to be enclosed in quotation marks, although both single-quoted and double-quoted paths are accepted. This directive is mutually exclusive with the Dev directive. |
|||||||
This optional directive specifies the maximum number of captured packets that this module can queue for parallel processing. When the queue becomes full, new packets will be dropped. The default value for this directive is inherited from the global-level LogqueueSize directive. |
|||||||
This is an optional group directive. It specifies the protocol, port number and protocol-specific fields which should be captured. May be used multiple times in the module definition, to specify multiple protocols. If no Protocol directive is specified, then all protocols will be captured. It has the following sub-directives:
|
JSON fields
See the list of JSON fields.
Protocols and fields
See the list of Protocols and fields.
Examples
In this example, the File directive defines the path and filename of a .pcap file containing packets saved by Wireshark.
The Filter directive defines a filter that selects only TCP packets targeted for port 443. The output is formatted as JSON while written to file.
<Extension _json>
Module xm_json
</Extension>
<Input pcap>
Module im_pcap
File "/tmp/example.pcap"
Filter tcp dst port 443
</Input>
<Output file>
Module om_file
File "/tmp/output.json"
Exec to_json();
</Output>In this example, the configuration illustrates how the Protocol group directive can be defined multiple times within the same module instance. Three types of network packets are to be captured: HTTP requests; TCP for the source and destination ports of all visible TCP traffic; and Ethernet to log the MAC addresses of packet sources and their destinations. The events are formatted to JSON while writing to a file.
This approach has two distinct advantages. It produces events that include all fields of all three protocols, which enables correlation between protocols that yield source and destination information with those protocols that do not provide such fields. Additionally, it achieves this goal using a single module instance instead of multiple instances, which reduces system resource consumption.
<Extension _json>
Module xm_json
</Extension>
<Input pcap>
Module im_pcap
<Protocol>
Type http
Field http.request.uri
Field http.request.method
Field http.response.code
Field http.response.phrase
</Protocol>
<Protocol>
Type tcp
Field tcp.src_port
Field tcp.dst_port
Field tcp.flag
</Protocol>
<Protocol>
Type ethernet
Field eth.src_mac
Field eth.dest.mac
</Protocol>
</Input>
<Output file>
Module om_file
File "tmp/output"
Exec to_json();
</Output>In this example, each of the three protocols is managed by a separate module instance. The events are formatted to JSON while being written to each of their respective files. This approach can be used when there is a need to analyze each protocol in isolation from each other. Because three input instances are used, more system resources will be consumed when compared to the multi-protocol, single-instance approach.
<Extension _json>
Module xm_json
</Extension>
<Input pcap_tcp>
Module im_pcap
<Protocol>
Type tcp
Field tcp.src_port
Field tcp.dst_port
Field tcp.flag
</Protocol>
</Input>
<Input pcap_http>
Module im_pcap
<Protocol>
Type http
Field http.request.uri
Field http.request.method
Field http.response.code
Field http.response.phrase
</Protocol>
</Input>
<Input pcap_eth>
Module im_pcap
<Protocol>
Type Ethernet
Field eth.src_mac
Field eth.dest.mac
</Protocol>
</Input>
<Output tcp_file>
Module om_file
File "tmp/tcp_output"
Exec to_json();
</Output>
<Output http_file>
Module om_file
File "tmp/http_output"
Exec to_json();
</Output>
<Output eth_file>
Module om_file
File "tmp/eth_output"
Exec to_json();
</Output>In this example, the configuration illustrates how to use an Exec block to filter events based on protocol fields.
The Protocol directive of the im_pcap module instance captures iec61850iso traffic.
The output module instance checks if the iso8650.aare.aso_context_name field exists.
If it does, and its value equals 28ca220203, the event is dropped.
Events that are not dropped are formatted to JSON and written to file.
<Extension _json>
Module xm_json
</Extension>
<Input pcap>
Module im_pcap
File "/tmp/example.pcap"
<Protocol>
Type iec61850iso
</Protocol>
</Input>
<Output file>
Module om_file
File "/tmp/output.json"
<Exec>
if defined(${iso8650.aare.aso_context_name})
and ${iso8650.aare.aso_context_name} == "28ca220203"
{
drop();
}
to_json();
</Exec>
</Output>This configuration collects Modbus packets from a network interface. Captured messages are then converted to JSON.
<Extension json>
Module xm_json
</Extension>
<Input modbus>
Module im_pcap
Dev \Device\NPF_{30CA6F95-396D-45FE-B392-BFA2B3AF4AAD} (1)
<Protocol>
Type modbus (2)
</Protocol>
Exec to_json(); (3)
</Input>| 1 | Specifies the network inteface/device |
| 2 | Specifies the protocol type |
| 3 | Formats the output to JSON |
This configuration collects BACnet packets from a network interface. Captured messages are then converted to JSON.
<Extension json>
Module xm_json
</Extension>
<Input bacnet>
Module im_pcap
Dev \Device\NPF_{403FC877-F019-4013-9387-A78FC58403CB} (1)
Filter port 47808 (2)
<Protocol>
Type bacnet (3)
</Protocol>
Exec to_json(); (4)
</Input>| 1 | Specifies the network inteface/device |
| 2 | Specifies the port filter |
| 3 | Specifies the protocol type |
| 4 | Formats the output to JSON |
This configuration collects DNS packets from multiple PCAP files using nxlog-processor(8). It utilizes an external bash script that lists the files and makes them available to nxlog-processor. Once the nxlog-processor command runs, it reads the oldest file and deletes it. When reading multiple files, nxlog-processor has to be triggered for each PCAP file to be processed. This can be achieved by a cron job on Linux.
This example uses the /tmp directory for the files involved.
Change your directory according to your environment.
|
#!/bin/sh -e
echo FILE `ls -tr /tmp/*.pcap | head -1`<Input pcap>
Module im_pcap
include_stdout /tmp/script.sh (1)
<Protocol> (2)
Type dns
Field dns.query
</Protocol>
</Input>| 1 | Specifies the script to run |
| 2 | Specifies the protocol |
Below is the command with the parameters to run nxlog-processor
# ./nxlog-processor -c /opt/nxlog/etc/nxlog.conf; rm -f `ls -tr /tmp/*.pcap | head -1`This configuration collects DNS packets from multiple PCAP files using nxlog-processor(8). It utilizes an external cmdlet with PowerShell code that lists the files and makes them available to nxlog-processor. Once the nxlog-processor command runs, it reads the oldest file and deletes it. When reading multiple files, nxlog-processor has to be triggered for each PCAP file to be processed. This can be achieved by a scheduled task in Windows.
This example uses the C:\testpcap\ directory for the files involved.
Change your directory according to your environment.
|
@( Set "_= (
REM " ) <#
)
@Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
set powershell=powershell.exe
REM Use this if you need 64-bit PowerShell (has no effect on 32-bit systems).
REM if defined PROCESSOR_ARCHITEW6432 (
REM set powershell=%SystemRoot%\SysNative\WindowsPowerShell\v1.0\powershell.exe
REM )
REM Use this if you need 32-bit PowerShell.
REM if NOT %PROCESSOR_ARCHITECTURE% == x86 (
REM set powershell=%SystemRoot%\SysWOW64\WindowsPowerShell\v1.0\powershell.exe
REM )
%powershell% -ExecutionPolicy Bypass -NoProfile ^
-Command "iex ((gc '%~f0') -join [char]10)"
EndLocal & Exit /B %ErrorLevel%
#>
# PowerShell code starts here.
# To get the pcap file with the earliest date
Get-ChildItem -Path "C:\testpcap" -Filter "*.pcap" | Sort-Object -Property LastWriteTime | Select-Object -First 1 | ForEach-Object {
Write-Output "FILE '$($_.FullName)'"
}<Input pcap>
Module im_pcap
include_stdout C:\testpcap\powershell.cmd (1)
<Protocol> (2)
Type dns
Field dns.query
</Protocol>
</Input>| 1 | Specifies the script to run |
| 2 | Specifies the protocol |
Below is the command with the parameters to run nxlog-processor
nxlog-processor.exe -c "C:\Program Files\nxlog\conf\nxlog.conf" & cmd /C "FOR /F "delims=" %I IN ('DIR /B /O:D "C:\testpcap\*.pcap"') DO (DEL /Q /F "C:\testpcap\%I" & EXIT /B)"