Event correlation

NXLog Enterprise Edition includes a high-performance event correlation module called pm_evcorr, which allows easy creation of basic and complex event correlation rules.

While NXLog is not a SIEM, you can implement event correlation and trigger actions at the log forwarder level with NXLog language features and the pm_evcorr module, allowing you to collect, enrich, and forward events efficiently.

Forwarder-level event correlation allows you to send only needed data to your SIEM, reducing network traffic and data costs. Additionally, you can enrich correlated events at the source with vital information to ensure events contain all necessary data that is otherwise spread across multiple events or numerous event sources.

In the examples below, we explore a few real-world applications and showcase the ease of using the pm_evcorr module to make these correlations.

Detecting unusual Windows authentication failures

While standard login and authentication failures occur regularly for legitimate reasons, rapid login failures may signal a malicious actor trying to gain unauthorized access to a system through brute force or dictionary attacks.

Using Threshold correlation allows you to set a baseline for the number of acceptable login failures and trigger additional actions if surpassed. In the following example, all failed login events over the threshold are enriched with a warning message. A SIEM can be configured to act on this message, aiding administrators in identifying malicious login failures for immediate investigation.

Example 1. Triggering actions using Threshold

This configuration uses the im_msvistalog input module to capture event ID 4625 – An account failed to log on. Next, it passes the events to the pm_evcorr module for processing. The event is enriched with a custom message if there are more than five failed login events within ten seconds. Finally, events are forwarded to a SIEM in JSON format using the om_tcp output module.

nxlog.conf

<Extension json>
    Module       xm_json
</Extension>

<Input eventlog>
    Module       im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0" Path="Security">
                <Select Path="Security">*[System[(EventID=4625)]]</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Processor corr_processor>
    Module       pm_evcorr
    TimeField    EventTime (1)
    <Thresholded>
        Condition    $EventID in 4625 (2)
        Threshold    5 (3)
        Interval     10 (4)
        <Exec>
            $SecurityWarning = "WARNING: Failed login threshold exceeded!"; (5)
        </Exec>
    </Thresholded>
</Processor>

<Output siem>
    Module       om_tcp
    Host         192.168.1.123:3500
    Exec         to_json(); (6)
</Output>

<Route login_events_route>
    Path         eventlog => corr_processor => siem
</Route>

1	The TimeField directive specifies the field to compare one event’s time to another.
2	The Condition directive tracks events with ID 4625.
3	The Threshold directive specifies that an event must occur 5 times to trigger the action defined in the Exec block.
4	The Interval directive specifies a 10-second time window in which the threshold value must be met to trigger the action.
5	If the rule is triggered, it enriches the event with a new field named `$SecurityWarning`.
6	Converts the record to JSON format using the to_json() procedure of the xm_json module.

You can configure the Exec block to do much more, such as enriching the event with custom data, alerting administrators, correlating with other events, or even calling external processes.

The Pair correlation type allows you to combine information from two distinct but related events.

For instance, Windows generates two separate events when a file is deleted from a monitored directory. Essential details about the deletion are found in event ID 4660 – An object was deleted. However, the name of the deleted file is only found in a prior event with ID 4663 – An attempt was made to access an object.

In the following example, event ID 4663 triggers the correlation rule. When the rule is triggered, pm_evcorr checks for a subsequent event with ID 4660, and if found, it extracts the filename from the previous event. The filename is used to enrich the deletion event for easier analysis by users and SIEMs alike.

Example 2. Consolidating important information from multiple events with Pair

This configuration uses the im_msvistalog input module to capture object access and deletion events on Windows. Next, it passes the events to the pm_evcorr module for processing. If event ID 4663 is followed by event ID 4660, the latter is enriched with the name of the deleted file. Finally, events are forwarded to a SIEM in JSON format using the om_tcp output module.

nxlog.conf

<Extension json>
    <Module   xm_json
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0" Path="Security">
                <Select Path="Security">*[System[(EventID=4660 or EventID=4663)]]</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Processor corr_processor>
    Module    pm_evcorr
    <Pair>
        TriggerCondition    $EventID == 4663 (1)
        RequiredCondition   $EventID == 4660 AND \
                            $EventRecordID == get_prev_event_data("EventRecordID") + 1 (2)
        Interval            5
        Exec                $FileName = get_prev_event_data("ObjectName"); (3)
    </Pair>
</Processor>

<Output siem>
    Module    om_tcp
    Host      192.168.1.123:3500
    Exec      to_json(); (4)
</Output>

<Route file_deletion_events_route>
    Path      eventlog => corr_processor => siem
</Route>

1	The TriggerCondition directive specifies that events with ID 4663 activate the correlation rule.
2	The RequiredCondition directive specifies that the action defined by the Exec statement should be executed when event ID 4663 is immediately followed by event ID 4660. You can retrieve previous event data with the get_prev_event_data() function of the pm_evcorr module.
3	If the rule is triggered, the `$ObjectName` field is retrieved from the previous event and added to the current event as `$FileName`.
4	Converts the record to JSON format using the to_json() procedure of the xm_json module.

You can use multiple Pair directives to correlate and combine any number of events from various sources.

Detecting unusual ICMP traffic levels

NXLog Enterprise Edition exclusive feature

While some ICMP traffic is expected and is the type of traffic usually associated with ping requests, high ICMP traffic levels in a short period can signal a ping sweep.

Using Threshold correlation allows you to set a baseline for the amount of expected ICMP traffic on your network and take action if that threshold is exceeded. The following example combines the Threshold and Simple correlation methods to enrich and forward network packets over the threshold and drop packets within the limit. By dropping packets within the limit, you exclude noise events from being forwarded to your SIEM so you can focus on events that signal a potential threat.

Example 3. Combining multiple correlation types

This configuration uses the im_pcap input module to capture all ICMP traffic on the specified network interface. Next, it passes the packets to the pm_evcorr module for processing. If a packet exceeds the defined threshold, it is enriched with a custom message; otherwise, the packet is dropped. Finally, events are forwarded to a SIEM in JSON format using the om_tcp output module.

nxlog.conf

define NIC_1 \Device\NPF_{C8A1E6B7-DEBD-4D1F-9922-A5BE6CB274BE}

<Extension json>
    Module       xm_json
</Extension>

<Input icmp_capture>
    Module       im_pcap
    Dev          %NIC_1%
    Filter       "icmp"
    <Protocol>
        Type    icmp
    </Protocol>
    <Protocol>
        Type    ip
    </Protocol>
    <Protocol>
        Type    ethernet
    </Protocol>
    <Protocol>
        Type    arp
    </Protocol>
</Input>

<Processor corr_processor>
    Module       pm_evcorr
    TimeField    EventTime (1)
    <Thresholded>
        Condition    $icmp.type IN "Echo (ping) request" (2)
        Threshold    20 (3)
        Interval     2 (4)
        <Exec>
            $SecurityWarning = "WARNING: High ICMP ping traffic detected!"; (5)
        </Exec>
    </Thresholded>
    <Simple>
        Exec         if not defined($SecurityWarning) drop(); (6)
    </Simple>
</Processor>

<Output siem>
    Module       om_tcp
    Host         192.168.1.123:3500
    Exec         to_json(); (7)
</Output>

<Route icmp_events_route>
    Path         icmp_capture => corr_processor => siem
</Route>

1	The TimeField directive specifies the field to compare one event’s time to another.
2	The Condition directive specifies to track ICMP packets.
3	The Threshold directive specifies that the same packet must occur 20 times to trigger the action defined in the Exec block.
4	The Interval directive specifies a 20-second window in which the threshold value must be met to trigger the action.
5	If the rule is triggered, it enriches the packet with a new field named `$SecurityWarning`.
6	Packets that do not trigger the rule are discarded with the drop() procedure.
7	Converts the record to JSON format using the to_json() procedure of the xm_json module.

You can configure the Exec block to do much more, such as enriching the event with custom data, alerting administrators, correlating with other events, or even calling external processes.

Event correlation

Detecting unusual Windows authentication failures

Combining data from related events

Detecting unusual ICMP traffic levels