File Integrity Monitoring (im_fim)

This module is capable of scanning files and directories and reporting detected changes and deletions. On the first scan, the checksum of each file is recorded. This checksum is then compared to the checksum value calculated during successive scans. The im_fim module works at the filesystem level, so it only has access to file information such as ownership and last modification date, and no information about which user made a change.

Files are checked periodically, not in real-time. If there are multiple changes between two scans, only the cumulative effect is logged. For example, if one user modifies a file and another user reverts the changes before the next scan occurs, only the change in modification time is detected.

For real-time monitoring, auditing must be enabled on the host operating system. See File Integrity Monitoring for more information.

To examine the supported platforms, see the list of installation packages.

Filepath definition

The module treats filenames as case-insensitive on Windows and case-sensitive on Unix/Linux systems. When you specify a relative filename, NXLog Agent sets its working directory to / unless you define a different directory with the global SpoolDir directive.

On Windows, the directory separator is the backslash (\). For compatibility, you can use the forward slash (/) as a directory separator, but only when the filename does not include wildcards. If you specify a filename with wildcards, you must use the backslash (\) as the directory separator.

Wildcards

You can use wildcards in filenames and directories. Wildcards are not regular expressions but patterns commonly used by Unix shells to expand filenames (also known as "globbing").

The supported wildcards are:

?: Matches a single character only.
*: Matches zero or more characters.
\*: Matches the asterisk (*) character.
\?: Matches the question mark (?) character.
[…]: Matches a single character using a character class. The class description consists of individual characters and character ranges separated by a hyphen (-). If the class description starts with ^ or !, the class matches any character that is not listed. You can prefix any character with a backslash (\), which the parser ignores. This allows you to include special characters such as ]. -, ^, and !.

Escape characters

By default, the escape sequence is the backslash character (\). This character is also the directory separator on Windows. Because of this, the module doesn’t support escaping wildcard characters on Windows. See the EscapeGlobPatterns directive for more information.

The module evaluates string literals differently depending on the quotation type:

Single quoted strings are interpreted as-is without escaping, for example 'C:\t???\*.log' stays C:\t???\*.log.
Escape sequences in double-quoted strings are processed, for example "C:\\t???\*.log" becomes C:\t???\*.log.

In both cases, the evaluated string is the same and is separated into parts with different glob patterns at different levels. On this example, the parts are C:, t???, and *.log. NXLog Agent matches the parts at the relevant directory levels to find all matching files.

Configuration

The im_fim module accepts the following directives in addition to the common module directives. The File directive is required.

Required directives

The following directive is required for the module to start.

File

This mandatory directive specifies the name of the input file to scan. It must be a string type expression. More than one occurrence of the File directive can be used.

See Filepath definition for more details on using this directive.

Optional directives

Digest

This specifies the digest method (hash function) to be used to calculate the checksum. The default is sha1. The following message digest methods can be used: md2, md5, mdc2, rmd160, sha, sha1, sha224, sha256, sha384, and sha512.

EscapeGlobPatterns

This boolean directive specifies whether the backslash (\) character in glob patterns or wildcarded entries should be enabled as an escape sequence. If set to TRUE, this directive implies that the backslash character (\) needs to be escaped by another backslash character (\\). File and directory patterns on Windows do not require escaping and are processed as non-escaped even if this directive is set to TRUE. The default is FALSE.

Exclude

This directive can specify a file or a set of files (using wildcards) to be excluded from the scan. More than one occurrence of the Exclude directive can be specified.

ExcludeSize

This directive can be used to specify an upper file size limit, in bytes. The checksum calculation will be skipped for files that exceed this limit, and changes will only be reported based on the file name and attributes.

NoEscape

This boolean directive specifies whether the backslash (\) in file paths should be disabled as an escape sequence. By default, NoEscape is FALSE (the path separator on Windows needs to be escaped).

Use the EscapeGlobPatterns global directive instead.

Recursive

Set this directive to TRUE to search within sub-directories of any directory specified in the File directive(s). You can use wildcards in combination with Recursive, but it will only apply the recursive action on the stem of the path. See the examples below.

The default is FALSE; the module will not search within sub-directories.

File Directive

Matches Directory

Matches Filename

Examples

/var/log/error.log

/var/log/*

error.log

/var/log/error.log
/var/log/apt/error.log

/var/log/*.log

/var/log/*

*.log

/var/log/error.log
/var/log/apt/history.log

/var/log/*/error.log

/var/log/*/*

error.log

/var/log/apt/error.log
/var/log/journal/error.log
/var/log/journal/tmp/error.log

/var/*/apt/error.log

/var/*/apt/*

error.log

/var/log/apt/error.log
/var/log/apt/tmp/error.log
/var/lib/apt/error.log
/var/lib/apt/log/error.log

FollowSymlinks

This directive specifies whether the module should treat symlinked directories and files as regular directories and files. If the directive is TRUE, the module treats symlinked directories and symlinked files as normal directories and files. If the Recursive directive is also TRUE, it will have the same effect as it has for a regular directory. The Exclude directive with the symlinks and target files works as usual except when the symlink’s name is defined in the File directive and the symlink’s target is specified in the Exclude directive. The default value of the FollowSymlinks directive is FALSE – the module does not follow the symlinked directories.

The directive works with symlinks created with the mklink command on Windows and the ln command on Unix-like operating systems. The directive supports symlinks made to directories and files.

ScanInterval

This directive specifies how long the module will wait between scans for modifications, in seconds. The default is 86400 seconds (1 day). The value of ScanInterval can be set to 0 to disable periodic scanning and instead invoke scans via the start_scan() procedure.

Functions

The following functions are exported by im_fim.

type: boolean is_scanning(): Returns TRUE if scanning is in progress. You must call this function using the -> operator. See Calling a function of a specific module instance for more information.

Procedures

The following procedures are exported by im_fim.

start_scan();: Start the file integrity scan. This could be invoked from the Schedule block, for example.

Fields

The following fields are created by im_fim.

$raw_event (type: string): A list of event fields in key-value pairs.
$Digest (type: string): The calculated digest (checksum) value.
$DigestName (type: string): The name of the digest used to calculate the checksum value (for example, SHA1).
$EventTime (type: datetime): The time when the modification was detected.
$EventType (type: string): One of the following values: CHANGE, DELETE, RENAME, or NEW.
$FileName (type: string): The name of the file that the changes were detected on.
$FileSize (type: integer): The size of the file in bytes after the modification.
$Hostname (type: string): The name of the originating computer.
$ModificationTime (type: datetime): The modification time (mtime) of the file when the change is detected.
$Object (type: string): One of the following values: DIRECTORY or FILE.
$PrevDigest (type: string): The calculated digest (checksum) value from the previous scan.
$PrevFileName (type: string): The name of the file from the previous scan.
$PrevFileSize (type: integer): The size of the file in bytes from the previous scan.
$PrevModificationTime (type: datetime): The modification time (mtime) of the file from the previous scan.
$Severity (type: string): The severity name: WARNING.
$SeverityValue (type: integer): The WARNING severity level value: 3.

Examples

Example 1. Periodic file integrity monitoring

With this configuration, NXLog Agent will monitor the specified directories recursively. Scans will occur every 3600 seconds (hourly).

nxlog.conf

<Input fim>
    Module            im_fim
    File              "/etc/*"
    Exclude           "/etc/mtab"
    File              "/bin/*"
    File              "/sbin/*"
    File              "/usr/bin/*"
    File              "/usr/sbin/*"
    Recursive         TRUE
    ScanInterval      3600
    FollowSymlinks    TRUE
</Input>

Example 2. Scheduled scan

The im_fim module provides a start_scan() procedure that can be called to invoke the scan. The following configuration sets ScanInterval to zero to disable periodic scanning and uses a Schedule block instead to trigger the scan every day at midnight.

nxlog.conf

<Input fim>
    Module          im_fim
    File            "/bin/*"
    File            "/sbin/*"
    File            "/usr/bin/*"
    File            "/usr/sbin/*"
    Recursive       TRUE
    ScanInterval    0
    <Schedule>
        When    @daily
        Exec    start_scan();
    </Schedule>
</Input>