Encryption (xm_crypto)

This module provides encryption and decryption functionality using the AES symmetric encryption algorithm. Decryption of input data is defined within im_file module instances, while encryption of output data is specified within om_file module instances. The functionality of xm_crypto can be combined with other modules providing data conversion such as xm_zlib.

To examine the supported platforms, see the list of installer packages in the Available Modules chapter.

Configuration

The xm_crypto module accepts the following directives in addition to the common module directives.

Password: This optional directive specifies a password to be used by the aes_encrypt and aes_decrypt data converters while processing data. This directive is mutually exclusive with the PasswordFile directive.

PasswordFile: This optional directive specifies a path to a file which stores the password to be used by the aes_encrypt and aes_decrypt data converters while processing data. This directive is mutually exclusive with the Password directive.

UseSalt: This optional Boolean directive specifies whether to use a randomly-generated combination of characters for encryption and decryption. The default value is TRUE.

Iter: This optional directive enhances security by enabling the PBKDF2 algorithm and setting the iteration count during the key generation. For more details, see the EVP_BytesToKey section on the OpenSSL website. The default value is 0.

Data conversion

The xm_crypto module implements data converters to be used with the im_file and om_file modules. These are specified in the InputType and OutputType directives of their respective module and are invoked using dot notation:

<InstanceName>.<DataConverterName>

Where <InstanceName> is the given name of the xm_crypto instance and <DataConverterName> is the name of the converter being invoked.

The following data converters are available to encrypt and decrypt log data.

aes_encrypt

This data converter is used to encrypt log data. It should be specified in the OutputType directive after the output writer function. The encrypted result is similar to running the following OpenSSL command:

openssl enc -aes256 -md sha256 -pass pass:password -in input_filename -out output_encrypted_filename

Rotation of files is done automatically when encrypting log data with the aes_encrypt data converter. The rotation pattern is original_file -→ original_file.1 -→ original_file.2 -→ original_file.n. There is no built-in removal or cleanup of files.

aes_decrypt

This data converter is used to decrypt log data. It should be specified in the InputType directive before the input reader function. The decrypted result is similar to running the following OpenSSL command:

openssl enc -aes256 -d -md sha256 -pass pass:password -in encrypted_filename -out output_decrypted_filename

Functions

The following functions are exported by xm_crypto.

string aes_decrypt(unknown arg): Decrypt the argument, which can be either string or binary, using AES symmetric-key algorithm, returns the decrypted string.

string aes_encrypt(unknown arg): Encrypt the argument using AES symmetric-key algorithm, returns the encrypted string.

By default, aes_encrypt and aes_decrypt use cipher block chaining (CBC) mode with 256-bit key.

Examples

The examples below describe various ways of processing logs with the xm_crypto module.

Example 1. Encryption of logs

The following configuration uses the im_file module to read log messages. The aes_encrypt data converter encrypts data at the output and saves the results to a file.

nxlog.conf

<Extension cryptography>
    Module           xm_crypto
    Password         <PASSWORD>
</Extension>

<Input input_file>
    Module           im_file
    File             '/tmp/input'
</Input>

<Output output_file>
    Module           om_file
    File             '/tmp/output'
    OutputType       cryptography.aes_encrypt
</Output>

Example 2. Decryption of Logs

The following configuration uses the aes_decrypt data converter to decrypt log messages at the input. The result is saved to a file.

nxlog.conf

<Extension cryptography>
    Module           xm_crypto
    UseSalt          TRUE
    PasswordFile     passwordfile
</Extension>

<Input input_file>
    Module           im_file
    File             '/tmp/input'
    InputType        cryptography.aes_decrypt
</Input>

<Output output_file>
    Module           om_file
    File             '/tmp/output'
</Output>

Example 3. Decryption and encryption of logs

The configuration below uses the aes_decrypt data converter to decrypt input data. The Exec directive in the input instance runs the to_syslog_ietf() procedure to convert messages to the IETF Syslog format. The result is encrypted with the aes_encrypt converter in the output instance and saved to a file.

<Extension cryptography>
    Module           xm_crypto
    UseSalt          TRUE
    PasswordFile     passwordfile
</Extension>

<Extension syslog>
    Module           xm_syslog
</Extension>

<Input input_file>
    Module           im_file
    File             '/tmp/input'
    InputType        cryptography.aes_decrypt
    Exec             to_syslog_ietf();
</Input>

<Output output_file>
    Module           om_file
    File             '/tmp/output'
    OutputType       cryptography.aes_encrypt
</Output>

Data conversion operations can be chained together to create a workflow. For example, the xm_zlib module functionality can be combined with the xm_crypto module to perform compression and encryption operations on log files.

Data converters are processed sequentially from left to right, thus the order that they are specified in is important. When specifying data converters, decryption should always occur before decompression in input instances, while compression should always precede encryption in output instances, as illustrated in the following table.

Table 1. Sequential order of operations for compression/encryption and decompression/decryption
	Directive	First Operation	Second Operation	Third Operation
Compression + Encryption	OutputType	Output writer function	compress	aes_encrypt
Decompression + Decryption	InputType	aes_decrypt	decompress	Input reader function

Example 4. Processing data with multiple data converters

The configuration below processes a zlib-compressed and encrypted log file containing log records in the NXLog Binary format. The input instance decrypts the file using the aes_decrypt converter of the xm_crypto module, and then decompresses it using the decompress converter of the xm_zlib module. Each log record is processed and if it contains the info string it is passed on to the output instance.

The processed log data is written to a file by the output instance in the NXLog Binary format. The data is compressed using the compress converter and finally encrypted using the aes_encrypt converter.

<Extension zlib>
    Module              xm_zlib
    Format              zlib
    CompressionLevel    9
    CompBufSize         16384
    DecompBufSize       16384
</Extension>

<Extension cryptography>
    Module              xm_crypto
    UseSalt             TRUE
    Password            <PASSWORD>
</Extension>

<Input input_file>
    Module              im_file
    File                '/tmp/input'
    InputType           cryptography.aes_decrypt, zlib.decompress, Binary
    Exec                if not ($raw_event =~ /info/) drop();
</Input>

<Output output_file>
    Module              om_file
    File                '/tmp/output'
    OutputType          Binary, zlib.compress, cryptography.aes_encrypt
</Output>

The default input reader and output writer function for the im_file and om_file modules is LineBased. When using the default function, it can be omitted from the respective InputType or OutputType directive.

Example 5. Encrypting of the sensitive data field using the aes_encrypt function

The following configuration uses the aes_encrypt function to encrypt the $raw_event field and rewrite it.

nxlog.conf

ReadTimeout 0.5

<Extension crypto>
    Module           xm_crypto

# default TRUE; uses openssl's "Salted__" header in output
    UseSalt          TRUE
    Password password_text
</Extension>

<Input in>
    Module im_file
    File "modules/extension/crypto/crypto-08.in"
    ReadFromLast FALSE
    SavePos FALSE
</Input>

<Output out>
    Module om_file
    File "tmp/output"
    OutputType   LineBased
    <Exec>
        $raw_event = aes_encrypt($raw_event);
    </Exec>
</Output>

Example 6. Decrypting of the sensitive data field using the aes_decrypt function

The following configuration uses the aes_decrypt function to decrypt the $raw_event field and rewrite it.

nxlog.conf

ReadTimeout 0.5

<Extension crypto>
    Module           xm_crypto

# default TRUE; uses openssl's "Salted__" header in output
    UseSalt          TRUE
    Password password_text
</Extension>

<Input in>
    Module              im_file
    File                'tmp/test.log.enc'
    ReadFromLast        FALSE
    SavePos             TRUE
</Input>


<Output out>
    Module om_file
    File "tmp/output"

    <Exec>
        $raw_event = aes_decrypt($raw_event);
    </Exec>
</Output>

Example 7. Encrypting of the sensitive json field using the aes_encrypt function

The following configuration uses the base64encode on top of aes_encrypt to encrypt the $password field and save it to $encfield.

nxlog.conf

ReadTimeout 0.5

<Extension json_parser>
    Module      xm_json
</Extension>

<Extension crypto>
    Module           xm_crypto

# default TRUE; uses openssl's "Salted__" header in output
    UseSalt          TRUE
    Password password_text
</Extension>

<Input in>
    Module im_file
    File "path/to/json/file"
    ReadFromLast FALSE
    SavePos FALSE
    InputType   json_parser

    <Exec>
	$encfield = base64encode(aes_encrypt($password));
	delete($password);
	to_json();
    </Exec>

</Input>

<Output out>
    Module om_file
    File "path/to/output"
    OutputType   LineBased
</Output>

Example 8. Decrypting of the sensitive json field using the aes_decrypt function

The following configuration uses the aes_decrypt on top of base64decode to decrypt the $encfield field and save it to $password.

nxlog.conf

ReadTimeout 0.5

<Extension json_parser>
    Module      xm_json
</Extension>

<Extension crypto>
    Module           xm_crypto

# default TRUE; uses openssl's "Salted__" header in output
    UseSalt          TRUE
    Password password_text
</Extension>

<Input in>
    Module im_file
    File "path/to/json/file"
    ReadFromLast FALSE
    SavePos FALSE
    InputType   json_parser

    <Exec>
	$password = aes_decrypt(base64decode($encfield)); delete($encfield);
	to_json();
    </Exec>
</Input>

<Output out>
    Module om_file
    File "path/to/output"
    OutputType   LineBased
</Output>

For more information and examples on combined data conversion operations see Compression and Encryption in the User Guide.