NXLog Agent log compression and encryption

NXLog Agent’s xm_zlib and xm_crypto extension modules provide data converters to compress/decompress and encrypt/decrypt logs. You can use these data converters with the im_file and om_file modules.

Data converters

It is good to understand how NXLog Agent data converters work to utilize compression and encryption. Data converters have two functions:

  • To convert output data into the required format, e.g., compressing a file.

  • To convert input data into a format NXLog Agent can read, e.g., decompressing a file.

You can use data converters with stream-oriented modules that support the InputType and OutputType directives. These directives are also used to specify an input reader and output writer function, which defines how the instance should interpret or write logs, e.g., one record per line or datagram.

Reader/writer functions and data converters are executed sequentially from left to right. Therefore, you must specify them in the correct processing order. For example, the following diagram illustrates the flow for an om_file instance with two data converters.

gv output data converters

Similarly, the following diagram illustrates the flow for an im_file instance with two data converters.

gv input data converters

Log compression

The xm_zlib module provides data converters to compress and decompress logs. The following table shows the order of operations for compressing and decompressing log files.

Directive First operation Second operation

Compression

OutputType

Output writer function (e.g., LineBased)

compress

Decompression

InputType

decompress

Input reader function (e.g., LineBased)

See Compress and decompress logs for configuration examples.

Log encryption

The xm_crypto module provides data converters to encrypt and decrypt logs. The following table shows the order of operations for encrypting and decrypting log files.

Directive First operation Second operation

Encryption

OutputType

Output writer function (e.g., LineBased)

aes_encrypt

Decryption

InputType

aes_decrypt

Input reader function (e.g., LineBased)

See Encrypt and decrypt logs for configuration examples.

Combined log compression and encryption

You can combine data converters provided by the xm_zlib and xm_crypto modules to compress and encrypt logs simultaneously. The following table shows the order for combining compression and encryption operations.

Directive First operation Second operation Third operation

Compression + Encryption

OutputType

Output writer function (e.g., LineBased)

compress

aes_encrypt

Decompression + Decryption

InputType

aes_decrypt

decompress

Input reader function (e.g., LineBased)

See Combine compression and encryption for configuration examples.