NXLog Agent data compression and encryption

NXLog Agent’s xm_zlib and xm_crypto extension modules provide data converters to compress/decompress and encrypt/decrypt log files. You can use these data converters with the im_file and om_file modules.

Data converters

It is good to understand how NXLog Agent data converters work to utilize compression and encryption. Data converters have two functions:

To convert output data into the required format, e.g., compressing a file.
To convert input data into a format NXLog Agent can read, e.g., decompressing a file.

You can use data converters with stream-oriented modules that support the InputType and OutputType directives. These directives are also used to specify an input reader and output writer function, which defines how the instance should interpret or write data, e.g., one record per line or datagram.

Reader/writer functions and data converters are executed sequentially from left to right. Therefore, you must specify them in the correct processing order. For example, the following diagram illustrates the flow for an om_file instance with two data converters.

Similarly, the following diagram illustrates the flow for an im_file instance with two data converters.

Data compression

The xm_zlib module provides data converters to compress and decompress log files. The following table shows the order of operations for compressing and decompressing log files.

	Directive	First operation	Second operation
Compression	OutputType	Output writer function (e.g., LineBased)	compress
Decompression	InputType	decompress	Input reader function (e.g., LineBased)

Directive

First operation

Second operation

Compression

OutputType

Output writer function (e.g., LineBased)

compress

Decompression

InputType

decompress

Input reader function (e.g., LineBased)

See Compress and decompress data for configuration examples.

Data encryption

The xm_crypto module provides data converters to encrypt and decrypt log files. The following table shows the order of operations for encrypting and decrypting log files.

	Directive	First operation	Second operation
Encryption	OutputType	Output writer function (e.g., LineBased)	aes_encrypt
Decryption	InputType	aes_decrypt	Input reader function (e.g., LineBased)

Directive

First operation

Second operation

Encryption

OutputType

Output writer function (e.g., LineBased)

aes_encrypt

Decryption

InputType

aes_decrypt

Input reader function (e.g., LineBased)

See Encrypt and decrypt data for configuration examples.

Combined data compression and encryption

You can combine data converters provided by the xm_zlib and xm_crypto modules to compress and encrypt log files simultaneously. The following table shows the order for combining compression and encryption operations.

	Directive	First operation	Second operation	Third operation
Compression + Encryption	OutputType	Output writer function (e.g., LineBased)	compress	aes_encrypt
Decompression + Decryption	InputType	aes_decrypt	decompress	Input reader function (e.g., LineBased)

Directive

First operation

Second operation

Third operation

Compression + Encryption

OutputType

Output writer function (e.g., LineBased)

compress

aes_encrypt

Decompression + Decryption

InputType

aes_decrypt