NXLog Agent log compression and encryption
Data converters
It is good to understand how NXLog Agent data converters work to utilize compression and encryption. Data converters have two functions:
-
To convert output data into the required format, e.g., compressing a file.
-
To convert input data into a format NXLog Agent can read, e.g., decompressing a file.
You can use data converters with stream-oriented modules that support the InputType and OutputType directives. These directives are also used to specify an input reader and output writer function, which defines how the instance should interpret or write logs, e.g., one record per line or datagram.
Reader/writer functions and data converters are executed sequentially from left to right. Therefore, you must specify them in the correct processing order. For example, the following diagram illustrates the flow for an om_file instance with two data converters.
Similarly, the following diagram illustrates the flow for an im_file instance with two data converters.
Log compression
The xm_zlib module provides data converters to compress and decompress logs. The following table shows the order of operations for compressing and decompressing log files.
Directive | First operation | Second operation | |
---|---|---|---|
Compression |
Output writer function (e.g., LineBased) |
||
Decompression |
Input reader function (e.g., LineBased) |
See Compress and decompress logs for configuration examples.
Log encryption
The xm_crypto module provides data converters to encrypt and decrypt logs. The following table shows the order of operations for encrypting and decrypting log files.
Directive | First operation | Second operation | |
---|---|---|---|
Encryption |
Output writer function (e.g., LineBased) |
||
Decryption |
Input reader function (e.g., LineBased) |
See Encrypt and decrypt logs for configuration examples.
Combined log compression and encryption
You can combine data converters provided by the xm_zlib and xm_crypto modules to compress and encrypt logs simultaneously. The following table shows the order for combining compression and encryption operations.
Directive | First operation | Second operation | Third operation | |
---|---|---|---|---|
Compression + Encryption |
Output writer function (e.g., LineBased) |
|||
Decompression + Decryption |
Input reader function (e.g., LineBased) |
See Combine compression and encryption for configuration examples.