Bandwidth usage
You can transfer logs between NXLog Agent instances and NXLog Platform in several ways. Some of the modules, such as the TCP and SSL modules, do not provide data compression. On the other hand, the NXLog Transport modules im_batchcompress and om_batchcompress transfer events in compressed and optionally encrypted batches.
The following chart compares the data requirements for the TCP, SSL (with TLSv1.2), and NXLog Transport modules. The dataset consisted of BSD syslog records parsed with the parse_syslog() procedure. The values in this chart reflect the total bi-directional bytes transferred at the packet level. The ratios may vary depending on network conditions and the compressibility of the log data.
As you can see from the chart, transferring logs with the NXLog Transport modules significantly reduces bandwidth usage. The following table further compares sending the same dataset using different methods and modules.
| Compression method | Modules used | Event size | Diff vs baseline | Sender CPU usage | Receiver CPU usage | EPS sender | EPS receiver |
|---|---|---|---|---|---|---|---|
None |
om_tcp, im_tcp |
112 |
0.00% |
141 |
215.07 |
83091.8 |
84169.9 |
None |
om_ssl, im_ssl |
301.7 |
+169.38% |
141.34 |
191.9 |
33161.4 |
47482.9 |
SSLCompression |
om_ssl, im_ssl |
293.2 |
+161.79% |
138.98 |
190.69 |
34497.7 |
47128.5 |
Batch compression |
om_batchcompress, im_batchcompress |
18.4 |
-83.57% |
119.69 |
181.1 |
36252.1 |
77491.8 |
Compression ratios show that enabling SSL compression yields only a minimal reduction in message size. NXLog Transport fares much better because it compresses data in batches, leading to better compression ratios. For configuration examples using NXLog Transport, see Transfer logs between NXLog Agent instances.
| For other strategies to reduce the size of log data, see Filter logs and Trim log events. |