Buffering and flow control
NXLog implements several buffering features. Of these, two are particularly important and are enabled by default: log queues and flow control.
- Log queues
-
Every processor and output module instance has an input log queue for events that have not yet been processed by that module instance. When the preceding module has processed an event, it is placed in this queue. Log queues are enabled by default for all processor and output module instances; adjusting log queue sizes is the preferred way to control buffering behavior.
- Flow control
-
NXLog’s flow control functionality provides automatic, zero-configuration handling of many cases where buffering would otherwise be required. Flow control takes effect when the following sequence of events occurs in a route:
-
a processor or output module instance is not able to process log data at the incoming rate,
-
that module instance’s log queue becomes full, and
-
an input, processor, or output module instance has flow control enabled (the default).
In this case, flow control will cause the input or processor module instance to suspend processing until the succeeding module instance is ready to accept more logs.
Flow control settings can be configured per module instance. If two succeeding module instances in a route have conflicting flow control settings, disabled flow control takes precedence.
When a route contains multiple output instances, it is possible to disable flow control for one output instance so that if it is blocked, it will not suspend log processing for the entire route. In this case, the preceding input or processor module instance will continue to process logs and forward them to the output instances. The blocked output instance will discard events until it recovers.
-
For more information about these and other buffering features, including log queue persistence, disabling flow control, read/write buffers, and examples for specific scenarios, see Using buffers.
