macOS ULS (im_maculs)

This module natively collects Apple logs from Apple’s unified logging system (ULS) on macOS. ULS logs are typically used for application debugging and performance analysis. ULS logs are also available on iOS, tvOS, and watchOS. See Apple’s Logging documentation for more information.

The im_maculs module supports the following chunk tags as described in this open source project.

Value	Identifier
0x1000	Header
0x6001	Firehose
0x6002	Oversize
0x6003	StateDump
0x600b	Catalog
0x600d	ChunkSet

Value

Identifier

0x1000

Header

0x6001

Firehose

0x6002

Oversize

0x6003

StateDump

0x600b

Catalog

0x600d

ChunkSet

Chunk tag 0x6004 (SimpleDump) is not supported.

To examine the supported platforms, see the list of installer packages in the Available Modules chapter.

The im_maculs module is only available on macOS 10.12 and later.

Logs from Apple’s unified logging system (ULS) are stored in a proprietary undocumented format. As a result, some fields might not be parsed by the module, leading to missing data in the output. To display a corresponding message when an unrecognized message is encountered, use the PrintDiagnosticErrors directive.

Configuration

The im_maculs module accepts the following directives in addition to the common module directives.

ActiveFiles: Specifies the maximum number of simultaneously opened file descriptors. It does not affect the amount of data kept in memory. For optimum efficiency, it is recommended to use at least 20 active files (the default value). The minimum supported value is 2.

CacheSize: Specifies the maximum size of cache for DSC and UUIDText files. Values specified as integers are interpreted as bytes. Human-readable formats using k, M or G suffixes can also be used. The smallest supported cache size is 256K. The default is 2M.

Caching: This boolean directive defaults to TRUE, which instructs the module instance to cache all required information from DSC and UUIDText files in memory. This increases overall performance since meta files do not have to be repeatedly read from disk. The cache will persist as long as the nxlog process is not terminated. If the available cache size is exceeded, older files are removed from cache to make room for newer files. Setting this directive to FALSE can result in a serious performance penalty.

DirCheckInterval: Specifies how frequently, in seconds, the module will check the monitored directory for new files. The default value is 2, twice the default value of the PollInterval directive. Fractional seconds may be specified. It is recommended to increase the default if there are many files which cannot be rotated out and the nxlog process is causing high CPU load.

PollInterval: Specifies in seconds how often to check for new log entries. Fractional values are allowed. The default value is 1.

ReadFromLast

This optional boolean directive instructs the module to only read logs which arrive after NXLog is started. This directive comes into effect if a saved position is not found, for example on first start, or when the SavePos directive is FALSE. When the SavePos directive is TRUE and a previously saved position is found, the module will always resume reading from the saved position. If ReadFromLast is FALSE, the module will read all logs from the beginning of the file. This can result in a lot of messages and is usually not the expected behavior. If this directive is not specified, it defaults to TRUE.

The following matrix shows the outcome of this directive in conjunction with the SavePos directive:

ReadFromLast SavePos Saved Position Outcome

ReadFromLast	SavePos	Saved Position	Outcome
`TRUE`	`TRUE`	No	Reads events that are logged after NXLog is started.
`TRUE`	`TRUE`	Yes	Reads events from saved position.
`TRUE`	`FALSE`	No	Reads events that are logged after NXLog is started.
`TRUE`	`FALSE`	Yes	Reads events that are logged after NXLog is started.
`FALSE`	`TRUE`	No	Reads all events.
`FALSE`	`TRUE`	Yes	Reads events from saved position.
`FALSE`	`FALSE`	No	Reads all events.
`FALSE`	`FALSE`	Yes	Reads all events.

TRUE

Reads events that are logged after NXLog is started.

TRUE

Yes

Reads events from saved position.

TRUE

FALSE

Reads events that are logged after NXLog is started.

TRUE

FALSE

Yes

Reads events that are logged after NXLog is started.

FALSE

TRUE

Reads all events.

FALSE

TRUE

Yes

Reads events from saved position.

FALSE

Reads all events.

FALSE

Yes

Reads all events.

Based on the assumption that .tracev3 files are edited by /bin/log strictly in alphabetical order, once a file has been read and a newer file has been added, older files are no longer of interest to the module and do not need to be maintained in memory. Consequently, files that supersede previously read files are normally read in order of their appearance after nxlog starts up.

PrintDiagnosticErrors: This optional directive specifies whether diagnostic messages should be printed. This directive is useful for troubleshooting missing data in the output due to unrecognized input data. If this directive is not specified, it defaults to FALSE.

SavePos

If this boolean directive is set to TRUE, the position of all TraceV3 files will be saved when NXLog exits. The file position will be read from the cache file upon startup. The default is TRUE, the file position will be saved if this directive is not specified. This directive affects the outcome of the ReadFromLast directive. The SavePos directive can be overridden by the global NoCache directive.

As in case of ReadFromLast directive, only the files with their positions saved in cache and those that have superseded them will be read.

TimeSyncPath: This optional directive specifies root path of TimeSync (.timesync) files. If not specified, it defaults to /var/db/diagnostics/timesync/.

TraceV3Path: This optional directive specifies root path of TraceV3 (.tracev3) file(s). If specified path is a directory, it will be handled recursively. The default path is /var/db/diagnostics/.

UUIDTextPath: This optional directive specifies root path of UUIDText and DSC files. If not specified, it defaults to /var/db/uuidtext/.

Fields

The following fields are used by im_maculs.

$raw_event (type: string): A list of event fields in key-value pairs.

$activityIdentifier (type: integer): A numeric ID for an activity.

$bootUUID (type: string): The boot UUID of the event.

$category (type: string): The category used to log an event. Only works with log messages generated with os_log(3) APIs.

$eventMessage (type: string): The message for that log entry.

$EventTime (type: datetime): The date and time of the event.

$eventType (type: string): The type of event: logEvent, signpostEvent or stateEvent.

$formatString (type: string): This gives the format string used to convert variable content into a string for output.

$machTimestamp (type: string): The number of Mach system clock ticks, which is effectively elapsed nanoseconds.

$messageType (type: string): Type of the logged message e.g. Default, Info, Activity, Debug, Error, Fault and Loss.

$parentActivityIdentifier (type: integer): A numeric ID for the parent activity.

$processID (type: integer): The ID of the process that originated the event.

$processImagePath (type: string): The full path of the process that originated the event.

$processImageUUID (type: string): The UUID of the process that originated the event.

$senderImagePath (type: string): The full path of the library, framework, kernel extension, or mach-o image, that originated the event.

$senderImageUUID (type: string): The UUID of the library, framework, kernel extension, or mach-o image, that originated the event.

$senderProgramCounter (type: integer): The program counter of the library, framework, kernel extension, or mach-o image, that originated the event.

$signpostID (type: string): A numeric ID for a Signpost.

$signpostName (type: string): A string name for a Signpost.

$signpostScope (type: string): A Signpost scope string, such as thread, process or system.

$signpostType (type: string): The Signpost type: begin, end, or event.

$subsystem (type: string): The subsystem used to log an event. Only works with log messages generated with os_log(3) APIs.

$threadID (type: string): The ID of the thread that originated the event.

$traceID (type: string): The ID of a trace event

$TTL (type: integer): The time-to-live (TTL) of the log message.

Examples

Example 1. Collecting ULS logs on macOS

With this configuration, NXLog will collect all available ULS logs from hundreds of log sources.

To learn more, about how to collect macOS logs with the im_maculs module, see the Unified logging since Sierra section in the NXLog User Guide.

nxlog.conf

<Input in>
    Module              im_maculs

    UUIDTextPath        "/var/db/uuidtext"
    TimeSyncPath        "/var/db/diagnostics/timesync"
    TraceV3Path         "/var/db/diagnostics"

    Caching             TRUE
    CacheSize           256M

    PollInterval        1.0
    DirCheckInterval    2.0

    ActiveFiles         20
    SavePos             FALSE
    ReadFromLast        FALSE
</Input>