macOS ULS (im_maculs)

This module natively collects Apple logs from Apple’s unified logging system (ULS) on macOS. ULS logs are typically used for application debugging and performance analysis. ULS logs are also available on iOS, tvOS, and watchOS. See Apple’s Logging documentation for more information.

The im_maculs module supports the following chunk tags as described in this open source project.

Value	Identifier
0x1000	Header
0x6001	Firehose
0x6002	Oversize
0x6003	StateDump
0x600b	Catalog
0x600d	ChunkSet
0x6004	SimpleDump

Value

Identifier

0x1000

Header

0x6001

Firehose

0x6002

Oversize

0x6003

StateDump

0x600b

Catalog

0x600d

ChunkSet

0x6004

SimpleDump

The im_maculs module is only available on macOS 10.12 and later.

Logs from Apple’s unified logging system (ULS) are stored in a proprietary undocumented format. As a result, some fields might not be parsed by the module, leading to missing data in the output. To display a corresponding message when an unrecognized message is encountered, use the PrintDiagnosticErrors directive.

Configuration

The im_maculs module accepts the following directives in addition to the common module directives.

Optional directives

ActiveFiles

Specifies the maximum number of simultaneously opened file descriptors. It does not affect the amount of data kept in memory. For optimum efficiency, it is recommended to use at least 20 active files (the default value). The minimum supported value is 2.

CacheSize

Specifies the maximum size of cache for DSC and UUIDText files. Values specified as integers are interpreted as bytes. Human-readable formats using k, M or G suffixes can also be used. The smallest supported cache size is 256K. The default is 2M.

Caching

This boolean directive defaults to TRUE, which instructs the module instance to cache all required information from DSC and UUIDText files in memory. This increases overall performance since meta files do not have to be repeatedly read from disk. The cache will persist as long as the nxlog process is not terminated. If the available cache size is exceeded, older files are removed from the cache to make space for newer files. Setting this directive to FALSE can result in a serious performance penalty.

DirCheckInterval

Specifies how frequently, in seconds, the module will check the monitored directory for new files. The default value is 2, twice the default value of the PollInterval directive. Fractional seconds may be specified. We recommended increasing the default if many files cannot be rotated out and the nxlog process is causing a high CPU load.

PollInterval

Specifies in seconds how often to check for new log entries. Fractional values are allowed. The default value is 1.

ReadFromLast

This optional boolean directive instructs the module to only read logs that arrive after NXLog Agent is started. This directive comes into effect if a saved position is not found, for example on the first start, or when the SavePos directive is FALSE. When the SavePos directive is TRUE and a previously saved position is found, the module will always resume reading from the saved position. If ReadFromLast is FALSE, the module will read all logs from the beginning of the file. This can result in a lot of messages and is usually not the expected behavior. If this directive is not specified, it defaults to TRUE.

The following matrix shows the outcome of this directive in conjunction with the SavePos directive:

ReadFromLast SavePos SavedPosition Outcome

TRUE

Reads events that are logged after NXLog Agent is started.

TRUE

Yes

Reads events from the saved position.

TRUE

FALSE

Reads events that are logged after NXLog Agent is started.

TRUE

FALSE

Yes

Reads events that are logged after NXLog Agent is started.

FALSE

TRUE

Reads all events.

FALSE

TRUE

Yes

Reads events from the saved position.

FALSE

Reads all events.

FALSE

Yes

Reads all events.

Based on the assumption that .tracev3 files are edited by /bin/log strictly in alphabetical order, once a file has been read and a newer file has been added, older files are no longer of interest to the module and do not need to be maintained in memory. Consequently, files that supersede previously read files are normally read in order of their appearance after nxlog starts up.

PrintDiagnosticErrors

This optional directive specifies whether diagnostic messages should be printed. This directive is useful for troubleshooting missing data in the output due to unrecognized input data. If this directive is not specified, it defaults to FALSE.

SavePos

If this boolean directive is set to TRUE, the position of all TraceV3 files will be saved when NXLog Agent exits. The file position will be read from the cache file upon startup. The default is TRUE, the file position will be saved if this directive is not specified. This directive affects the outcome of the ReadFromLast directive. The SavePos directive can be overridden by the global NoCache directive.

As in the case of the ReadFromLast directive, only files with the last read position saved in the cache and those that have superseded them will be read.

TimeSyncPath

This optional directive specifies the root path of TimeSync (.timesync) files. If not specified, it defaults to /var/db/diagnostics/timesync/.

TraceV3Path

This optional directive specifies root path of TraceV3 (.tracev3) file(s). If the specified path is a directory, it will be handled recursively. The default path is /var/db/diagnostics/.

UUIDTextPath

This optional directive specifies the root path of UUIDText and DSC files. If not specified, it defaults to /var/db/uuidtext/.

Fields

The following fields are used by im_maculs.

$raw_event (type: string): A list of event fields in key-value pairs.

$activityIdentifier (type: integer): A numeric ID for an activity.

$bootUUID (type: string): The boot UUID of the event.

$category (type: string): The category used to log an event. Only works with log messages generated with os_log(3) APIs.

$eventMessage (type: string): The message for that log entry.

$EventTime (type: datetime): The date and time of the event.

$eventType (type: string): The type of event: logEvent, signpostEvent or stateEvent.

$formatString (type: string): This gives the format string used to convert variable content into a string for output.

$machTimestamp (type: string): The number of Mach system clock ticks, which is effectively elapsed nanoseconds.

$messageType (type: string): Type of the logged message e.g. Default, Info, Activity, Debug, Error, Fault and Loss.

$parentActivityIdentifier (type: integer): A numeric ID for the parent activity.

$processID (type: integer): The ID of the process that originated the event.

$processImagePath (type: string): The full path of the process that originated the event.

$processImageUUID (type: string): The UUID of the process that originated the event.

$senderImagePath (type: string): The full path of the library, framework, kernel extension, or mach-o image, that originated the event.

$senderImageUUID (type: string): The UUID of the library, framework, kernel extension, or mach-o image, that originated the event.

$senderProgramCounter (type: integer): The program counter of the library, framework, kernel extension, or mach-o image, that originated the event.

$signpostID (type: string): A numeric ID for a Signpost.

$signpostName (type: string): A string name for a Signpost.

$signpostScope (type: string): A Signpost scope string, such as thread, process or system.

$signpostType (type: string): The Signpost type: begin, end, or event.

$subsystem (type: string): The subsystem used to log an event. Only works with log messages generated with os_log(3) APIs.

$threadID (type: string): The ID of the thread that originated the event.

$traceID (type: string): The ID of a trace event

$TTL (type: integer): The time-to-live (TTL) of the log message.

Examples

Example 1. Collecting ULS logs on macOS

With this configuration, NXLog Agent will collect all available ULS logs from hundreds of log sources.

nxlog.conf

<Input in>
    Module              im_maculs

    UUIDTextPath        "/var/db/uuidtext"
    TimeSyncPath        "/var/db/diagnostics/timesync"
    TraceV3Path         "/var/db/diagnostics"

    Caching             TRUE
    CacheSize           256M

    PollInterval        1.0
    DirCheckInterval    2.0

    ActiveFiles         20
    SavePos             FALSE
    ReadFromLast        FALSE
</Input>