Collecting macOS system logs

NXLog Agent provides specialized modules to collect ULS, Endpoint Security auditing logs, and other log sources from macOS Sierra 10.12 and newer versions.

Regaining visibility of redacted, private data

ULS redacts sensitive data that could be used for malicious purposes. Dynamic strings, collections, and objects are redacted with <private>, as seen in the following Console example.

Private data redacted in the macOS console

An Apple developer documented how to show private log messages in Catalina’s Console.app. The mobile configuration profile provided in this solution, enable-unified-log-private-data.mobileconfig, makes the private message details visible on macOS Catalina 10.15.3 and later.

Private data visibility restored in the console

Capturing ULS logs using im_maculs

The im_maculs module reads ULS logs natively and can collect any logs generated by macOS. This centralized logging facility captures all events and stores them internally as structured data.

Each ULS event is stored using one of two schemas:

Standard: Most macOS logs, including information, warnings, errors, and debugging logs, use the standard schema.
Signpost: macOS signposts, i.e., application performance monitoring and debugging events, use the signpost schema. For more information, see Recording Performance Data in the Apple Developer Documentation.

The macOS unified logging system handles hundreds of log sources. Given the large number of logs stored in this global repository of system and application events, it is crucial to be selective when configuring each im_maculs module instance. Ideally, each instance should have a filter that captures only one or two log sources, as shown in the following example.

Example 1. Collecting macOS ULS logs

The following configuration reads all ULS logs generated from multiple log sources and saves only two types of events:

Events containing an $eventType field with a value of signpostEvent
Events with the $subsystem set to com.apple.runningboard

nxlog.conf

<Extension json>
    Module          xm_json
</Extension>

<Input macos_uls>
    Module          im_maculs
    UUIDTextPath    "/var/db/uuidtext"
    TimeSyncPath    "/var/db/diagnostics/timesync"
    TraceV3Path     "/var/db/diagnostics"
    <Exec>
        # Filter only the events needed
        if $eventType == 'signpostEvent' or
           $subsystem == 'com.apple.runningboard'
        {
            to_json(); (1)
        }
        else
        {
            drop(); (2)
        }
    </Exec>
</Input>

1	The to_json() procedure converts the record to JSON format and adds the NXLog Agent core fields.
2	The drop() procedure discards any event that fails to match the requirements.

The following JSON sample shows an event from the com.apple.runningboard subsystem that uses the standard schema.

Standard schema output sample

{
  "eventMessage": "Acquired process power assertion with ID 41468 for pid 650",
  "formatString": "Acquired process power assertion with ID %{public}d for pid %{public}d",
  "activityIdentifier": 6872151,
  "subsystem": "com.apple.runningboard",
  "category": "power",
  "threadID": 11163152,
  "senderImageUUID": "cb3dc3fb-6454-3510-a1c2-0cc65d53b22d",
  "bootUUID": "80243e89-2bee-44e5-b841-736299e7c5e3",
  "processImagePath": "/usr/libexec/runningboardd",
  "senderImagePath": "/System/Library/PrivateFrameworks/RunningBoard.framework/Versions/A/RunningBoard",
  "EventTime": "2021-04-04T23:46:35.207740-05:00",
  "machTimestamp": 43611630764906,
  "messageType": "Default",
  "processImageUUID": "901f428b-2676-3774-b480-9b1604a32363",
  "processID": 371,
  "senderProgramCounter": 39036,
  "parentActivityIdentifier": 0,
  "TTL": 0,
  "EventReceivedTime": "2021-04-04T23:46:44.565064-05:00",
  "SourceModuleName": "macos_uls",
  "SourceModuleType": "im_maculs"
}

The following JSON sample shows a Signpost event.

Signpost schema output sample

{
  "eventType": "signpostEvent",
  "signpostID": -1229851353791926500,
  "signpostScope": "process",
  "formatString": "%{public, signpost.description:begin_time}llu",
  "activityIdentifier": 0,
  "subsystem": "com.apple.SkyLight",
  "category": "performance_instrumentation",
  "threadID": 3328,
  "senderImageUUID": "e50df124-c72e-3349-9b99-788bf3160420",
  "signpostType": "event",
  "bootUUID": "80243e89-2bee-44e5-b841-736299e7c5e3",
  "processImagePath": "/System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/Resources/WindowServer",
  "senderImagePath": "/System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/SkyLight",
  "signpostName": "CompositeLoop",
  "EventTime": "2021-04-04T22:57:19.476038-05:00",
  "machTimestamp": 43540691469555,
  "eventMessage": "43540691207161",
  "processImageUUID": "ddc145f0-07cc-3b36-9e61-742786349f3f",
  "processID": 353,
  "senderProgramCounter": 2753288,
  "parentActivityIdentifier": 0,
  "TTL": 0,
  "EventReceivedTime": "2021-04-04T22:59:17.683574-05:00",
  "SourceModuleName": "macos_uls",
  "SourceModuleType": "im_maculs"
}

Capturing ULS events with the built-in log stream command

Viewing the events emitted by the macOS built-in log command can be helpful for forensic analysis. You can achieve this with the im_exec module, which can execute a command with optional arguments.

Example 2. Using im_exec to run log stream

This configuration uses the im_exec input module to call the built-in log command and generate a stream of ULS events.

For a complete list of command line options, invoke man log within a terminal.

nxlog.conf

<Extension json>
    Module      xm_json
</Extension>

<Input macos_log>
    Module      im_exec
    Command     /usr/bin/log
    Arg         stream
    Arg         --style=ndjson
    Arg         --type=log (1)
    <Exec>
        parse_json();
        to_json(); (2)
    </Exec>
</Input>

1	For more verbose output, use the `--level` option to explicitly choose one of three log level options: `default`, `info`, or `debug`.
2	The to_json() procedure converts the record back to JSON format and adds the NXLog Agent core fields.

The configuration creates line-delimited JSON (NDJSON) records suitable for routing to various NXLog Agent output modules and forwarding to most SIEMs. The following ULS event sample is pretty-printed for display purposes.

Sample log stream event

{
  "EventReceivedTime": "2021-04-08T22:28:22.002528-05:00",
  "SourceModuleName": "macos_log",
  "SourceModuleType": "im_exec",
  "traceID": 1824493551157252,
  "eventMessage": "ocsp responder: (null) did not include status of requested cert",
  "eventType": "logEvent",
  "source": null,
  "formatString": "ocsp responder: %@ did not include status of requested cert",
  "activityIdentifier": 8049628,
  "subsystem": "com.apple.securityd",
  "category": "ocsp",
  "threadID": 13337131,
  "senderImageUUID": "AD8A1343-96A6-3E87-8CFA-14FE13754B06",
  "backtrace": {
    "frames": [
      {
        "imageOffset": 256208,
        "imageUUID": "AD8A1343-96A6-3E87-8CFA-14FE13754B06"
      }
    ]
  },
  "bootUUID": "",
  "processImagePath": "/usr/libexec/trustd",
  "timestamp": "2021-04-08T22:28:22.002306-05:00",
  "senderImagePath": "/usr/libexec/trustd",
  "machTimestamp": 51793593259945,
  "messageType": "Default",
  "processImageUUID": "AD8A1343-96A6-3E87-8CFA-14FE13754B06",
  "processID": 565,
  "senderProgramCounter": 256208,
  "parentActivityIdentifier": 0,
  "timezoneName": "",
  "Hostname": "mm"
}

Collecting Endpoint Security logs with im_maces

The im_maces module collects logs from Apple’s Endpoint Security auditing system on macOS 10.15 and later versions. Endpoint Security is an audit subsystem that monitors system events for potentially malicious activity.

Example 3. Collecting Apple Endpoint Security system events

This configuration uses the im_maces input module to collect the listed NotifyEvents.

nxlog.conf

<Input macos_notify>
    Module          im_maces
    NotifyEvents    get_task, proc_check, write
</Input>

Collecting other macOS logs

NXLog Agent provides several modules to collect logs from other macOS log sources, including OS X El Capitan and older versions.

Basic Security Mode (BSM) auditing: The im_bsm module collects BSM audit logs directly from the kernel. Alternatively, you can collect BSM logs from the log files and parse them with the xm_bsm extension.
Process accounting logs: Monitor executed commands and process activity with the im_acct input module.
Log files: You can use the im_file input module to collect file-based logs in any format, such as syslog, JSON, and CSV, and parse them with one of the specialized extensions.
File Integrity Monitoring: Monitor file and directory changes with NXLog Agent. For more information, see our File Integrity Monitoring integration guide.
macOS System Log files: Collect logs from Apple Log (*.asl) files and parse them with the xm_asl extension. This option does not apply to macOS Sierra 10.12 and newer versions.

Disclaimer

While we endeavor to keep the information in our guides up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

NXLog Agent version 5.2.663
Apple macOS 10.15.3 (Catalina)
Apple macOS 11.2.3 (Big Sur)

Last revision: 11 April 2021