Windows Event Collector (im_wseventing)
This module collects Windows events forwarded by Microsoft Windows clients with Windows Event Forwarding (WEF). It takes the role of the collector (Subscription Manager) to accept events over the WS-Management protocol. WS-Eventing is a subset of WS-Management used to forward Windows events.
In comparison, the im_msvistalog module collects Windows events locally or remotely; however, it requires an NXLog agent running on Windows. See Collecting logs from Windows Event Log for more information. On the other hand, the im_wseventing module can be used on all supported platforms, including GNU/Linux systems, to remotely collect Windows Event Logs without requiring additional software installed on the log source.
To examine the supported platforms, see the list of installer packages in the Available Modules chapter. |
Overview of Windows Event Forwarding
WEF is a service that enables forwarding administrative and operational event logs from a Windows server to a central location. There are two roles in a WEF setup:
-
the Windows Event Collector (WEC), also known as a Subscription Manager, which is a server configured to receive events;
-
the WEF client, also known as a forwarder, is a machine configured to forward events to the collector.
The link between the two is known as a subscription and can be configured either as a push or pull service.
NXLog can act as a WEC by configuring the im_wseventing module to collect forwarded events, thus using the push method. Windows machines are then configured as WEF clients, forwarding events to NXLog. The advantage of this method is that you can configure clients to forward their events via Group Policy, removing the overhead of managing a list of clients from which events are to be collected.
The connection between the WEC and the client is mutually authenticated and encrypted using Kerberos by default. However, if you cannot implement Kerberos authentication in your environment, an HTTPS option using certificate-based authentication is also possible. NXLog supports either of these authentication methods.
This documentation refers to the NXLog machine as the WEC server and the Windows machine forwarding events as the WEF client.
While other products, such as IBM WebSphere DataPower, implement the WS-Eventing protocol, the im_wseventing module’s primary purpose is to collect and parse forwarded events from Microsoft Windows. Therefore, we cannot guarantee compatibility with other products. |
Using Kerberos authentication
This section explains how to configure a Microsoft Active Directory domain user to be used with Kerberos authentication. Once these steps are complete, you can configure NXLog to collect events forwarded by WEF clients.
Configuring Kerberos on Linux
The steps and examples below assume the following values:
-
example.com
is the FQDN of the Active Directory domain. -
EXAMPLE
is the name of the Active Directory domain. -
linux
is the name of the Linux WEC server.
WEC server prerequisites
For Kerberos authentication to work successfully, the Linux machine acting as a WEC server must meet the following prerequisites:
-
The OS must return the correct hostname.
-
The machine must be configured with a static IP address.
-
DNS settings must point to the same DNS server used in the Active Directory domain.
-
The time must be synchronized with the Active Directory domain.
Refer to the documentation of your operating system for instructions how to configure and verify these settings.
Create and map the Active Directory domain user
-
On the Active Directory domain controller, create a new user as follows:
-
The account name should match the hostname of the Linux WEC server.
-
Set a password for the user.
-
Disable the User must change password at next logon option.
-
Enable the Password never expires option.
-
The account should have the Kerberos AES 128 bit encryption and Kerberos AES 256 bit encryption options enabled.
-
-
On the DNS server, create an A record for the Linux WEC server with an associated PTR record for reverse lookup.
-
On the Active Directory domain controller, open a command prompt and execute these commands. Use the same
<password>
that was specified when the above user was created. These commands map the domain account to the Kerberos principal names and generate two keytab files containing the shared secret.> ktpass /princ hosts/linux.example.com@EXAMPLE.COM /pass <password> /mapuser EXAMPLE\linux -pType KRB5_NT_PRINCIPAL /out hosts-nxlog.keytab /crypto AES256-SHA1 > ktpass /princ http/linux.example.com@EXAMPLE.COM /pass <password> /mapuser EXAMPLE\linux -pType KRB5_NT_PRINCIPAL /out nxlog.keytab /crypto AES256-SHA1
Refer to the Microsoft documentation for more information on the ktpass tool.
-
Copy the resulting
hosts-nxlog.keytab
andnxlog.keytab
files to the Linux host.
Configure Kerberos on the WEC server
-
Confirm that the Kerberos krb5 client and utility software are installed on the WEC server. The required package can be installed, for example, with
yum install krb5-workstation
orapt install krb5-user
. -
Edit the default Kerberos configuration file, usually located at
/etc/krb5.conf
.-
In section
[domain_realm]
, add:.example.com = EXAMPLE.COM example.com = EXAMPLE.COM
-
In section
[realms]
, add:EXAMPLE.COM = { kdc = example.com admin_server = example.com }
-
Use
ktutil
to merge the two keytab files generated above.# ktutil
The output:
ktutil: rkt /root/hosts-nxlog.keytab ktutil: rkt /root/nxlog.keytab ktutil: wkt /root/nxlog-result.keytab ktutil: q
-
Validate the merged keytab.
# klist -e -k -t /root/nxlog-result.keytab
The output:
Keytab name: FILE:/root/nxlog-result.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 5 17.01.2021 04:16:37 hosts/linux.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 4 17.01.2021 04:16:37 http/linux.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
-
Either copy the keytab into place, or merge it if there are already keys in
/etc/krb5.keytab
.-
To copy the keytab:
cp /root/nxlog-result.keytab /etc/krb5.keytab
-
To merge the keytab and validate the result:
# ktutil
The output:
ktutil: rkt /etc/krb5.keytab ktutil: rkt /root/nxlog-result.keytab ktutil: wkt /etc/krb5.keytab ktutil: q
Validate:
# klist -e -k -t /etc/krb5.keytab
The output:
Keytab name: FILE:/etc/krb5.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ <other entries> 5 17.01.2021 04:20:08 hosts/linux.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 4 17.01.2021 04:20:08 http/linux.example.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
-
Verify that the access permissions of the
/etc/krb5.keytab
file allow the user that NXLog is running under to read it. If not, Kerberos authentication will fail as described in the Troubleshooting section.
-
-
-
Test that the authentication with Active Directory is working successfully when using the keytab. Run the following command on the Linux WEC server and if the configuration is correct a ticket-granting ticket (TGT) will be created and cached. This step should be executed with the user that NXLog is running under, by default the
nxlog
user.# kinit -kt /etc/krb5.keytab http/linux.example.com@EXAMPLE.COM
-
Verify the ticket was obtained by running
klist
with the same user as the previous step:# klist
The output:
Ticket cache: KCM:0 Default principal: http/linux.example.com@EXAMPLE.COM Valid starting Expires Service principal 28/01/21 11:41:44 28/01/21 21:41:44 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 04/02/21 11:41:44
-
Configure and run NXLog. See the configuration example below.
-
Make sure the port defined in the im_wseventing configuration is accessible from the WEF clients. The local firewall rules on the Linux WEC server may need to be updated.
Configure the WEF Client
In this example, WEF is enabled on a single Windows node by editing the local policy of the server. To enable WEF on multiple nodes, a group policy on the domain should be configured.
-
Run
gpedit.msc
and go to Computer Configuration > Administrative Templates > Windows Components > Event Forwarding. -
Open and enable the Configure target Subscription Manager setting.
-
Click Show… beside the SubscriptionManagers option.
-
Type into the Value field:
Server=http://linux.example.com:80,Refresh=30
. -
Open command prompt and run
gpupdate /force
. -
At this point the WinRM service on the Windows client should connect to NXLog. If LogConnections is set to
TRUE
a connection attempt should be logged innxlog.log
and you should soon start seeing events arriving.
The Refresh interval represents the time in seconds and on production
systems should be set to a higher value (e.g. Refresh=1200 ).
When the Windows client is set to reconnect too frequently it will result in
a lot of connection/disconnection messages in nxlog.log .
|
Configuring Kerberos on Windows
The steps and examples below assume the following values:
-
example.com
is the FQDN of the Active Directory domain. -
EXAMPLE
is the name of the Active Directory domain. -
wecserver
is the name of the WEC server.
WEC server prerequisites
Before configuring Kerberos authentication, the Windows host acting as a WEC server needs to satisfy the following requirements:
-
The OS must return the correct hostname.
-
The machine must be configured with a static IP address.
-
DNS settings must point to the same DNS server used by the domain controller.
-
The time must be synchronized with the domain controller.
-
If a firewall is enabled, it must allow inbound TCP traffic on port 8080 (or the port of your choice) to the WEC server.
Configure DNS Server settings
On your Windows DNS Server:
-
Open Server Manager and navigate to Tools > DNS to open DNS Manager.
-
Verify that
Host (A)
and associated pointerPTR
records exist for the WEC server. -
If not, expand <server_name> > Forward Lookup Zones and right-click on your domain name.
-
Select New Host (A or AAAA…).
-
Enter the WEC server details and enable the Create associated pointer (PTR) record option.
-
Click Add Host.
Create and map the Active Directory domain user
-
On the Active Directory domain controller, create a new user as follows:
-
The account name should match the hostname of the WEC server.
-
Set a password for the user.
-
Disable the User must change password at next logon option.
-
Enable the Password never expires option.
-
The account should have the Kerberos AES 128 bit encryption and Kerberos AES 256 bit encryption options enabled.
-
-
Note down the account logon name, e.g.,
winserver@example.com (EXAMPLE\wecserver)
-
Open a command prompt and execute these commands. Use the same
<password>
that you specified when you created the above user. These commands map the domain account to the Kerberos principal names and generate two keytab files containing the shared secret.> ktpass /princ hosts/wecserver.example.com@EXAMPLE.COM /pass <password> /mapuser EXAMPLE\wecserver -pType KRB5_NT_PRINCIPAL /out hosts-nxlog.keytab /crypto AES256-SHA1 > ktpass /in hosts-nxlog.keytab /princ http/wecserver.example.com@EXAMPLE.COM /pass <password> /mapuser EXAMPLE\wecserver -pType KRB5_NT_PRINCIPAL /out krb5.keytab /crypto AES256-SHA1
This will result in a single
krb5.keytab
file containing both principals. -
Verify the keytab file:
> ktpass /in krb5.keytab
The output should be similar to the following. The warning message can be ignored.
Existing keytab: Keytab version: 0x502 keysize 95 hosts/wecserver.EXAMPLE.com@EXAMPLE.COM ptype 1 (KRB5_NT_PRINCIPAL) vno 17 etype 0x12 (AES256-SHA1) keylength 32 (0xfda8dc9a12445f01773179ec94ea9441317846741366759b9cd03da7d291013b) keysize 94 http/wecserver.EXAMPLE.COM@EXAMPLE.COM ptype 1 (KRB5_NT_PRINCIPAL) vno 18 etype 0x12 (AES256-SHA1) keylength 32 (0xd351a92138a72c87ab04fee9ff8d6f769048125b81c47cd141d1f41ced52bbbf) WARNING: No principal name specified.
Refer to the Microsoft documentation for more information on the ktpass tool.
Configure the WEC server
-
Download the relevant MIT Kerberos for Windows package from the MIT Kerberos Distribution Page.
-
Execute the package and follow the wizard to perform a Complete installation.
-
Once installation is complete, restart the WEC server.
-
Copy the
krb5.keytab
file created in the previous section from your domain controller toC:\Program Files\nxlog
on your WEC server. -
Open
C:\ProgramData\MIT\Kerberos5\krb5.ini
with a text editor and add the following configuration:[libdefaults] default_realm = EXAMPLE.COM default_keytab_name = "C:\Program Files\nxlog\krb5.keytab" default_tkt_enctypes = aes256-cts-hmac-sha1-96 default_tgs_enctypes = aes256-cts-hmac-sha1-96 default_ccache_name = c:\Windows\Temp\krbcache forwardable = true [domain_realm] .EXAMPLE.com = EXAMPLE.COM EXAMPLE.com = EXAMPLE.COM [realms] EXAMPLE.COM = { kdc = dc.EXAMPLE.com:88 admin_server = dc.EXAMPLE.com }
Change
dc.EXAMPLE.com
according to the name of your DNS Server. -
Open command prompt and execute the following commands:
> cd C:\Program Files\MIT\Kerberos\bin > klist -e -k -t "C:\Program Files\nxlog\krb5.keytab"
The output:
Keytab name: FILE:C:\Program Files\nxlog\krb5.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 15 01/01/70 00:00:00 http/wecserver.EXAMPLE.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96) 16 01/01/70 00:00:00 hosts/wecserver.EXAMPLE.com@EXAMPLE.COM (aes256-cts-hmac-sha1-96)
-
Execute the following command to test that the authentication with Active Directory is working successfully when using the keytab. If the configuration is correct, a ticket-granting ticket (TGT) will be created and cached.
> kinit -kt "C:\Program Files\nxlog\krb5.keytab" http/wecserver.EXAMPLE.com@EXAMPLE.COM
-
Verify the ticket was obtained:
> klist
The output should be similar to:
Ticket cache: API:krb5cc Default principal: http/wecserver.EXAMPLE.com@EXAMPLE.COM Valid starting Expires Service principal 03/18/23 00:07:14 03/18/23 10:07:14 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 03/19/23 00:07:13
-
Configure and restart NXLog. See the configuration example below.
Configure the WEF Client
In this example, WEF is enabled on a single Windows node by editing the local policy of the server. To enable WEF on multiple nodes, a group policy on the domain should be configured.
-
Run
gpedit.msc
and go to Computer Configuration > Administrative Templates > Windows Components > Event Forwarding. -
Open and enable the Configure target Subscription Manager setting.
-
Click Show… beside the SubscriptionManagers option.
-
Type into the Value field:
Server=http://wecserver.example.com:8080/wsman,Refresh=30
. -
Open command prompt and run
gpupdate /force
. -
At this point, the WinRM service on the Windows client should connect to NXLog. If LogConnections is set to
TRUE
a connection attempt should be logged innxlog.log
and you should soon start seeing events arriving.
The Refresh interval represents the time in seconds and on production
systems should be set to a higher value (e.g. Refresh=1200 ).
When the Windows client is set to reconnect too frequently it will result in
a lot of connection/disconnection messages in nxlog.log .
|
Using HTTPS certificate-based authentication
To set up Windows Event Forwarding over HTTPS the following steps are required:
-
X509 certificate generation using either OpenSSL or the Windows certificate manager,
-
configuration of the NXLog im_wseventing module,
-
configuration of Windows Remote Management (WinRM) on each Windows client.
These steps are covered in greater detail below.
We will refer to the host running NXLog with the im_wseventing
module as WEC server.
We will use the name WEF client when referring to the Windows hosts
sending the logs using WEF.
|
Certificate requirements:
-
The WEF client certificate must have the
X509 v3 Extended Key Usage: TLS Web Client Authentication
extension. -
The WEC server certificate must have the
X509 v3 Extended Key Usage: TLS Web Server Authentication
extension. -
Make sure that the intended purpose of the certificates are set to
Client Authentication
andServer Authentication
respectively.
If these extended key usage attributes are not set you will likely encounter an error when trying to configure WEF and the connection to the server will fail. |
Generate SSL certificates
This section provides information on generating self-signed certificates for your WEF setup. For production, using certificates issued by a third-party certificate authority (CA) is more secure.
Generating certificates with OpenSSL
To use the Windows Certificates console, skip to the next section.
The following scripts are available in our public git repository to facilitate generating certificates with OpenSSL:
-
genca.sh
- generates the CA certificate and private key. -
gencert-client.sh
- generates the WEF client certificate and exports it together with the CA in PFX format to be imported into the Windows certificate store. -
gencert-server.sh
- generates the WEC server certificate to be used by theim_wseventing
module.
Refer to the readme file for instructions on the scripts.
When generating the certificates please ensure that the CN in the WEC server certificate subject matches the reverse DNS name.
Otherwise, you may get errors in the Microsoft Windows/Event-ForwardingPlugin/Operational event log saying The SSL certificate contains a common name (CN) that does not match the hostname.
|
If you are using an intermediary CA please make sure that the ca-cert.pem
file contains—in correct order—the public part of every issuer’s certificate.
The easiest way to achieve this is to 'cat' the pem
certificates together.
For more complex requirements follow this guide on how to set up a CA and generate certificates with OpenSSL.
Generating certificates with Windows Certificates
For more information on creating certificates on Windows see Request Certificates by Using the Certificate Request Wizard on Microsoft Learn.
Make sure to create the certificates with the required extensions as noted above.
Once you issue the certificates, you will need to export the WEC server certificate in PFX format.
The PFX must contain the private key.
The password may be omitted.
The PFX file can then be converted to the PEM format required by im_wseventing using openssl
:
openssl pkcs12 -in server.pfx -nocerts -nodes -out server-key.pem
openssl pkcs12 -in server.pfx -nokeys -nodes -out server-cert.pem
You will also need to export the CA certificate (without the private key) in the same way and convert it into ca-cert.pem
.
Configure the im_wseventing module
You will need to use server-key.pem
, server-cert.pem
and ca-cert.pem
for the HTTPSCertKeyFile
, HTTPSCertFile
, and HTTPSCAFile
respectively.
Optionally you can use the QueryXML option to filter on specific channels or events.
See the configuration example below for how your nxlog.conf
should look.
Once the configuration is complete you may start the nxlog
service.
Configure WinRM and WEF
Configure WinRM on the client for certificate-based authentication and enable Windows Event Forwarding. These steps can be automated for ease of deployment across multiple clients. See the example PowerShell script in our public git repository.
Install, configure, and enable Windows Remote Management (WinRM) on each WEF client:
-
Make sure the Windows Remote Management (WS-Management) service is installed, running, and set to Automatic startup type.
-
If WinRM is not already installed, see these instructions on MSDN: Installation and Configuration for Windows Remote Management.
-
Check that the proper client authentication method (Certificate) is enabled for WinRM. Issue the following command:
winrm get winrm/config/Client/Auth
You should see the following output:
Auth Basic = false Digest = true Kerberos = true Negotiate = true Certificate = true CredSSP = true [Source="GPO"]
If Certificate authentication is set to false, it should be enabled with the following command:
winrm set winrm/config/client/auth @{Certificate="true"}
Windows Remoting does not support event forwarding over unsecured transport (such as HTTP). Therefore it is recommended to disable the Basic authentication:
winrm set winrm/config/client/auth @{Basic="false"}
-
Import the CA and WEF client certificates if you used OpenSSL to generate these.
-
Open the Certificate MMC snap-in for the Local Computer (
certlm.msc
). -
Right click on Trusted Root Certification Authorities and select All Tasks > Import…. Import the
client.pfx
file. Enter the private key password if set and make sure the Include all extended properties check-box is selected. -
Repeat the previous step to import the certificate to the Personal certificate store.
After importing is complete, expand Personal > Certificates and double-click on the WEF client certificate to check if the full certificate chain is available and trusted.
-
-
Grant the NetworkService account the proper permissions to access the WEF client certificate by using the Windows HTTP Services Certificate Configuration Tool (WinHttpCertCfg.exe) and check that the NetworkService account has access to the private key file of the client authentication certificate by running the following command:
winhttpcertcfg -l -c LOCAL_MACHINE\my -s <certificate subject name>
Where
LOCAL\my
refers to thePersonal
certificate store of theLocal Computer
and<certificate subject name>
is theCN
value that was specified when creating the WEF client certificate. For example:winhttpcertcfg -l -c LOCAL_MACHINE\my -s winclient.example.com
If NetworkService is not listed in the output, grant it permissions by running the following command:
winhttpcertcfg -g -c LOCAL_MACHINE\my -s <certificate subject name> -a NetworkService
-
To forward Security events, the
Network Service
account needs to be given access to the Secuity Event Log. Refer to the section on [forwarding-events-from-the-security-log].
Next, configure the policy on the WEF client to enable event forwarding. In this example, WEF is enabled on a single Windows node by editing the local policy of the server. To enable WEF on multiple nodes, a group policy on the domain should be configured.
-
Run the Group Policy MMC snap-in (
gpedit.msc
) and go to Computer Configuration > Administrative Templates > Windows Components > Event Forwarding. -
Right-click the Configure target Subscription Manager setting and select Properties. Enable the setting, and under Options click the Show button to add a server address.
-
Add at least one setting that specifies the NXLog WEC server. The SubscriptionManager properties window contains a Help pane that describes the syntax for the setting. If you have used the
gencert-server.sh
script the subscription manager string should be in the following format:Server=HTTPS://<FQDN of im_wseventing>:<port>/wsman/,Refresh=<Refresh interval in seconds>, IssuerCA=<certificate authority certificate thumbprint>
An example would be as follows:
Server=HTTPS://nxlogserver.example.com:5985/wsman/,Refresh=14400,IssuerCA=57F5048548A6A983C3A14DA80E0626E4A462FC04
To find the IssuerCA fingerprint, open Certificates MMC snap-in, expand Local Computer > Personal > Certificates and open the CA certificate. Copy the
Thumbprint
from theDetails
tab. Please make sure to eliminate spaces and the invisible non-breaking space that may be present before the first character of the thumbprint on Windows 2008. -
After the SubscriptionManager setting has been added, ensure the policy is applied by running:
gpupdate /force
-
At this point, the WinRM service on the WEF client should connect to NXLog. If LogConnections is set to
TRUE
a connection attempt should be logged innxlog.log
and you should soon start seeing events arriving.
Forwarding events from the Security log
In adherence to C2-level Security, access to audit data of security-related events is limited to authorized administrators. WinRM runs as a network service and by default does not have access to the Security log, therefore it will not be able to forward Security events.
To give it access to the Security log on Windows 2008 and newer the Network Service
account should be added to the Event Log Readers
group.
To give it access on older versions of Windows:
-
Open Group Policy Editor by running
gpedit.msc
. -
Go to Computer Configuration > Administrative Templates > Windows Components > Event Log Service > Security.
-
Right click on the Configure Log Access and click on Edit
-
Enable the policy and under Log Access enter:
O:BAG:SYD:(A;;0xf0005;;;SY)(A;;0x5;;;BA)(A;;0x1;;;S-1-5-32-573)(A;;0x1;;;NS)
-
Run
gpupdate /force
to apply changes.
Troubleshooting
WEF if not easy to configure and there may be many things that can go wrong. To troubleshoot WEF, check the Windows Event Log under the following channels:
-
Applications and Services Logs/Microsoft/Windows/Event-ForwardingPlugin
-
Applications and Services Logs/Microsoft/Windows/Windows Remote Management
-
Applications and Services Logs/Microsoft/Windows/CAPI2
The Operational
log for these channels is enabled by default. The Analytic
and Debug
logs may also prove useful and can be enabled from the Windows
Event Viewer as follows:
-
From the menu click on View and select Show Analytic and Debug Logs.
-
Under each of these channels, click on the Debug log and in the Actions pane select Enable Log.
-
Repeat the previous step for the Analytic log when available.
Unfortunately the diagnostic messages in the Windows Event Log may not always be helpful and in some cases may even be missing. In such cases searching for the error code on the internet may point to more useful information. This section provides troubleshooting tips for commonly encountered issues.
The SSL certificate name does not match the hostname
The WEF client fails to connect to the WEC server and the Microsoft/Windows/Event-ForwardingPlugin/Operational
Event Log contains the following error:
The SSL certificate contains a common name (CN) that does not match the hostname.
This error occurs when the CN in the WEC server certificate does not match the reverse DNS. Run nslookup <server_ip>
on the client and verify that the correct name for the server is returned.
You can use the openssl
tool on the WEC server to verify that the subject CN matches the same name by using the following command:
openssl x509 -in server-cert.pem -text -noout
If the FQDN and the reverse DNS of the WEC server are not properly set up,
the WinRM service may fail to connect to the CRL URL to download
the revocation list. In that case an error messages will be logged under
Applications and Services Logs/Microsoft/Windows/CAPI2
as follows:
<Result value="80092013">The revocation function was unable to check
revocation because the revocation server was offline.</Result>
The WEF client fails to forward events to the Subscription Manger
If a secure connection cannot be established between the WEF client and the
WEC server, the Microsoft/Windows/Event-ForwardingPlugin/Operational
Event
Log may contain the following error:
The forwarder is having a problem communicating with the subscription manager
at address https://nxlogserver.example.com:5985/wsman/.
Error code is <error_code> and the Error Message is <error_message>.
Note that the error message can sometimes be empty.
When using HTTPS, this event is logged with Error code 2150858882
when a certificate matching the specified IssuerCA thumbprint is not found.
Verify that the thumbprint specified does not have any invisible characters as explained in this Microsoft support article.
Additionally, check that the thumbprint specified is for the CA certificate and not the WEF client, and that it can be found under Local Computer > Personal in the Certificates MMC snap-in.
When using Kerberos, this event may be logged with Error code 2150858999
and the following message:
The WinRM client cannot process the request.
It cannot determine the content of the HTTP response from the destination computer.
This error can occur when authentication fails on the WEC server side and
NXLog responds with a 400 Bad Request
. In such case the NXLog
log file contains the following error:
ERROR [im_wseventing|wsevents] GSS-API error gss_accept_sec_context - type: major code: 851968, msg: Unspecified GSS failure. Minor code may provide more information
ERROR [im_wseventing|wsevents] GSS-API error gss_accept_sec_context - type: minor code: -1765328203, msg: Key table entry not found
ERROR [im_wseventing|wsevents] [im_wseventing.c:1898/im_wseventing_create_kerb_auth_response()] Internal error
Carefully revise the steps required to Configure the WEF Client.
In particular, make sure that the hostname on the WEC server is resolved to the same hostname as specified in the keytab principal.
Run nslookup <server_ip>
on the WEC server and verify that the name returned is correct.
For example if the principal in the keytab is specified as:
http/linux.example.com@EXAMPLE.COM
Then the name returned by nslookup
should be linux.example.com
.
This error can also occur if the user that NXLog is running under is not
able to read the keytab file. Check the access permissions of
/etc/krb5.keytab
and verify that they allow at least Read
access to the
user. By default NXLog runs with the nxlog
user. If the keytab file
was manually copied, the access permissions may have been changed.
Configuring im_wseventlog to log connections
By default, the module does not log connection attempts which would otherwise be
useful for troubleshooting purposes. This can be turned on with the
LogConnections directive.
Note that the Windows Event Forwarding service may disconnect during the TLS
handshake with the following message logged in nxlog.log
when LogConnections
is enabled. This is normal as long as there is another connection attempt right
after the disconnection.
INFO connection accepted from 10.2.0.161:49381
ERROR im_wseventing got disconnected during SSL handshake
INFO connection accepted from 10.2.0.161:49381
See the Microsoft documentation on Windows Event Forwarding to a workgroup Collector Server for further instructions and troubleshooting tips.
Configuration
The im_wseventing module accepts the following directives in addition to the common module directives. The Address and ListenAddr directives are required.
- Address
-
This mandatory directive accepts a URL address. This address is sent to the WEF client to notify it where the events should be sent. For example,
Address https://nxlogserver.example.com:5985/wsman
.
- ListenAddr
-
This mandatory directive specifies the address of the interface where the module should listen for WEF client connections. Normally the any address
0.0.0.0
is used.
- AddPrefix
-
If this boolean directive is set to
TRUE
, names of fields parsed from the<EventData>
portion of the event XML will be prefixed withEventData.
. For example,$EventData.SubjectUserName
will be added to the event record instead of$SubjectUserName
. The same applies to<UserData>
. This directive defaults toFALSE
: field names will not be prefixed.
- CaptureEventXML
-
This boolean directive defines whether the module should store raw XML-formatted event data. If set to
TRUE
, the module stores raw XML data in the $EventXML field. By default, the value is set toFALSE
: the$EventXML
field is not added to the event record.
- ConnectionRetry
-
This optional directive specifies the reconnection interval. The default value is
PT60.0S
(60 seconds).
- ConnectionRetryTotal
-
This optional directive specifies the maximum number of reconnection attempts. The default is 5 attempts. If the WEF client exceeds the retry count it will consider the subscription to be stale and will not attempt to reconnect.
- Expires
-
This optional directive can be used to specify a duration after which the subscription will expire, or an absolute time when the subscription will expire. By default, the subscription will never expire.
- GssDllName
-
This optional directive specifies the name of the GSSAPI provider DLL on Windows. It is not available on Linux. The default value is
gssapi64.dll
, which is compatible with MIT Kerberos. If the module logs a "Failed to load gssapi64.dll" error, it means the DLL was not found in the search path. Ensure that you reboot the system after installing Kerberos. Alternatively, if you installed a different GSSAPI provider, you must specify the correct DLL name. For example, in the case of Heimdal, you need to set it togssapi.dll
.
- HeartBeats
-
Heartbeats are dummy events that do not appear in the output. These are used by the WEF client to signal that logging is still functional if no events are generated during the specified time period. The default heartbeat value of
PT3600.000S
may be overridden with this optional directive.
- HTTPSAllowUntrusted
-
This boolean directive specifies that the remote connection should be allowed without certificate verification. If set to
TRUE
the remote will be able to connect with an unknown or self-signed certificate. The default value isFALSE
: all HTTPS connections must present a trusted certificate.
- HTTPSCADir
-
This specifies the path to a directory containing certificate authority (CA) certificates, which will be used to check the certificate of the remote HTTPS client. The certificate filenames in this directory must be in the OpenSSL hashed format. A remote’s self-signed certificate (which is not signed by a CA) can also be trusted by including a copy of the certificate in this directory.
- HTTPSCAFile
-
This specifies the path of the certificate authority (CA) certificate, which will be used to check the certificate of the remote HTTPS client. To trust a self-signed certificate presented by the remote (which is not signed by a CA), provide that certificate instead.
- HTTPSCAThumbprint
-
This optional directive specifies the certificate thumbprint of the certificate authority (CA), which is used to look up the CA certificate from the Windows certificate store. The hexadecimal fingerprint string can be copied straight from Windows Certificate Manager (certmgr.msc), whitespaces are automatically removed. This directive is only supported on Windows. This directive and the HTTPSCADir and HTTPSCAFile directives are mutually exclusive.
- HTTPSCertFile
-
This specifies the path of the certificate file to be used for the HTTPS handshake.
- HTTPSCertKeyFile
-
This specifies the path of the certificate key file to be used for the HTTPS handshake.
- HTTPSCertThumbprint
-
This optional directive specifies the certificate thumbprint to be used for the SSL handshake. The hexadecimal fingerprint string can be copied straight from Windows Certificate Manager (certmgr.msc), whitespaces are automatically removed. This directive is only supported on Windows. This directive and the HTTPSCertFile and HTTPSCertKeyFile directives are mutually exclusive.
The private key associated with the certificate must be exportable.
-
If you generate the certificate request using Windows Certificate Manager, enable the Make private key exportable option from the certificate properties.
-
If you import the certificate with the Windows Certificate Import Wizard, make sure that the Mark this key as exportable option is enabled.
-
If you migrate the certificate and associated private key from one Windows machine to another, select Yes, export the private key when exporting from the source machine.
-
- HTTPSCRLDir
-
This specifies the path to a directory containing certificate revocation lists (CRLs), which will be consulted when checking the certificate of the remote HTTPS client. The certificate filenames in this directory must be in the OpenSSL hashed format.
- HTTPSCRLFile
-
This specifies the path of the certificate revocation list (CRL) which will be consulted when checking the certificate of the remote HTTPS client.
- HTTPSKeyPass
-
With this directive, a password can be supplied for the certificate key file defined in HTTPSCertKeyFile. This directive is not needed for passwordless private keys.
- HTTPSSSLCipher
-
This optional directive can be used to set the permitted SSL cipher list, overriding the default. Use the format described in the ciphers(1ssl) man page.
- HTTPSSSLCiphersuites
-
This optional directive can be used to set the permitted cipher list for TLSv1.3. Use the same format as in the HTTPSSSLCipher directive. Refer to the OpenSSL documentation for a list of valid TLS v1.3 cipher suites. The default value is:
TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
- HTTPSSSLCompression
-
This boolean directive allows you to enable data compression when sending data over the network. The compression mechanism is based on the zlib compression library. If the directive is not specified, it defaults to
FALSE
(the compression is disabled).Some Linux packages (for example, Debian) use the OpenSSL library provided by the OS and may not support the zlib compression mechanism. The module will emit a warning on startup if the compression support is missing. The generic deb/rpm packages are bundled with a zlib-enabled libssl library.
- HTTPSSSLProtocol
-
This directive can be used to set the allowed SSL/TLS protocol(s). It takes a comma-separated list of values which can be any of the following:
SSLv2
,SSLv3
,TLSv1
,TLSv1.1
,TLSv1.2
andTLSv1.3
. By default, theTLSv1.2
andTLSv1.3
protocols are allowed. Note that the OpenSSL library shipped by Linux distributions may not supportSSLv2
andSSLv3
, and these will not work even if enabled with this directive.
- LogConnections
-
This boolean directive can be used to turn on logging of connections. Since WEF connections can be quite frequent and excessive it could generate a lot of noise. On the other hand it can be useful to troubleshoot clients. This is disabled by default.
- MaxElements
-
This optional directive specifies the maximum number of event records to be batched by the client. If this is not specified the default value is decided by the client.
- MaxEnvelopeSize
-
This optional directive can be used to set a limit on the size of the allowed responses, in bytes. The default size is 153600 bytes. Event records exceeding this size will be dropped by the client and replaced by a drop notification.
- MaxTime
-
This optional directive specifies the maximum amount of time allowed to elapse for the WEF client to batch events. The default value is
PT30.000S
(30 seconds).
- Port
-
This specifies the port on which the module will listen for incoming connections. The default is port 5985.
- Query
-
This directive specifies the query for pulling only specific Event Log sources. See the MSDN documentation about Event Selection. Note that this directive requires a single-line parameter, so multi-line query XML should be specified using line continuation:
Query <QueryList> \ <Query Id='1'> \ <Select Path='Security'>*[System/Level=4]</Select> \ </Query> \ </QueryList>
- QueryXML
-
This directive is the same as the Query directive above, except it can be used as a block. Multi-line XML queries can be used without line continuation, and the XML Query can be directly copied from Event Viewer.
<QueryXML> <QueryList> <!-- XML-style comments can span multiple lines in QueryXML blocks like this. --> <Query Id='1'> <Select Path='Security'>*[System/Level=4]</Select> </Query> </QueryList> </QueryXML>
Commenting with the # mark does not work within multi-line Query directives or QueryXML blocks.
In this case, use XML-style comments <!-- --> as shown in the example above.
Failure to follow this syntax for comments within queries will render the module instance useless.
Since NXLog does not parse the content of QueryXML blocks, this behavior is expected.
|
- SubscriptionName
-
The default value of
NXLog Subscription
may be overridden with this optional directive. This name will appear in the WEF client logs.
Fields
The following fields are used by im_wseventing.
The actual fields generated will vary depending on the particular event’s source data.
$raw_event
(type: string)-
A string containing the $EventID, $EventType, $EventTime, $Hostname, and $Message from the event.
$ActivityID
(type: string)-
A globally unique identifier for the current activity.
$Channel
(type: string)-
The Channel of the event source (for example,
Security
orApplication
).
$EventID
(type: integer)-
The event ID specific to the event source.
$EventTime
(type: datetime)-
The timestamp that indicates when the event was logged.
$EventType
(type: string)-
The type of the event, which is a string describing the severity. This is translated to its string representation. Possible values are:
CRITICAL
,ERROR
,AUDIT_FAILURE
,AUDIT_SUCCESS
,INFO
,WARNING
, andVERBOSE
.
$EventXML
(type: string)-
The raw event data in XML format. This field is available if the module’s CaptureEventXML directive is set to TRUE.
$ExecutionProcessID
(type: integer)-
The ID identifying the process that generated the event.
$ExecutionThreadID
(type: integer)-
The ID identifying the thread that generated the event.
$Hostname
(type: string)-
The name of the computer that generated the event.
$Keywords
(type: string)-
The keywords used to classify the event, as a hexadecimal number.
$Level
(type: string)-
The level of the event as a string, resolved from $LevelValue. Possible values include:
Success
,Information
,Warning
,Error
,Audit Success
, andAudit Failure
.
$LevelValue
(type: integer)-
The level of the event.
$Message
(type: string)-
The message from the event.
$MessageID
(type: string)-
The unique identifier of the message.
$Opcode
(type: string)-
The Opcode string resolved from OpcodeValue.
$OpcodeValue
(type: integer)-
The Opcode number of the event as in EvtSystemOpcode.
$param1
(type: string)-
Additional event-specific data (
$param1
,$param2
, and so on).
$ProviderGuid
(type: string)-
The globally unique identifier of the event’s provider. This corresponds to the name of the provider in the $SourceName field.
$RecordNumber
(type: integer)-
The number of the event record.
$Severity
(type: string)-
The normalized severity name of the event. See $SeverityValue.
$SeverityValue
(type: integer)-
The normalized severity number of the event, mapped as follows.
Event Log Severity Normalized Severity 0/Audit Success
2/INFO
0/Audit Failure
4/ERROR
1/Critical
5/CRITICAL
2/Error
4/ERROR
3/Warning
3/WARNING
4/Information
2/INFO
5/Verbose
1/DEBUG
$SourceName
(type: string)-
The event source which produced the event.
$Task
(type: string)-
The task defined in the event.
$UserData
(type: string)-
Event-specific data. This field is mutually exclusive with $EventData.
$UserID
(type: string)-
The Security Identifier (SID) of the account associated with the event.
$Version
(type: integer)-
The version number of the event.
Examples
This configuration enables a WEC server on Linux. It will listen for logs on port 80 and use Kerberos authentication. See Configuring Kerberos on Linux above for more information.
<Input wseventing>
Module im_wseventing
Address http://LINUX.EXAMPLE.COM:80/wsman
ListenAddr 0.0.0.0
Port 80
SubscriptionName test
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Application">*</Select>
<Select Path="Security">*</Select>
<Select Path="Setup">*</Select>
<Select Path="System">*</Select>
<Select Path="ForwardedEvents">*</Select>
<Select Path="Windows PowerShell">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
This configuration enables a WEC server on Windows. It will listen for logs on port 8080 and use Kerberos authentication. The two setenv global directives are required on Windows to specify the Kerberos keytab path and disable the replay cache. See Configuring Kerberos on Windows above for more information.
setenv KRB5_KTNAME=%INSTALLDIR%/krb5.keytab
setenv KRB5RCACHETYPE=none
<Input wseventing>
Module im_wseventing
Address http://wecserver.EXAMPLE.COM:8080/wsman
ListenAddr 0.0.0.0
Port 8080
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Application">*</Select>
<Select Path="Security">*</Select>
<Select Path="System">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
This configuration receives Windows logs over HTTPS.
Two queries are specified, the first for hostnames matching foo*
and the second for all other hostnames.
<Input wseventing>
Module im_wseventing
ListenAddr 0.0.0.0
Port 5985
Address https://linux.example.com:5985/wsman
HTTPSCertFile %CERTDIR%/server-cert.pem
HTTPSCertKeyFile %CERTDIR%/server-key.pem
HTTPSCAFile %CERTDIR%/ca.pem
<QueryXML>
<QueryList>
<Computer>foo*</Computer>
<Query Id="0" Path="Application">
<Select Path="Application">*</Select>
</Query>
</QueryList>
</QueryXML>
<QueryXML>
<QueryList>
<Query Id="0" Path="Application">
<Select Path="Application">*</Select>
<Select Path="Microsoft-Windows-Winsock-AFD/Operational">*</Select>
<Select Path="Microsoft-Windows-Wired-AutoConfig/Operational">*</Select>
<Select Path="Microsoft-Windows-Wordpad/Admin">*</Select>
<Select Path="Windows PowerShell">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>