F5 BIG-IP
F5 BIG-IP appliances can send logs to a remote syslog destination via TCP or UDP. TCP is a more reliable protocol and is recommended for sending logs over the network. On the other hand, UDP does not guarantee data delivery; therefore, logs may get lost or truncated, especially in high-volume logging environments.
Several BIG-IP sub-systems write traffic and audit logs to different files. Below is an example of Local Traffic Management (LTM) logs reporting pool members being up or down.
Aug 08 16:50:12 l-lb1 notice mcpd[7660]: 01070639:5: Pool /Common/q-qa-pool member /Common/q-qa1:25 session status forced disabled.
Aug 08 16:51:33 l-lb1 notice mcpd[7660]: 01070639:5: Pool /Common/q-qa-pool member /Common/q-qa1:25 session status enabled.
Furthermore, the following is a sample from an audit log file.
Aug 08 16:43:41 l-lb1 notice httpd[3064]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start="Mon Aug 08 16:43:41 2022".
Aug 08 17:10:33 l-lb1 notice httpd[1181]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start="Mon Aug 08 16:43:41 2022" end="Mon Aug 08 17:10:33 2022".
Refer to K16197: Reviewing BIG-IP log files for more information on F5 logs. In addition, you can find details on configuring BIG-IP logging for your version at the F5 Knowledge Center.
Collecting BIG-IP logs via TCP
The BIG-IP web interface does not provide a way to configure an external TCP syslog destination, so you must configure it via the command line.
-
Configure NXLog Agentto receive syslog messages via TCP; see Receiving BIG-IP logs via TCP below. Restart NXLog Agent.
-
Ensure all your BIG-IP devices can communicate with NXLog Agent. Depending on your setup, you may need to configure a new route or default gateway.
-
Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.
-
Remove any existing BIG-IP syslog configuration.
# tmsh list sys syslog include # tmsh modify sys syslog include none
-
Configure a new remote syslog destination. Replace
SYSLOG_IP
andPORT
with the IP address and port of NXLog Agent andLEVEL
with the logging level you require.# tmsh modify sys syslog include "destination remote_server {tcp(\"SYSLOG_IP\" port (PORT));};filter f_alllogs { level (LEVEL...emerg);}; log { source(local); filter(f_alllogs); destination(remote_server);};"
For example, this command redirects informational and higher-level logs (from
info
toemerg
) to an NXLog Agentat192.168.100.85
listening on TCP port 1514.# tmsh modify sys syslog include "destination remote_server { tcp(\"192.168.100.85\" port (1514));}; filter f_alllogs {level (info...emerg);}; log {source(local); filter(f_alllogs); destination(remote_server);};"
Once you execute this command, BIG-IP will stop local logging and start forwarding all appliance logs to the remote destination. -
In case of a High Availability (HA) group, synchronize the configuration changes to the other units.
# tmsh run cm config-sync to-group GROUP_NAME
Once the configuration synchronizes to all group members, each member will send logs with its hostname and IP address. In the event of a failover, logging will continue from both units regardless of which one is currently active.
This configuration uses the im_tcp input module to listen for logs on TCP port 1514.
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input tcp_listen>
Module im_tcp
ListenAddr 0.0.0.0:1514
<Exec>
parse_syslog(); (1)
to_json(); (2)
</Exec>
</Input>
1 | Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module. |
2 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"MessageSourceAddress": "192.168.100.95",
"EventReceivedTime": "2022-08-08T10:52:21.423899+03:00",
"SourceModuleName": "tcp_listen",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 16,
"SyslogFacility": "LOCAL0",
"SyslogSeverityValue": 5,
"SyslogSeverity": "NOTICE",
"SeverityValue": 2,
"Severity": "INFO",
"Hostname": "localhost.localdomain",
"EventTime": "2022-08-08T10:51:49.000000+03:00",
"SourceName": "tmsh",
"ProcessID": 20306,
"Message": "01420002:5: AUDIT - pid=20306 user=root folder=/Common module=(tmos)# status=[Command OK] cmd_data=list sys syslog include"
}
{
"MessageSourceAddress": "192.168.100.95",
"EventReceivedTime": "2022-08-08T10:52:33.796531+03:00",
"SourceModuleName": "tcp_listen",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 3,
"SyslogFacility": "DAEMON",
"SyslogSeverityValue": 6,
"SyslogSeverity": "INFO",
"SeverityValue": 2,
"Severity": "INFO",
"Hostname": "localhost.localdomain",
"EventTime": "2022-08-08T10:52:01.000000+03:00",
"SourceName": "systemd",
"ProcessID": 1,
"Message": "Started Session 4277 of user root."
}
NXLog Agentcan also be configured to parse log records, such as those containing key-value pairs, into structured data.
This configuration uses the im_tcp input module to listen for logs on TCP port 1514. It then uses regular expressions and the xm_kvp module to parse log records into structured data.
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension kvp>
Module xm_kvp
KVPDelimiter " "
KVDelimiter =
EscapeChar \\
</Extension>
<Input tcp_listen>
Module im_tcp
ListenAddr 0.0.0.0:1514
<Exec>
parse_syslog(); (1)
if $Message =~ /^([a-z]*) ([a-zA-Z]*)(.*)$/ (2)
{
$F5MsgLevel = $1;
$F5Process = $2;
$F5Message = $3;
if $F5Message =~ /^\[[0-9]*\]: ([0-9]*):([0-9]+): (.*)$/ (3)
{
$F5MsgID = $1;
$F5MsgSev = $2;
$F5Message = $3;
if $F5Message =~ /RAW: ([a-z]*)\(([a-z_]*)\): ([a-zA-Z]+=.+)/ (4)
{
$F5Process = $1;
$F5Module = $2;
kvp->parse_kvp($3); (5)
}
}
delete($F5Message); (6)
}
to_json(); (7)
</Exec>
</Input>
1 | Parses log records using the parse_syslog() procedure of the xm_syslog module, which creates the $Message field. |
2 | Uses a regular expression to extract the message level and process name from the $Message field. |
3 | Parses the message further to extract the message ID and severity. |
4 | Parses the message further to extract the process, module, and list of key-value pairs. |
5 | Parses the key-value pairs into fields using the parse_kvp() procedure of the xm_kvp module. |
6 | Deletes the redundant $F5Message field with the delete() procedure. |
7 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"MessageSourceAddress": "192.168.6.91",
"EventReceivedTime": "2022-08-08 00:08:43",
"SourceModuleName": "tcp_listen",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 10,
"SyslogFacility": "AUTHPRIV",
"SyslogSeverityValue": 5,
"SyslogSeverity": "NOTICE",
"SeverityValue": 2,
"Severity": "INFO",
"Hostname": "l-lb1",
"Message": "notice httpd[5320]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start=\"Mon Aug 08 00:07:59 2022\".",
"EventTime": "2022-08-08 00:07:59",
"F5MsgLevel": "notice",
"F5MsgID": "01070417",
"F5MsgSev": "5",
"F5Process": "httpd",
"F5Module": "mod_auth_pam",
"user": "john(john)",
"partition": "[All]",
"level": "Administrator",
"tty": "/usr/bin/tmsh",
"host": "192.168.9.78",
"attempts": "1",
"start": "Mon Aug 08 00:07:59 2022"
}
Collecting BIG-IP logs via UDP
When reliable delivery is not a concern, or you need to retain a local copy of the logs on each appliance, you can configure BIG-IP to send logs to a remote syslog destination via UDP.
-
Configure NXLog Agentto receive syslog messages via UDP; see Receiving BIG-IP logs via UDP below. Restart NXLog Agent.
-
Ensure all your BIG-IP devices can communicate with the NXLog Agent. Depending on your setup, you may need to configure a new route or default gateway.
-
Follow the instructions to configure UDP logging via the command line or web interface.
This configuration uses the im_udp input module to listen for logs on UDP port 514.
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input udp_listen>
Module im_udp
ListenAddr 0.0.0.0:514
<Exec>
parse_syslog(); (1)
to_json(); (2)
</Exec>
</Input>
1 | Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module. |
2 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"MessageSourceAddress": "192.168.100.86",
"EventReceivedTime": "2022-08-08T16:05:02.330953+03:00",
"SourceModuleName": "udp_listen",
"SourceModuleType": "im_udp",
"SyslogFacilityValue": 3,
"SyslogFacility": "DAEMON",
"SyslogSeverityValue": 6,
"SyslogSeverity": "INFO",
"SeverityValue": 2,
"Severity": "INFO",
"Hostname": "bigip",
"EventTime": "2022-08-08T16:05:01.000000+03:00",
"Message": "info systemd[1]: Started Session 2059 of user root."
}
Configuring UDP logging via the command line
-
Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.
-
Configure a new remote syslog destination. Replace
SYSLOG_IP
andPORT
with the IP address and port of NXLog Agent.# tmsh modify sys syslog remote-servers add { nxlog { host SYSLOG_IP remote-port PORT } }
For example, this command redirects logs to an NXLog Agentat
192.168.100.85
listening on UDP port 514.# tmsh modify sys syslog remote-servers add { nxlog { host 192.168.100.85 remote-port 514 } }
-
In case of a High Availability (HA) group, synchronize the configuration changes to the other units.
# tmsh run cm config-sync to-group GROUP_NAME
Once the configuration synchronizes to all group members, each member will send logs with its hostname and IP address. In the event of a failover, logging will continue from both units regardless of which one is currently active.
Configuring UDP logging via the web interface
If you haven’t already, first enable the BIG-IP web interface:
-
Execute the following command to open the configuration utility and follow the instructions. The utility automatically detects the current machine’s IP address and makes all the appropriate changes if required.
# config eth0
-
The first time you log in to the BIG-IP instance, use the default username
root
and passworddefault
. You will be prompted to change the root password. -
Execute the following command to change the password of the
admin
user.# tmsh modify auth user admin prompt-for-password
-
Configure the BIG-IP HTTP server to listen on port 80. Replace
BIG_IP_SERVER_IP
with the IP address of the BIG-IP server.# tmsh modify sys httpd include "Listen BIG_IP_SERVER_IP:80"
Once you enable access to the web interface, follow these steps to configure remote logging:
-
Open
https://BIG_IP_SERVER_IP
and log in with your account. In case of a High Availability (HA) group, ensure you are logged in to the active unit. You should see ONLINE (ACTIVE) in the top left corner. -
Navigate to System > Logs > Configuration > Remote Logging.
-
Type the Remote IP and Remote Port of NXLog Agentand click Add, followed by Update.
-
In case of a High Availability (HA) group, synchronize the configuration changes to the other units:
-
Click the yellow Changes Pending in the top left corner.
-
Select the active device marked as (Self).
-
Enable the Sync Device to Group option and click Sync.
Once the configuration synchronizes to all group members, each member will send logs with its hostname and IP address. In the event of a failover, logging will continue from both units regardless of which one is currently active.
-
Using SNMP traps
BIG-IP devices are capable of sending SNMP traps and contain predefined SNMP traps. They also support creating user-defined traps. You can find additional information about SNMP support for your version in the F5 Knowledge Center, under the Monitoring BIG-IP System Traffic with SNMP section in the External Monitoring of BIG-IP Systems: Implementations guide.
BIG-IP systems include Management Information Base (MIB) files stored on the device. Refer to K13322: Overview of BIG-IP MIB files for more information.
-
Configure NXLog Agentto receive SNMP traps. See Receiving SNMP traps below.
-
Ensure all your BIG-IP devices can communicate with NXLog Agent. Depending on your setup, you may need to configure a new route or default gateway.
-
Follow the instructions to configure SNMP traps via the command line or web interface.
This configuration uses the im_udp input module to listen for logs on UDP port 162 and the xm_snmp module to process SNMP traps.
<Extension json>
Module xm_json
</Extension>
<Extension snmp>
Module xm_snmp
MIBDir /usr/share/mibs/bigip
# The following section is required for SNMPv3
#<User snmp_user>
# AuthProto sha1
# AuthPasswd q1w2e3r4
# EncryptPasswd q1w2e3r4
# EncryptProto aes
#</User>
</Extension>
<Input udp_listen>
Module im_udp
ListenAddr 0.0.0.0:162
InputType snmp (1)
Exec to_json(); (2)
</Input>
1 | Sets the InputType directive to the xm_snmp module instance name. |
2 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"SNMP.CommunityString": "nxlog",
"SNMP.RequestID": 684814253,
"EventTime": "2022-08-08T15:06:31.765042+03:00",
"SeverityValue": 2,
"Severity": "INFO",
"iso.3.6.1.2.1.1.3.0": 5111365,
"iso.3.6.1.6.3.1.1.4.1.0": "iso.3.6.1.4.1.3375.2.4.0.3",
"iso.3.6.1.6.3.1.1.4.3.0": "iso.3.6.1.4.1.3375.2.4",
"MessageSourceAddress": "192.168.100.86",
"EventReceivedTime": "2022-08-08T15:06:31.765101+03:00",
"SourceModuleName": "udp_listen",
"SourceModuleType": "im_udp"
}
Configuring SNMP traps via the command line
-
Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.
-
Enable the predefined traps as required.
# tmsh modify sys snmp bigip-traps enabled # tmsh modify sys snmp agent-trap enabled # tmsh modify sys snmp auth-trap enabled
-
Create an SNMP user (SNMPv3 only). Replace
USERNAME
andPASSWORD
with your desired values.# tmsh modify sys snmp users add { USERNAME { username USERNAME auth-protocol sha privacy-protocol aes auth-password PASSWORD privacy-password PASSWORD } }
For example, this command creates a user
snmp_user
that uses MD5 authentication and encrypts the SNMPv3 data payload with AES. Refer to K13625: Overview of SNMPv3 agent access for more information.# tmsh modify sys snmp users add { snmp_user { username snmp_user auth-protocol md5 privacy-protocol aes auth-password q1w2e3r4 privacy-password q1w2e3r4 } }
-
Configure the remote SNMP destination. Replace
NAME
,COMMUNITY
,IP_ADDRESS
, andPORT
with the appropriate values. ReplaceNETWORK
withother
ormanagement
if traps go out of the management interface.# tmsh modify sys snmp traps add { NAME { community COMMUNITY host IP_ADDRESS port PORT network NETWORK } }
For example, this command enables forwarding SNMPv2 traps to
192.168.100.85
.# tmsh modify sys snmp traps add { 192_168_6_143 { community nxlog host 192.168.100.85 port 162 network other } }
When using SNMPv3, you need to specify additional parameters, including security-level, auth-protocol, auth-password, privacy-protocol, and privacy-password. For example, this command enables forwarding SNMPv3 traps to
192.168.100.85
using SHA and AES.# tmsh modify sys snmp traps add { nxlog { version 3 host 192.168.100.85 port 162 network other security-level auth-privacy security-name snmp_user auth-protocol sha auth-password q1w2e3r4 privacy-protocol aes privacy-password q1w2e3r4 } }
If the BIG-IP configuration has been migrated or cloned, SNMPv3 may not work because the SNMP engine ID is not unique. If you encounter this problem, reset the engine ID as described in K6821: SNMP v3 fails if the SNMP engine ID is not unique. -
In case of a High Availability (HA) group, synchronize the configuration changes to the other units.
# tmsh run cm config-sync to-group GROUP_NAME
Configuring SNMP traps via the web interface
-
Open
https://BIG_IP_SERVER_IP
and log in with your account. ReplaceBIG_IP_SERVER_IP
with the IP address of your device. In case of a High Availability (HA) group, ensure you are logged in to the active unit. You should see ONLINE (ACTIVE) in the top left corner. -
Navigate to System > SNMP > Traps. Select the required SNMP events and click Update.
-
Create an SNMP user (SNMPv3 only). Navigate to System > SNMP > Agent > Access (v3). Click Create and specify the user name, authentication type and password, privacy protocol and password, and access type. Specify an OID value to limit access to certain OIDs or use
.1
to allow full access. -
Navigate to System > SNMP > Traps > Destination and click Create. Specify the SNMP version, community name, destination IP address, port, and network, then click Finished.
SNMPv3 requires additional parameters, including security, authentication, and privacy settings.
-
In case of a High Availability (HA) group, synchronize the configuration changes to the other units.
-
Click the yellow Changes Pending in the top left corner.
-
Select the active device marked as (Self).
-
Enable the Sync Device to Group option and click Sync.
Once the configuration synchronizes to all group members, each member will send logs with its hostname and IP address. In the event of a failover, logging will continue from both units regardless of which one is currently active.
-
F5 high-speed logging
F5 BIG-IP devices support High-Speed Logging (HSL). This protocol sends as much data as the remote destination can accept. Combined with load balancing, BIG-IP makes it possible to configure multiple NXLog Agents in one of the available load balancing methods.
BIG-IP can send its internal logs via HSL in addition to logs passing through the device. Since the load balancer usually resides on the network’s edge and all web traffic passes through it, collecting logs from BIG-IP may be easier and faster than processing individual web server logs.
HSL supports forwarding logs via TCP or UDP. TCP guarantees reliable delivery. However, when load balancing is enabled, BIG-IP will reuse existing TCP connections to each node to reduce overhead. As a result, load distribution between members may be less accurate.
The steps below have been tested with BIG-IP v12. |
To configure HSL, you must create a node for each NXLog Agentand add it to a pool:
-
Connect to the BIG-IP device via SSH.
-
Create a node for each NXLog Agent. Replace
NODE_NAME
andIP_ADDRESS
with the relevant values.# tmsh create ltm node NODE_NAME { address IP_ADDRESS session user-enabled }
For example, this command creates a node
nxlog1
.# tmsh create ltm node nxlog1 { address 192.168.100.85 session user-enabled }
-
Create a pool with all the nodes.
# tmsh create ltm pool POOL_NAME { members add { NODE1:PORT { address IP_ADDRESS1 }} NODE2:PORT { address IP_ADDRESS2 }} monitor PROTOCOL
For example, this command creates a pool named
nxlog-pool
with the node created in the previous step.# tmsh create ltm pool nxlog-pool { members add { nxlog1:1514 { address 192.168.100.85 }} monitor tcp }
Forwarding internal BIG-IP logs to a High-Speed Logging pool
Follow these steps to send internal BIG-IP logs to the pool created above.
-
Connect to the BIG-IP device via SSH.
-
Create a remote logging destination. Replace
DESTINATION_NAME
with a name for the destination,POOL_NAME
with the name you specified when you created the pool,DISTRIBUTION
with one of the distribution options listed below, andPROTOCOL
withtcp
orudp
. Distribution options include:- Adaptive
-
Sends to one of the pool members until this member cannot process logs at the required rate or the connection is lost.
- Balanced
-
Uses the load balancing method configured on the pool and selects a new member each time it sends data.
- Replicated
-
Sends each log to all members of the pool.
# tmsh create sys log-config destination remote-high-speed-log DESTINATION_NAME pool-name POOL_NAME distribution DISTRIBUTION protocol PROTOCOL
-
Create a log publisher. Replace
PUBLISHER_NAME
with a name for the publisher andDESTINATION_NAME
with the destination name used in the previous step.# tmsh create sys log-config publisher PUBLISHER_NAME destinations add {DESTINATION_NAME}
-
Create a log filter. Replace
FILTER_NAME
with a name for the filter,LEVEL
with the required logging level between Emergency and Debugging,PUBLISHER_NAME
with the name used in the previous step, andSOURCE
with a specific BIG-IP process orall
to forward all logs.# tmsh create sys log-config filter FILTER_NAME level LOGGING_LEVEL publisher PUBLISHER source SOURCE
For example, the following commands will send all internal logs to NXLog Agentpool via TCP.
# tmsh create sys log-config destination remote-high-speed-log nxlog-hsl pool-name nxlog distribution adaptive protocol tcp # tmsh create sys log-config publisher bigip-local-logs destinations add {nxlog-hsl} # tmsh create sys log-config filter bigip-all-local-logs level debug publisher bigip-local-logs source all
Forwarding network logs to a High-Speed Logging pool
Follow these steps on each virtual server to configure BIG-IP to forward network logs.
-
Configure NXLog Agentto receive syslog messages via TCP; see Receiving logs from BIG-IPs below. Restart NXLog Agent.
-
Create a logging profile for requests. In most cases, it is enough to log requests only. However, you can also configure a profile to log responses and errors. In the example below:
- LOGGING_PROFILE_NAME
-
A name for the new logging profile.
- PROTOCOL
-
mds-tcp
ormds-udp
, depending on whether NXLog Agentis listening for TCP or UDP connections. - POOL
-
The name of the pool created above.
- TEMPLATE
-
A list of HTTP request fields to log. See Configuring the BIG-IP System as a DHCP Relay Agent in the BIG-IP Local Traffic Manager: Implementations guide for more information.
# tmsh create ltm profile request-log LOGGING_PROFILE_NAME { request-log-protocol PROTOCOL request-log-pool POOL_NAME request-logging enabled request-log-template "TEMPLATE" }
-
Assign the logging profile to a virtual server. The same logging profile can be assigned to multiple virtual servers. Replace
VIRTUAL_SERVER_NAME
with the virtual server name andLOGGING_PROFILE_NAME
with the profile name used in the previous step.# tmsh create ltm virtual VIRTUAL_SERVER_NAME {profiles add {LOGGING_PROFILE_NAME {}}}
For example, the following commands configure logging to NXLog Agentpool via TCP.
# tmsh create ltm profile request-log LOGGING_PROFILE_NAME { request-log-protocol mds-tcp request-log-pool POOL_NAME request-logging enabled request-log-template "client $CLIENT_IP:$CLIENT_PORT request $HTTP_REQUEST server $SERVER_IP:$SERVER_PORT status $HTTP_STATUS" } # tmsh modify ltm virtual VIRTUAL_SERVER_NAME {profiles add {LOGGING_PROFILE_NAME {}}}
This configuration uses the im_tcp input module to listen for logs on TCP port 1514.
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input tcp_listen>
Module im_tcp
ListenAddr 0.0.0.0:1514
<Exec>
parse_syslog(); (1)
to_json(); (2)
</Exec>
</Input>
1 | Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module. |
2 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"MessageSourceAddress": "192.168.6.91",
"EventReceivedTime": "2017-05-10 19:16:43",
"SourceModuleName": "tcp_listen",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 1,
"SyslogFacility": "USER",
"SyslogSeverityValue": 5,
"SyslogSeverity": "NOTICE",
"SeverityValue": 2,
"Severity": "INFO",
"EventTime": "2017-05-10 19:16:43",
"Hostname": "192.168.6.91",
"Message": "client 192.168.9.78:63717 request GET /cmedia/img/icons/mime/mime-unknown.png?v170509919 HTTP/1.1 server 192.168.6.101:80 status "
}
NXLog Agentcan also be configured to parse log records according to the sequence of fields specified in the template.
This configuration uses the im_tcp input module to listen for logs on TCP port 1514 and a regular expression to parse records into structured data.
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input tcp_listen>
Module im_tcp
ListenAddr 0.0.0.0:1514
<Exec>
parse_syslog(); (1)
if $Message =~ /^client (.*) request (.*) server (.*) status (.*)$/ (2)
{
$HTTP_Client = $1;
$HTTP_Request = $2;
$HTTP_Server = $3;
$HTTP_Status = $4;
delete($Message); (3)
}
to_json(); (4)
</Exec>
</Input>
1 | Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module, which creates the $Message field. |
2 | Uses a regular expression to extract the HTTP client, request, server, and status from the $Message field. |
3 | Deletes the redundant $Message field with the delete() procedure. |
4 | Converts log records to JSON format using the to_json() procedure of the xm_json module. |
{
"MessageSourceAddress": "192.168.6.91",
"EventReceivedTime": "2017-05-10 20:06:24",
"SourceModuleName": "tcp_listen",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 1,
"SyslogFacility": "USER",
"SyslogSeverityValue": 5,
"SyslogSeverity": "NOTICE",
"SeverityValue": 2,
"Severity": "INFO",
"EventTime": "2017-05-10 20:06:24",
"Hostname": "192.168.6.91",
"HTTP_Client": "192.168.9.78:65275",
"HTTP_Request": "GET /?disabledcookies=true HTTP/1.1",
"HTTP_Server": "192.168.6.100:80",
"HTTP_Status": ""
}
Using BIG-IP load balancing for an NXLog Agentpool
You can use BIG-IP as a load balancer to distribute traffic between multiple NXLog Agents. To accomplish this, you must create a virtual server that accepts syslog messages and forwards them to the pool.
For example, this command creates a virtual server named nx-virtual-serv
that listens on IP address 192.168.100.85
TCP port 1514
and forwards logs to the pool nxlog-pool
.
# tmsh create ltm virtual nx-virtual-serv { destination 192.168.100.85:1514 mask 255.255.255.255 pool nxlog-pool profiles add { tcp{} } }
After you execute this command, configure your log sources to forward syslog messages to the specified IP address.