System Event Logging

Arista switches log Event Notifications using the Syslog protocol. By default, EOS logs the event notifications internally to the folder, /var/log/messages but can also display them on the console or log them to an external server. Configure severity levels and log message destinations using the CLI, and configure individual processes and protocols to adjust or limit the messages from them. Use the show logging command to obtain visibility into the buffered and persistent logs which contain critical system, process, and operational messages.

For a full list of Syslog messages, visit Arista Support. Once on the site, click Software Downloads, and then navigate to EOS > Active Releases > latest_release .

Click on the latest released version, and select Docs. Once expanded, select the latest_release-SysMsgGuide.pdf. The SysMsgGuide contains the latest version of the Syslog Messages in EOS.

System Event Logging Guidelines

Consider the following guidelines when configuring system logging on the switch:
  • Buffer Size - Configure the number of messages stored in the buffer using the logging buffered size command. By default, EOS retains a large, but volatile, buffer and discards messages upon reboot unless you configure persistent logging.
  • Severity - When you specify a severity level, EOS only processes messages with that severity and higher. When filtering with a severity level, EOS only displays messages with that severity and lower.
  • Real-time Output - To view messages as they occur during your terminal session, enable terminal monitoring with the command, terminal monitor.
  • Custom Prefixes - Configure custom prefixes for system log messages.

Severity Thresholds

Specifying a severity sets the minimum threshold of messages logged using an integer from 0 to 7, or a string from the following list:
  • 0 - Emergency: System unusable
  • 1 - Alert: Take immediate action
  • 2 - Critical: Critical conditions
  • 3 - Error: Error conditions
  • 4 - Warning: Warning conditions
  • 5 - Notice: Normal but significant conditions
  • 6 - Informational: Informational messages
  • 7 - Debug: Debug level messages

For example, if you configure a logging level of 6, EOS logs everything with levels 0-6.

Logging Locations

EOS logs system events in several locations depending on the nature of the log event. Logging occurs in the following locations and includes different types of events:
  • Buffer Logging
  • Console Logging
  • Persistent Logging
  • Monitor logging
  • Synchronous Logging
  • Trap Logging

Buffer Logging

Use logging buffered to retain logs in the buffer space on the switch. Set the syslog level as well as the buffer size. Arista Networks recommends a larger buffer size to retain messages for a week or longer.

Console Logging

Displays system notifications directly on the switch console or terminal session to provide real-time visibility into the health and operational status of the switch as events occur.

Persistent Logging

Enable persistent logging to write system logs to non-volatile flash and retain them after a switch reloads. However, utilizing persistent logging may fill up the flash when lots of log events occur on the switch.

To enable persistent logging with a maximum buffer size of 10,000 bytes, use the following command:
switch(config)# logging persistent 10000

Synchronous Logging

Synchronous logging ensures logging messages appear on the console but do not interrupt CLI output from command output. The log messages display after the CLI completes the command output.

The logging synchronous has the following format:
switch(config)# logging synchronous [ level severity | all]

level severity specify log messages of a lower severity level to log synchronously and specifies all log messages to log synchronously.

Trap Logging

Enable trap logging to send log messages to a remote server. Specifying a severity level logs only log messages with a severity at or above that level to the remote server. To add a remote server IPv4 address, 192.168.96.147, use the following commands:
switch(config)# logging host 192.168.96.147
switch(config)#
To add a fully qualified domain name (FQDN), mycompany.com, as the remote server, use the following command:
switch(config)# logging host mycompany.com
Use the following command to enable trap logging with severity 6:
switch(config)# logging trap system 4
The logging trap system command has the following parameters:
  • contain - Specify text contained in a log message.
  • facility - Specify one of 23 facilities, such as 1, for user-level messages.
  • severity - Specify a severity from 0 (emergencies) to 7 (debugging).
  • tag - Specify a tag that correlates with a program name.

Enabling System Logging

Use the following command to enable system logging on the switch:
switch(config)# logging on
switch(config)#

The no | defaultversion of the command disables system logging on the switch and removes the configuration from the running-config.

Configuring the Source Interface

After enabling a remote server to receive log messages, specify a local interface to derive the source IP address using the following syntax:
logging vrf vrf_name local-interface interface

Configuring Custom Prefixes for System Log Messages

Add custom prefixes to system log messages to create a format for syslog messages sent to syslog servers. The custom prefix can consist of any combination of underscores, numbers and upper and lower case letters, but does not allow other characters including whitespace.

Note: You cannot configure the parameters, hostname and prefix, at the same time. Configuring one automatically disables the other command.
Use the following command to add the prefix, router_1a, to syslog messages:
switch(config)# logging format prefix router_1a
switch(config)#

Configuring Active Standby Syslog Servers

EOS supports a standby server in addition to the existing logging mechanism for UDP syslog servers. Specify a logging group and then add remote syslog servers. The first configured syslog server becomes the active server and receives forwarded syslog messages when reachable through ICMP. The second configured host server becomes the backup server and receives syslog messages when the active syslog server becomes unreachable on the network.

The syslog server works with Connectivity Monitor to determine server liveliness by periodically sending ICMP probes at a configurable interval. Syslog servers configured in the active-standby mode start the Connectivity Monitor agent to probe the configured servers, and allows the syslog servers to work with the connectionless UCP protocol.

Configure syslog servers and related options in the Logging Group Configuration mode. Use the following commands to configure the logging group, Group_1 on vrf bluengreen, and host 172.16.21.23, with port 80:
switch(config)# logging group server Group_1
switch(config-logging-group-Group_1)# logging vrf bluengreen host 172.16.21.23 port 80
switch(config-logging-group-Group_1)#
Configure the second syslog server, 192.168.1.3:
switch(config-logging-group-Group_1)# host 192.168.1.3
switch(config-logging-group-Group_1)#

The host, 172.16.21.23, becomes the active server, and the host, 192.168.1.3, becomes the standby server on port 514, the default port.

Configure a local interface, if not already configured on the VRF, to derive the source IP address of ICMP probes and syslog messages. Any host configured on the VRF then use the interface, Ethernet1. Use the following commands to configure an interface, Ethernet1:
switch(config)# logging vrf bluengreen local-interface Ethernet1
switch(config)#

Configuring ICMP Probes

An ICMP probe determines the syslog server status and by default, sends one every 5 seconds. The probe consists of 5 ICMP echo packets, and if all of the packets fail, the syslog server has a down status. Use the following commands to configure sending an ICMP probe every 10 seconds:
switch(config-logging-group-Group_1)# probe interval 10
switch(config-logging-group-Group_1)#

The interval can be configured from 1 to 600 seconds.

Configuring Success Thresholds

When a syslog server becomes unavailable, sending a number of sequentially passing probes determines the health of the server. If configured to 5, then 5 successful probes must be received before returning the status to available. This becomes useful when the server has a high variability of liveliness and helps prevent erroneous state changes and syslogs. The threshold has a default value of 0 which means that a syslog server has a status of available as long as it receives one probe successfully.

Use the following commands to configure a success threshold of 5:
switch(config-logging-group-Group_1)# probe threshold success 5
switch(config-logging-group-Group_1)

Displaying Syslog Server Status

The show logging group server command displays the status of the syslog servers. It also indicates the primary and standby servers and the recipient of the syslog messages.
switch# show logging group server Group_1
switch(config)#show logging group server
Logging group group1
Logging to 172.16.21.23 port 80 vrf bluengreen via udp (primary), state: probing
Last available: n/a
Logging to 192.168.1.3 vrf default via udp (standby), state: active
Last available: 0:00:10 ago

The output displays the primary status as probing because the syslog server has no availability, and the standby server as active with the endpoint status of 10 seconds indicating the last time the server received syslog messages.

Displaying System Logging Information

To display information about the system logging configuration, use the show logging command:
switch# show logging
Syslog logging: enabled
Buffer logging: level notifications
Console logging: level debugging
Persistent logging: level debugging
Monitor logging: level debugging
Synchronous logging: disabled
Trap logging: level informational
    Logging to '192.168.93.147' port 6514 in VRF default via udp
    Logging to '192.168.96.147' port 514 in VRF default via udp
    Logging to '192.168.96.147' port 6514 in VRF default via udp
    Logging to '192.168.96.147' port 514 in VRF purple via udp
Sequence numbers: disabled
Syslog facility: local4
Hostname format: Hostname only
Repeat logging interval: disabled
Repeat messages: disabled
Root login logging: disabled

External configuration:
    active: 
    inactive: 

Facility                   Severity            Effective Severity
--------------------       -------------       ------------------
aaa                        debugging           debugging    
accounting                 debugging           debugging    
acl                        debugging           debugging    
agent                      debugging           debugging    
ale                        debugging           debugging    
arp                        debugging           debugging    
bfd                        debugging           debugging    
bgp                        debugging           debugging
<------>
vrf                        debugging           debugging    
vrrp                       debugging           debugging    
vxlan                      debugging           debugging    
ztp                        debugging           debugging

The output displays the level of logging for each type of logging location and the configuration on the switch. Each feature displays the severity and effective severity of the log messages.

Managing TCAM Capacity Warnings

Strata chipsets, present in the 7010, 7050X, 7060X, 7250X, 7260X, and 7300X series switches, provide event logging for the hardware capacity of TCAM tables on a per-slice basis, and trigger a capacity warning by default whenever any TCAM slice exceeds 90% capacity. As a result, default TCAM logging can generate high levels of syslog messages on these platforms. The hardware capacity alert table command can adjust the capacity levels for the warnings above the 90% default. In that case, this adjustment can be made per TCAM resource and per slice. The command can also disable TCAM hardware capacity messages at the level Warning and below for a given slice. To disable messages, set the threshold to 0 or use the no version of the command.

Examples
  • This command reduces hardware capacity Syslog warnings by increasing the capacity threshold to 99% for EFP table monitoring in slice 2. 
    switch(config)# hardware capacity alert table EFP feature Slice-2 threshold 99
  • This command reduces messages by disabling hardware capacity Syslog warnings entirely for the IFP table in slice 5. 
    switch(config)# hardware capacity alert table IFP feature Slice-5 threshold 0
  • This command reduces messages by disabling hardware capacity Syslog warnings entirely for the VFP table in all slices. 
    switch(config)# no hardware capacity alert table VFP
Note: Hardware capacity messages are user-configurable only at or below the “Warning” level. When depleting TCAM management software, it always sends “Error” messages to Syslog and affected features when all TCAM resources are used.

Layer 2 MAC Learn Failure Detection

A hash collision occurs when two or more distinct pieces of data map to the same entry or slot in the hardware table. It can happen when the hash function used to calculate the index for a MAC address results in the already occupied index and causes a failure of inserting the later MAC address to the hardware table.

Configuring Layer 2 MAC Learn Failure Detection

Use the following commands to enable system logging of MAC Learn Failures on the switch:
switch(config)# platform trident hardware mac-address-table collision-tracking
Use the no version of the command to disable the feature:
switch(config)# no platform trident hardware mac-address-table collision-tracking

Displaying Layer MAC Learn Failure Information

Use the following command to display missing MAC addresses in the hardware table:
switch# show platform trident l2 hardware mac-address-table missing

   Vlan  |   Mac Address         |   Present in SW?   |   Present in HW   | Missing in HW |
---------|-----------------------|--------------------|-------------------|---------------|
   2141  |   00:00:04:00:00:09   |         Y          |                   | Switchcard1/0 |
---------|-----------------------|--------------------|-------------------|---------------|
   2141  |   00:00:03:00:19:0c   |         Y          |                   | Switchcard1/0 |
---------|-----------------------|--------------------|-------------------|---------------|
                
Total entries missing in software:  0                                                    
                
Total entries missing in hardware:  2                                                    
Switchcard1/0:  2

Sample Syslog Message for L2 MAC Learn Failure

%ETH---MACADDRBANKFULL: Unable to program learned dynamic host table entry for MAC address MAC address in VLAN VLAN Id due to hardware resource exhaustion.