Monitoring Fabric and Production Networks

This chapter describes viewing information about the DANZ Monitoring Fabric (DMF) and connected production networks.

Monitor DMF Interfaces

Select the Menu control to view statistics for the specific interface and select Monitor Stats. The system displays the following dialog box.
Figure 1. Monitor Interface Stats

This window displays statistics for up to four selected interfaces and provides a line graph (sparkline) that indicates changes in packet rate or bandwidth utilization. The auto-refresh rate for these statistics is ten seconds—mouse over the sparkline to view the range of values represented. To clear statistics for an interface, select the Menu control and select Clear Stats.

To view statistics for multiple interfaces, enable the checkbox to the left of the Menu control for each interface, select the Menu control in table, and select Monitor Selected Stats.
Figure 2. Monitoring > Interfaces

To view the interfaces assigned a specific role, use the Monitoring > Interfaces command and select the Filter, Delivery, or Service sub-option from the menu.

Viewing Oversubscription Statistics

To view peak bit rate statistics used to monitor bandwidth utilization due to oversubscription, select the Menu in the Interfaces table, select Show/Hide Columns, and enable the Peak Bit Rate checkbox on the dialog box that appears.

After enabling the Peak Bit Rate column, a column appears in the Interfaces table that indicates the relative bandwidth utilization of each interface. When using less than 50% of the bandwidth, the bar appears in green; 50-75% changes the bar to yellow, and over 75% switches the bar color to red.

To display statistics for a specific interface, select Monitor Stats from the Menu control to the left of the row.

To reset the statistics counters, select Clear Stats from the Menu control.

Note: DANZ Monitoring Fabric (DMF) Controllers generate SNMP traps for link saturation and packet loss. For more information, please refer to the DMF Deployment Guide - SNMP Trap Generation for Packet Drops and Link Saturation chapter.

View Fabric-Connected Devices

To view a display of the devices connected to the Controller, select Fabric > Connected Devices from the main menu. The system displays the following screen.
Figure 3. Connected Devices

The Switch Interfaces table displays the unique devices connected to each out-of-band filter or delivery switch.It lists each interface's MAC address (Chassis ID) on every device connected to the fabric as a separate device.

The Unique Device Names table lists all unique device names with a count of interfaces in parentheses. Selecting a link in a row in this list filters the contents of the Switch Interfaces table.

To view a display of the devices discovered by the Controller through the Link Aggregation Control Protocol (LACP), select Fabric > Connected LACP from the main menu.

The system displays the following screen.
Figure 4. Connected LACP

This page displays the devices discovered by the Controller through LACP.

Capacity Planning - Current versus Verified Scale

Before DMF release 8.9, the DMF fabric lacked direct visibility into current scale metrics relative to verified scale limits. This feature exposes real-time scale data against verified benchmarks via REST APIs, GUI, and CLI commands. The verified scale represents the capacity tested under reference conditions. The feature is compatible with all platforms and no new configuration is necessary; the feature uses existing settings to provide immediate visibility into the scale.

 

Navigate to Monitoring and select Fabric Scale.

Figure 5. Fabric Scale

The Fabric Scale dashboard contains two main sections, Fabric Scale Metrics and Category Scale Metrics.

Figure 6. Fabric Scale Dashboard

Fabric Scale Metrics

The first section provides a high-level overview of Fabric Scale Metrics.

Figure 7. Fabric Scale Metrics

All fabric-level scale metrics are displayed visually and compared against verified limits, with meter-range indicators highlighting utilization levels. Each meter represents the relationship between the current usage and the verified supported limit.

Hovering over a meter displays an information card showing the calculated values for:

Figure 8. Meter Rollover Data
  • Current
  • Available
  • Verified

In addition to the meters, Verified Scale summary cards display for Per-Switch Scale Metrics, Per-Policy Scale Metrics, and Per-Address Group Scale Metrics, providing a quick reference to the supported scale limits for these categories.

Category-Level Scale Metrics

The second section, Category Scale Metrics, provides a detailed, categorized view of scale metrics at a more granular level.

Figure 9. Category Scale Metrics

This section is organized by:

  • Switches
  • Policies
  • Address Groups

Based on the selection, DMF displays a corresponding table allowing review of validated scale usage details for the selected category.

Color Coding

The meter color reflects the current scale utilization:

  • Blue: ≤ 70%
  • Orange: ≤ 95%
  • Red: > 95%

Display Mode

Select the Utili. (%) or the Count drop-down to view the metrics as percentages or counts. The dashboard updates with the chosen view.

Figure 10. Display Mode - Percent versus Count

Using the Command Line Interface

Monitor Interface Configuration

To display the currently configured interfaces, enter the show interface-names command, as shown in the following example.
Ctrl-2> show interface-names
            
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Filter Interface(s)  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF      Switch              IF Name   Dir State Speed  VLAN Tag Analytics Ip address Connected Device
-|-----------|-------------------|---------|---|-----|------|--------|---------|----------|----------------|
1 Lab-traffic Arista-7050SX3-T3X5 ethernet7 rx  up    10Gbps 0        True
            
~ Delivery Interface(s) ~
None.
            
            
~ Service Interface(s) ~
None.
           
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Recorder Fabric Interface(s)  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF          Switch              IF Name    Dir           State Speed  Connected Device
-|---------------|-------------------|----------|-------------|-----|------|------------------|
1 PR-NewHW-Intf   Arista-7050SX3-T3X5 ethernet25 bidirectional up    25Gbps PR-NewHW   ens1f0
2 RMA-CNrail-intf Arista-7050SX3-T3X5 ethernet35 bidirectional up    25Gbps RMA-CNrail ens1f0
          
Note: The name is used when configuring a policy.
To display a summary of the current DANZ Monitoring Fabric (DMF) configuration, enter the show fabric command, as in the following example.
controller-1# show fabric
~~~~~~~~~~~~~~~~~~~~~ Aggregate Network State ~~~~~~~~~~~~~~~~~~~~~
Number of switches : 3
Inport masking : False
Start time : 2018-03-16 15:42:43.322000 PDT
Number of unmanaged services : 0
Filter efficiency : 0:1
Number of switches with service interfaces : 0
Total delivery traffic (bps) : 168bps
Number of managed service instances : 2
Number of service interfaces : 0
Match mode : l3-l4-offset-match
Number of delivery interfaces : 6
Max pre-service BW (bps) : 20Gbps
Auto VLAN mode : push-per-policy
Number of switches with delivery interfaces : 2
Number of managed devices : 1
Uptime : 5 hours, 4 minutes
Total ingress traffic (bps) : 160bps
Max filter BW (bps) : 221Gbps
Auto Delivery Interface Strip VLAN : True
Number of core interfaces : 12
Overlap : True
Number of switches with filter interfaces : 2
State : Enabled
Max delivery BW (bps) : 231Gbps
Total pre-service traffic (bps) : 200bps
Track hosts : True
Number of filter interfaces : 5
Number of active policies : 2
Number of policies : 5
~~~~~~~~~~~~~ Aggregate Interface Statistics ~~~~~~~~~~~~~
# Interface Type     Dir Packets Bytes  Pkt Rate Bit Rate
-|------------------|---|-------|------|--------|--------|
1 Filter Interface   rx  2444    455611 0        160bps
2 Delivery Interface tx  4050    421227 0        168bps
---------------------example truncated--------------------
controller-1#

View Switch Configuration

To verify the switch interface configuration, enter the show topology command, as shown in the following example.
controller> show topology
~~~~~~~~~~~~~~~~~~~~~ Filter Interface(s) ~~~~~~~~~~~~~~~~~~~~~
# DMF IF    Switch IF   Name     state speed   Connected Device
-|---------|-----------|--------|-----|-------|----------------|
1 f1        filter-sw-1 s11-eth1 up    10 Gbps
2 f2        filter-sw-1 s11-eth2 up    10 Gbps
~~~~~~~~~~~~~~~~~~~~ Delivery Interface(s) ~~~~~~~~~~~~~~~~~~~~
# DMF IF    Switch IF   Name     state speed   Connected Device
-|---------|-----------|--------|-----|-------|----------------|
1 d1        filter-sw-2 s12-eth1 up    10 Gbps
2 d2        filter-sw-2 s12-eth2 up    10 Gbps
~~~~~~~~~~~~~~~~~~~~~~~~~ Service Interface(s) ~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF           Switch IF Name    Dir state speed   Connected Device
-|----------------|---------|-------|---|-----|-------|----------------|
1 post-serv-intf-1 core-sw-1 s9-eth2     up    10 Gbps
2 pre-serv-intf-1  core-sw-1 s9-eth1     up    10 Gbps
~~~~~~~~~~~~~~~~~~~~~~~~ Core Interface(s) ~~~~~~~~~~~~~~~~~~~~~~~~
# Src Switch    Src IF   Src Speed Dst Switch    Dst IF   Dst Speed
-|-------------|--------|---------|-------------|--------|---------|
1 core-sw-3     s13-eth2 10 Gbps   delivery-sw-2 s15-eth3 10 Gbps
2 core-sw-3     s13-eth1 10 Gbps   delivery-sw-1 s14-eth3 10 Gbps
3 filter-sw-1   s11-eth3 10 Gbps   core-sw-2     s10-eth1 10 Gbps
4 core-sw-2     s10-eth1 10 Gbps   filter-sw-1   s11-eth3 10 Gbps
5 delivery-sw-2 s15-eth3 10 Gbps   core-sw-3     s13-eth2 10 Gbps
6 core-sw-2     s10-eth2 10 Gbps   filter-sw-2   s12-eth3 10 Gbps
7 filter-sw-2   s12-eth3 10 Gbps   core-sw-2     s10-eth2 10 Gbps
8 delivery-sw-1 s14-eth3 10 Gbps   core-sw-3     s13-eth1 10 Gbps
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Statistics ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#  DMF IF    Switch IF     Role     State   Packets        Bytes  Pkt Rate Bit Rate
--|---------|-------------|--------|--------|-----|-------|------|--------|--------|
1 f1         filter-sw-1   s11-eth1 filter   up    0       0      0        -
2 f2         filter-sw-1   s11-eth2 filter   up    0       0      0        -
3 d1         filter-sw-2   s12-eth1 delivery up    8       600    0        -
4 d2         filter-sw-2   s12-eth2 delivery up    8       600    0        32 bps
6 -          core-sw-3     s13-eth1 core     up    3432    257400 0        32 bps
7 -          delivery-sw-2 s15-eth3 core     up    3431    257325 0        32 bps
8 -          delivery-sw-1 s14-eth3 core     up    3430    257250 0        32 bps
9 -          core-sw-2     s10-eth1 core     up    3429    257175 0        32 bps
10 -         filter-sw-1   s11-eth3 core     up    3431    257325 0        32 bps
11 -         core-sw-3     s13-eth2 core     up    3432    257400 0        32 bps
12 -         filter-sw-2   s12-eth3 core     up    3429    257175 0        32 bps

View Connected Devices and LAGs

Some information on devices in the production network, discovered using LLDP and CDP, can be seen using the show connected-devices command. The data helps determine if filter interfaces are connected to the intended production device.

The show connected-devicescommand from login mode displays the devices connected to the DANZ Monitoring Fabric (DMF). This command displays information about devices connected to DMF switch interfaces. DMF extracts the information from link-level protocol packets such as LLDP, CDP, and UDLD and ignores expired link-level data.

Users can see the most recent events related to particular connected devices via the CLI command show connected-devices history device_alias.

Connecting a DMF switch interface to a SPAN port may result in inaccurate information because some vendor devices mirror link-level packets to the SPAN port.

To display details about the link aggregation groups connected to the DMF switch interfaces use the show connected-lacp command. DMF extracts the information from LACP protocol packets and expired LACP information is ignored as illustrated in the following.

Viewing Information about a Connected Production Network

Once the monitoring fabric is set up and connected to packet feeds from the production network, DANZ Monitoring Fabric (DMF) starts to gather information about the production network. By default, DMF provides a view of all hosts in the production network visible from the filter interfaces. View this information on the GUI page under Monitoring > Host Tracker . As shown below, the output displays the host MAC address, IP address, when and on which filter interface traffic from the host was seen, and DHCP lease information. To display this information, enter the show tracked-hosts command, as shown in the following example.
# show tracked-hosts
#   IP Address      MAC Address                                              Host name                           Filter interfaces        VLANs   Last seen Extra info
---|---------------|--------------------------------------------------------|-----------------------------------|------------------------|-------|---------|
1   10.0.0.3        40:a6:d9:7c:9f:9f                                        Apple                               wireless-poe-1           0       1 hours
2   10.0.0.6        98:fe:94:1c:37:06                                        Apple                               wireless-poe-1           0       42 min
3   10.0.0.6        dc:2b:61:81:64:45                                        Apple                               wireless-poe-1           0       3 hours
4   10.0.0.7        20:c9:d0:48:f3:3d                                        Apple                               wireless-poe-1           0       2 hours
5   10.0.0.11       60:03:08:9b:4f:48                                        Apple                               wireless-poe-1           0       13 min
6   10.0.1.3        14:10:9f:e4:e6:bf                                        Apple                               wireless-poe-1           0       51 min
--------------------------------------------------------------------output truncated----------------------------------------------------------------------

DMF also tracks the DNS names of hosts by capturing and analyzing packets using several different protocols. To manage host-name tracking, from config-analytics mode, use the track command, which has the following syntax:

[no] track { arp | dns | dhcp | icmp }

For example, the following command enables tracking using DNS:
controller-1(config)# analytics
controller-1(config-analytics)# track dns
Note:DNS traffic will not be included in DMF policies when enabling DNS for tracking.
Exclude host tracking for a specific filter interface using the no-analytics option with the role command.
controller-1(config)# switch DMF-FILTER-SWITCH-1
controller-1(config-switch)# interface ethernet20
controller-1(config-switch-if)# role filter interface-name TAP-PORT-01 no-analytics

This command disables all host tracking on interface TAP-PORT-01.

Capacity Planning - Current versus Verified Scale

Before DMF release 8.9, the DMF fabric lacked direct visibility into current scale metrics relative to verified scale limits. This feature exposes real-time scale data against verified benchmarks via REST APIs, the GUI, and CLI commands. The verified scale represents the capacity tested under reference conditions. The feature is compatible with all platforms and no new configuration is necessary; the feature uses existing settings to provide immediate visibility into the scale.

Show Commands

The scale feature introduces new CLI commands under the existing show fabric command structure. These commands provide visibility into current fabric resource usage compared to verified scale across different categories.

  • show fabric scale - Display fabric-wide current scale and verified scale.
  • show fabric scale category - Display verified scale for specified category.
  • show fabric scale category identifier-Display current scale and verified scale for specified identifier.
  • show fabric scale category all-Display current scale and verified scale for all instances of the specified category.
  • show fabric scale category all metric-name - Display current scale values for the specified metric across all instances of the specified category, along with the metric's verified scale.
Important:
  • The category can be a switch, policy, or address-group.
  • An identifier is the specific name within the category (switch name, policy name, or address-group name).
  • A metric-name is the name of the metric.
dmf-controller-1> show fabric scale 
~~~~~~~~~~~~~~~ Fabric Scale Metrics  ~~~~~~~~~~~~~~~
#  Metric               Current Scale Verified Scale 
--|--------------------|-------------|--------------|
1  switches             3             150
2  filter-interfaces    1             1500
3  delivery-interfaces  5             1000
4  total-policies       1             4000
5  managed-services     2             40
6  service-nodes        1             5
7  ipv4-address-groups  1             170
8  ipv6-address-groups  1             50
9  max-overlap-policies 4             10
10 connected-devices    3             100
11 groups               3             500
12 users                1             500
Notes:
Verified scale represents capacity that has been tested under reference conditions.

 

dmf-controller-1> show fabric scale <tab>
address-group  switch         |               <cr>
policy         ;      
dmf-controller-1> show fabric scale switch 
~~~~~ Per-Switch Scale Metrics  ~~~~~
# Metric              Verified Scale 
-|-------------------|--------------|
1 filter-interfaces   128
2 delivery-interfaces 128
3 managed-services    40
Notes:
Verified scale represents capacity that has been tested under 
reference conditions.
dmf-controller-1> show fabric scale switch <tab>
all        core1      delivery1  filter1    ;          |          >           <cr>     
dmf-controller-1> show fabric scale switch core1
~~~~~~~~~~~~~~~~~~~~ Switch Scale Metrics  ~~~~~~~~~~~~~~~~~~~~
# Switch Name Metric              Current Scale Verified Scale 
-|-----------|-------------------|-------------|--------------|
1 core1       filter-interfaces   0             128
2 core1       delivery-interfaces 0             128
3 core1       managed-services    2             40
Notes:
Verified scale represents capacity that has been tested under reference conditions
dmf-controller-1> show fabric scale switch all
~~~~~~~~~~~~~~~~~~~~ Switch Scale Metrics  ~~~~~~~~~~~~~~~~~~~~
# Switch Name Metric              Current Scale Verified Scale 
-|-----------|-------------------|-------------|--------------|
1 core1       filter-interfaces   0             128
2 core1       delivery-interfaces 0             128
3 core1       managed-services    2             40
4 delivery1   filter-interfaces   0             128
5 delivery1   delivery-interfaces 3             128
6 delivery1   managed-services    0             40
7 filter1     filter-interfaces   1             128
8 filter1     delivery-interfaces 2             128
9 filter1     managed-services    0             40
Notes:
Verified scale represents capacity that has been tested under reference conditions.
dmf-controller-1> show fabric scale policy all
~~~~~~~~~~~~~~~~~~~ Policy Scale Metrics  ~~~~~~~~~~~~~~~~~~~
# Policy Name Metric            Current Scale Verified Scale 
-|-----------|-----------------|-------------|--------------|
1 slc         filter-interfaces 5             1000
2 slc         services-chained  4             4
Notes:
Verified scale represents capacity that has been tested under reference conditions.
dmf-controller-1> show fabric scale address-group all
~~~~~~~~~~~~~~~~~~ Address Group Scale Metrics ~~~~~~~~~~~~~~~~~~
# Address Group Name Metric         Current Scale Verified Scale 
-|------------------|--------------|-------------|--------------|
1 temp               ipv4-addresses 1             20000
2 v6                 ipv6-addresses 2             100
Notes:
Verified scale represents capacity that has been tested under reference conditions.
dmf-controller-1> show fabric scale switch all <tab>
delivery-interfaces  managed-services     |                     <cr>
filter-interfaces    ;                    >
dmf-controller-1> show fabric scale switch all delivery-interfaces 
Metric         : delivery-interfaces
Verified Scale : 128
~~~ Switch Scale Metrics ~~~
# Switch Name Current Scale 
-|-----------|-------------|
1 core1       0
2 delivery1   3
3 filter1     2
Notes:
Verified scale represents capacity that has been tested under reference conditions.

Syslog Messages - Scale

  • WARN
    • SCALE4000: Verified scale file not found at {file path}.
  • ERROR
    • SCALE7000: Unable to read verified scale file {file path} due to security violation.
    • SCALE7001: Failed to load verified scale from {file path}.
    • SCALE7002: Ignoring invalid category {file path}.