Integrating vCenter with DMF
This chapter describes integrating VMware vCenter with the DANZ Monitoring Fabric (DMF) and monitoring Virtual Machines (VM) in the vCenter.
Overview
The DANZ Monitoring Fabric (DMF) allows the integration and monitoring of VMs in a VMware vCenter cluster. After integrating a vCenter with the DMF fabric, use DMF policies to select different types of traffic from specific VMs and apply managed services, such as deduplication or header slicing, to the selected traffic.
Currently, DMF supports the following versions of VMware vCenter for monitoring:
- vCenter Server 7.0.0
- vCenter Server 8.0.0
The DANZ Monitoring Fabric provides two options to monitor a VMware vCenter cluster:
- Monitoring using span ports: This method monitors VMware vCenter clustering using a separate monitoring network. The advantage of this configuration is that it has no impact on the production network and has a minimal effect on compute node CPU performance. However, in this configuration, each compute node must have a spare NIC to monitor traffic.
The following figure illustrates the topology used for local SPAN configuration:
Figure 1. Mirroring on a Separate SPAN Physical NIC (SPAN) - Monitoring using ERPAN/L2GRE tunnels: Use Remote SPAN (ERSPAN) to monitor VMs running on the ESX hosts within a vCenter instance integrated with DMF. ERSPAN monitors traffic to and from VMs anywhere in the network and does not require a dedicated physical interface card on the ESX host. However, ERSPAN can affect network performance, especially when monitoring VMs connected to the DMF Controller over WAN links or production networks with high utilization.
Using SPAN to Monitor VMs
This section describes the configuration required to integrate the DANZ Monitoring Fabric (DMF) Controller with one or more vCenter instances and to monitor traffic from VMs connected to the VMware vCenter after integration.
The following figure illustrates the topology required to integrate a vCenter instance with the monitoring fabric and deliver the traffic selected by DMF policies to specified delivery ports connected to different monitoring tools.

When integrated with vCenter, the DMF Controller uses Link Layer Discovery Protocol (LLDP) to automatically identify the available filter interfaces connected to the vCenter instance.
Using ERSPAN to Monitor VMs

The procedure for deploying ERSPAN is similar to SPAN but requires an additional step to define the tunnel endpoints used on the DMF network to terminate the ERSPAN session.
Configuration Summary for vCenter Integration
The following procedure summarizes the high-level steps required to integrate the vCenter and monitor traffic to or from selected VMs:
Defining a Tunnel Endpoint
Predefine the tunnel endpoints for creating tunnels when monitoring VMware vCenter traffic using either the GUI or the CLI.
GUI Procedure
To manage tunnel endpoints in the GUI, select
.
This page lists the tunnel endpoints that are already configured and provides information about each endpoint.

- Name: Type a descriptive name for the endpoint.
- Switch: Select the DMF switch from the selection list for the configured endpoint interface.
- Interface: Select the interface from the selection list for the endpoint.
- Gateway: Type the address of the default gateway.
- IP Address: Type the endpoint IP address.
- Mask: Type the subnet mask for the endpoint.
Integrate a vCenter Instance

This page displays information about the vCenter instances integrated with DMF. To add a vCenter instance for integration with DMF, perform the following steps:
Using a vCenter Instance as the Traffic Source in a DMF Policy

Proceed to the
page.








View vCenter Configuration

DANZ Monitoring Fabric (DMF) displays the vCenter Info page.

- Hosts
- Virtual Switches
- Physical Connections
- Virtual Machines
- Network Host Connection Details


Integrating vCenter with DMF using Mirror Stack
DANZ Monitoring Fabric (DMF) vCenter integration supports mirroring from vCenter hosts using the default TCP/IP stack. However, this can result in traffic drops and affect production traffic since mirror traffic can conflict with production traffic. DMF vCenter integration with Mirror Stack provides the functionality to use the mirror TCP/IP stack for mirror sessions. Mirror stack in the ESXi host allows decoupling the traffic and keeps the production traffic unaffected.
vCenter configurations in DMF will use a mirror stack by default; however, if upgrading from previous DMF versions, the already configured vCenter will be set to use the default TCP/IP stack.
Platform Compatibility
- vCenter Server 7.0.x
- vCenter Server 8.0.x
vCenter Configuration
Perform the following steps to configure the mirror stack on vCenter.
Repeat the steps for each ESXi host containing VMs to be monitored.
Configuring DMF
To configure TCP/IP Stack, navigate to TCP/IP Stack. Default Stack and Mirror Stack are the options.
. While adding or editing a vCenter configuration, select the appropriate choice using
Refer to the CLI show
commands to view the tcp-ip-stack
configuration. In addition, use the show
fabric errors
and show fabric warnings
commands to troubleshoot and verify that everything is functioning as expected.
Limitations
- A port mirroring session remains on the original distributed virtual switch (DVS) when a VM migrates between DVSs.
- Port mirroring sessions will persist on the DVS if a VM is renamed in vCenter while being monitored by DMF.
- DMF cannot create a port mirroring session in the DVS if a conflicting session with the same VM exists in the DVS. This is not a limitation in vCenter 7.
- When using mirror stack configuration in DMF, mirror sessions may still be created on the DVS for the ESXi host that doesn’t have a mirror stack configuration. This will result in no traffic being mirrored from the VM.
- Auto-generated filter interfaces by vCenter integration should not be deleted from the policy. If they are deleted manually from the policy, they will not be automatically re-added.
- DMF cannot monitor VMkernel adapters.
Resources
- Create a Port Mirroring Session - https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-networking/GUID-68B5DD45-DD3F-4E9B-A6CD-BE97026A846A.html
- Encapsulated Remote Mirroring (L3) Source - https://kb.vmware.com/s/article/67973
Create Wildcard Tunnels for VMware vCenter Monitoring
The current implementation of VMware vCenter creates one tunnel interface from every ESXi host to DMF.
Using a wildcard tunnel on DMF for VMware vCenter reduces the number of tunnels created.
Platform Compatibility
This feature is only compatible with switches that support wildcard tunneling.
Use the DANZ Monitoring Fabric (DMF) GUI to create wildcard tunnels as outlined below.

Select the Menu icon.

Limitations
Select Broadcom® switch ASICs support wildcard tunnels; ensure your switch model supports this feature before configuring it for vCenter.
Please refer to the Platform Compatibility section for more information.
Minimum Permissions for Non-admin Users
For a non-admin user to add, remove, edit, or monitor a vCenter via the DANZ Monitoring Fabric (DMF), the privilege level assigned to the non-admin user is VSPAN operation. To assign VSPAN operation privileges to a user, perform the following steps:
Monitor vCenter Traffic by VM Names
Match VMware vCenter-specific information in the policy. Specifically, this feature matches traffic using VMware vCenter Virtual Machine (VM) names and requires DANZ Monitoring Fabric (DMF) vCenter integration.
Configure vCenter VM name matches under the DANZ Monitoring Fabric (DMF) policies match rules section. For example:
Limitations
- This feature only works with vCenter integration and a direct Switch Port Analyzer (SPAN) from a switch with ESXi traffic.
- VM interface IP addresses connected to dvs will only be added to policy matches.
- The system may use extra TCAM entries if the management network uses dvs.
- Vmkernal names cannot be matched in the policy.
- When a VM name with multiple vNICs (multiple IP addresses) matches the policy, a TCAM entry is added for all the IP addresses.
- VM Names cannot be matched with the MAC option in the policy.
- If the vCenter becomes disconnected, policies associated with the VM names may not get correct matches or traffic.
Using the Command Line Interface
Defining a Tunnel Endpoint
controller-1(config)# tunnel-endpoint name switch switch interface ip-address address mask
mask gateway address
controller-1(config)# tunnel-endpoint OSEP1 switch F-SWITCH-1 ethernet24 ip-address 172.27.1.1
mask 255.255.255.0 gateway 172.27.1.2
The IP address assigned to this endpoint is 172.27.1.1, and the next hop address for connecting to the vCenter via OSEP1 (using ERSPAN) is 172.27.1.2.
Integrate a vCenter Instance
Refer to the following topics to monitor VMs using Encapsulated Remote SPAN (ERSPAN) or Switch Port Analyzer (SPAN) on a locally connected vCenter instance and VMs on a second locally connected vCenter instance.
VMs using ERSPAN on a Locally Connected vCenter Instance
To configure the DANZ Monitoring Fabric Controller for monitoring VMs using ERSPAN on a locally connected vCenter instance, perform the following steps:
VMs using SPAN on a Locally Connected vCenter Instance
vc-1
that have physical connections to DMF switches become the filter interfaces. If new connections are made later (or existing connections are changed), policies will be recomputed with the new interfaces.VMs on a Second Locally Connected vCenter Instance
View vCenter Configuration
controller-1# show vcenter
#vCenter Name vCenter Host Name or IP Last vCenter Update Time Detail State vSphere Version
--|------------|-----------------------|------------------------------|----------------------------|---------------|
1vc-10-9-0-75 10.9.0.75 2017-09-0918:02:35.980000 PDTConnected and authenticated. 6.5.0
2vc-10-9-0-76 10.9.0.76 2017-09-0918:02:36.488000 PDTConnected and authenticated. 6.5.0
3vc-10-9-0-77 10.9.0.77 2017-09-0918:02:35.908000 PDTConnected and authenticated. 6.0.0
4vc-10-9-0-78 10.9.0.78 2017-09-0918:02:33.507000 PDTConnected and authenticated. 6.5.0
5vc-10-9-0-79 10.9.0.79 2017-09-0918:02:32.248000 PDTConnected and authenticated. 6.5.0
6vc-10-9-0-80 10.9.0.80 2017-09-0918:02:32.625000 PDTConnected and authenticated. 6.0.0
7vc-10-9-0-81 10.9.0.81 2017-09-0918:02:34.672000 PDTConnected and authenticated. 6.0.0
8vc-10-9-0-82 10.9.0.82 2017-09-0918:02:33.008000 PDTConnected and authenticated. 6.0.0
9vc-10-9-0-83 10.9.0.83 2017-09-0918:02:30.011000 PDTConnected and authenticated. 6.0.0
10 vc-10-9-0-84 10.9.0.84 2017-09-0918:02:33.024000 PDTConnected and authenticated. 6.5.0
11 vc-10-9-0-85 10.9.0.85 2017-09-0918:02:34.827000 PDTConnected and authenticated. 6.0.0
12 vc-10-9-0-86 10.9.0.86 2017-09-0918:02:35.164000 PDTConnected and authenticated. 6.0.0
13 vc-10-9-0-87 10.9.0.87 2017-09-0918:02:38.042000 PDTConnected and authenticated. 6.5.0
14 vc-10-9-0-88 10.9.0.88 2017-09-0918:02:37.212000 PDTConnected and authenticated. 6.0.0
15 vc-10-9-0-89 10.9.0.89 2017-09-0918:02:33.436000 PDTConnected and authenticated. 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75
#vCenter Name vCenter Host Name or IP Last vCenter Update Time Detail State vSphere Version
--|------------|-----------------------|------------------------------|----------------------------|---------------|
1vc-10-9-0-75 10.9.0.75 2017-09-0918:02:44.698000 PDTConnected and authenticated. 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75 detail
vCenter Name : vc-10-9-0-75
vCenter Host Name or IP : 10.9.0.75
Last vCenter Update Time : 2017-09-09 18:02:49.463000 PDT
Detail State : Connected and authenticated.
vSphere Version : 6.5.0
controller-1#
controller-1# show vcenter vc-10-9-0-75 error
vCenter Name : vc-10-9-0-75
vCenter Host Name or IP : 10.9.0.75
State : connected
Detail State : Connected and authenticated.
Detailed Error Info :
controller-1#
Integrating vCenter with DMF using Mirror Stack
tcp-ip-stack
option in the vCenter config. The default and recommended value is mirror-stack
.
dmf-controller-1(conf)# vcenter vc8
dmf-controller-1(config-vcenter)# tcp-ip-stack
default-stack mirror-stack
dmf-controller-1(config-vcenter)# tcp-ip-stack mirror-stack
Show Commands
tcp-ip-stack
configuration.
mirror-stack
is configured, it will only show when using the details
token.dmf-controller-1(config-vcenter)# show running-config vcenter vc8 details
! vcenter
vcenter vc8
default-tunnel-endpoint r34-lag-leaf1b
hashed-password <hashed-password>
host-name <ip-address>
mirror-type encapsulated-remote
tcp-ip-stack mirror-stack
user-name 该邮件地址已受到反垃圾邮件插件保护。要显示它需要在浏览器中启用 JavaScript。
show
vcenter vCenter name inventory
command.
v8
is an example vCenter name.dmf-controller-1# show vcenter vc8 inventory
# vCenter ESXi Host Host DNS Name Cluster Product Name Hardware Model CPU Usage (%) Memory Usage (%) Virtual switches Mirror Stack VMkernel Adapter VMkernel Adapter IP Address
-|-------|-------------|-----------------------------------|---------------|--------------------------------|--------------|-------------|----------------|----------------|-----------------------------|---------------------------|
1 vc8 10.240.166.27 ESX27.qa.bsn.sjc.aristanetworks.com BSN-NSX-1 VMware ESXi 8.0.2 build-22380479 PowerEdge R430 2 15 3vmk1192.168.60.27
2 vc8 10.240.166.28 ESX28.qa.bsn.sjc.aristanetworks.com BSN-NSX-2 VMware ESXi 8.0.2 build-223804790 44
3 vc8 10.240.166.29 ESX29.qa.bsn.sjc.aristanetworks.com EdgeVMware ESXi 8.0.0 build-20513097 PowerEdge R430 4 23 3
4 vc8 10.240.166.33 ESX33.qa.bsn.sjc.aristanetworks.com vc8-mixed-stack VMware ESXi 8.0.2 build-223804790 63vmk1192.168.60.33
5 vc8 10.240.166.35 ESX35.qa.bsn.sjc.aristanetworks.com MGMTVMware ESXi 7.0.2 build-17867351 PowerEdge R430 2623 2
6 vc8 10.240.166.38 ESX38.qa.bsn.sjc.aristanetworks.com vc8-mixed-stack VMware ESXi 8.0.2 build-223804791 23 3vmk1192.168.60.38
dmf-rack#
Troubleshooting
Use the show fabric errors
and show fabric
warnings
commands to troubleshoot and verify that everything is functioning as expected.
dmf-controller-1# show fabric errors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vCenter related error ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#vCenter Name Error
--|------------|--------------------------------------------------------------------------------------------------------------------------------------------|
1vc701Unable to locate a matching route for Mirror TCP/IP stack in host ESX37.qa.bsn.sjc.aristanetworks.com for DMF endpoint 192.168.200.254
Create Wildcard Tunnels
The current implementation of VMware vCenter creates one tunnel interface from every ESXi host to DMF.
Using a wildcard tunnel on DMF for VMware vCenter reduces the number of tunnels created.
Platform Compatibility
This feature is only compatible with switches that support wildcard tunneling.
The CLI construct wildcard-tunnels
is available as a configuration option when configuring a VMware vCenter in DANZ Monitoring Fabric (DMF), as shown below:
cluster |
Configure tunnel-endpoint for cluster |
default-tunnel-endpoint |
Configure tunnel endpoints |
description |
Describe this vCenter |
hashed-password |
Set the vCenter password (to log into vCenter) |
host-name |
Set the vCenter hostname |
mirror-type |
Set the vCenter vm monitoring mode |
mirrored-packet-length |
Set the mirrored packet length |
password |
Set the vCenter password (to log into vCenter) |
sampling-rate |
Set the packet sampling rate |
user-name |
Set the vCenter user name (to log into vCenter) |
vm-monitoring |
Enter vm-monitoring config submode |
wildcard-tunnels |
Enable wildcard tunnels |
Enable wildcard tunnels by setting the above leaf parameter, as shown in the following example of vCenter configuration on the Controller node.
dmf-controller-1(config)# vcenter VC1
dmf-controller-1(config-vcenter)# wildcard-tunnels
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
wildcard-tunnels
dmf-controller-1(config-vcenter)#
Similarly, disable wildcard tunnels by issuing the no
command as shown below:
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
wildcard-tunnels
dmf-controller-1(config-vcenter)# no wildcard-tunnels
dmf-controller-1(config-vcenter)# show this
! vcenter
vcenter VC1
dmf-controller-1(config-vcenter)#
Show Commands
There is no specific show command for wildcard tunnels; however, check them in the vCenter running config. In addition, the show tunnels
command shows the tunnels created for the selected vCenter configuration with a wildcard remote IP address.
Troubleshooting
Verify errors and warnings are clear using the show fabric errors
and show fabric warnings
commands. The show
tunnels
command displays tunnels created based on the vCenter configuration on the Controller with a wildcard remote IP address. Use the show switch name table gre-tunnel
command to display tunnels programmed on the switch.
Monitor vCenter Traffic by VM Names
Configuration
dmf-controller-1(config)# policy v1
dmf-controller-1(config-policy)# action forward
dmf-controller-1(config-policy)# filter-interface filter-interface
dmf-controller-1(config-policy)# delivery-interface delivery-interface
dmf-controller-1(config-policy)# filter-vcenter vcenter-name
dmf-controller-1(config-policy)# 1 match ip src-vm-name vm-name dst-vm-name vm-name
dmf-controller-1(config-policy)# 2 match ip6 src-vm-name vm-name
Show Commands
show running-config policy policy
name
command to display the configuration.
dmf-controller-1# show running-config policy v1
! policy
policy v1
action forward
delivery-interface delivery-interface
filter-interface filter-interface
filter-vcenter vcenter-name
1 match ip src-vm-name vm-name dst-vm-name vm-name
2 match ip6 src-vm-name vm-name
show policy policy name
command displays the policy information, including stats.
dmf-controller-1# show policy v2
Policy Name: v2
Config Status: active - forward
Runtime Status : installed
Detailed Status: installed - installed to forward
Priority : 100
Overlap Priority : 0
# of switches with filter interfaces : 1
# of switches with delivery interfaces : 1
# of switches with service interfaces: 0
# of filter interfaces : 1
# of delivery interfaces : 1
# of core interfaces : 0
# of services: 0
# of pre service interfaces: 0
# of post service interfaces : 0
Push VLAN: 5
Post Match Filter Traffic: -
Total Delivery Rate: -
Total Pre Service Rate : -
Total Post Service Rate: -
Overlapping Policies : none
Component Policies : none
Installed Time : 2023-12-21 19:00:39 UTC
Installed Duration : 50 minutes, 11 secs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Match Rules ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Rule
-|--------------------------------------------------------------------------|
1 1 match ip src-vm-name DMF-RADIUS-SERVER-1 dst-vm-name DMF-TACACS-SERVER-1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Filter Interface(s)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF SwitchIF NameState Dir Packets Bytes Pkt Rate Bit Rate Counter Reset Time
-|----------------|-----------|----------|-----|---|-------|-----|--------|--------|------------------------------|
1 span_from_arista Arista-7050 ethernet20 uprx0 0 0-2023-12-21 19:00:39.941000 UTC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Delivery Interface(s)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# DMF IF SwitchIF NameState Dir Packets Bytes Pkt Rate Bit Rate Counter Reset Time
-|------------|-----------|------------|-----|---|-------|-----|--------|--------|------------------------------|
1 ubuntu-tools Arista-7050 ethernet49/2 uptx0 0 0-2023-12-21 19:00:39.941000 UTC
~ Service Interface(s) ~
None.
~ Core Interface(s) ~
None.
~ Failed Path(s) ~
None.
show vcenter vcenter name endpoint
command displays the vCenter VM information, including networks.
dmf-controller-1# show vcenter vcenter1 endpoint
#vCenter Name VM Name ESXi Host Name Network Interface Name MAC AddressIP Address Virtual Switch Portgroup Power State
--|------------|---------|--------------|----------------------|--------------------------|------------------------------------------|--------------|-------------|-----------|
1vcenter1 ub-11-216 10.240.155.216 Network adapter 100:50:56:8b:4d:03 (VMware) 1.1.11.216/24, fe80::250:56ff:fe8b:4d03/64 DVS-DMFvlan11powered-on
2vcenter1 ub-12-216 10.240.155.216 Network adapter 100:50:56:8b:72:a0 (VMware) 1.1.12.216/24, fe80::250:56ff:fe8b:72a0/64 DVS-DMFvlan12powered-on
3vcenter1 ub-13-216 10.240.155.216 Network adapter 100:50:56:8b:c0:06 (VMware) 1.1.13.216/24, fe80::250:56ff:fe8b:c006/64 DVS-DMFvlan-10 powered-on
4vcenter1 ub-14-216 10.240.155.216 Network adapter 100:50:56:8b:d1:d9 (VMware) 1.1.14.216/24, fe80::250:56ff:fe8b:d1d9/64 DVS-DMFvlan-10 powered-on
Troubleshooting
Fabric errors and warnings are very useful for troubleshooting this feature.
show fabric warnings
command, the following validation message displays when the vCenter integration cannot resolve the IP address for the VM name used in the policy.
dmf-controller-1# show fabric warnings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Policy related warning~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Policy Name Warning
-|-----------|------------------------------------------------------------------------------------------------------------|
1 v1No IP found for VMs [ub-15-216, ub-216-multinic, ub-217-vlan10, ub-14-216, ub-11-216] associated with policy
When VM names used in a policy are matched, the following validation message content appears when a vCenter instance is not associated with the policy.
dmf-controller-1# show fabric warnings
~~~~~~~~~~~~~~~~~~~ Policy related warning ~~~~~~~~~~~~~~~~~~~
# Policy Name Warning
-|-----------|-----------------------------------------------|
1 v1No vCenter associated to policy with VM matches