These sections describe VXLAN architecture, the data objects that comprise a VXLAN network, and process of bridging packets through a VXLAN network.
The VXLAN architecture extends an L2 network by connecting VLANs from multiple hosts through UDP tunnels called VXLAN segments. VXLAN segments are identified by a 24-bit virtual network identifier (VNI). Within a host, each VLAN whose network is extended to other hosts is associated with a VNI. An extended L2 network comprises the devices attached to VLANs from all hosts that are on VLANs that are associated with the same VNI.
The following figure displays the data objects that comprise a VXLAN implementation on a local host.
- VXLAN Tunnel End Point (VTEP): a host with at least one VXLAN Tunnel Interface (VTI).
- VXLAN Tunnel Interface (VTI): a switchport linked to a UDP socket that is shared with VLANs on various hosts. Packets bridged from a VLAN to the VTI are sent out the UDP socket with a VXLAN header. Packets arriving on the VTI through the UDP socket are demuxed to VLANs for bridging.
- Virtual Network Identifier (VNI): a 24-bit number that distinguishes between the VLANs carried on a VTI. It facilitates the multiplexing of several VLANs over a single VTI.
VNIs can be expressed in digital or dotted decimal formats. VNI values range from 1 to 16777215 or from 0.0.1 to 255.255.255.
The network in the figure above has the following assignments:
- VTEP IP address of 10.10.1.1
- UDP port of 4789
- One VTI that supports three VXLAN segments (UDP tunnels): VNI 200, VNI 2000, and VNI 20000
- Five VLANs, of which three VLANs can communicate with remote devices over Layer 2.
A VXLAN gateway is a service that exchanges VXLAN data and packets with devices connected to different network segments. VXLAN traffic must pass through a VXLAN gateway to access services on physical devices in a distant network.
A VXLAN gateway requires the following information:
- An IP address that is designated as the VXLAN interface source.
- VLAN to VNI mapping.
- VTEP list for each VNI.
- A method for handling broadcast, unknown unicast, and multicast (BUM) packets.
Arista switches manually perform VXLAN gateway services. The switch connects to VXLAN gateways that serve other network segments. MAC address learning is performed in hardware from inbound VXLAN packets.
When a packet enters a VLAN from a member (ingress) port, the VLAN learns the source address by adding an entry to the MAC address table that associates the source to the ingress-port. The VLAN then searches the table for destination address. If the MAC address table lists the address, the packet is sent out the corresponding port. If the MAC address table does not lists the address, the packet is flooded to all ports except the ingress port.
VXLANs extend VLANs through the addition of a VXLAN address table that correlates remote MAC addresses to their port and resident host IP address. Packets that are destined to a remote device are sent to the VXLAN tunnel interface (VTI), which is the switchport that is linked to the UDP socket. The packet is encapsulated with a VXLAN header which includes the VNI associated with the VLAN and the IP mapping of the destination host. The packet is sent through a UDP socket to the destination VTEP IP. The VTI on the remote host extracts the original packet and bridges it to the VLAN associated with the VNI on the remote host.
UDP port 4789 is recognized as the VXLAN socket and listed as the destination port on the UDP packets. The UDP source port field is filled with a hash of the inner header to facilitate load balancing.
- VNI 200: VTEP 10.20.2.2: VLAN 1200 and VTEP 10.30.3.3: VLAN 200
- VNI 2000: VTEP 10.10.1.1: VLAN 300, VTEP 10.20.2.2: VLAN 1400, and VTEP 10.30.3.3: VLAN 300
- VNI 20000: VTEP 10.10.1.1: VLAN 200, and VTEP 10.20.2.2: VLAN 1600
VXLAN routing is enabled by creating a VLAN interface on the VXLAN-enabled VLAN and assigning an IP address to the VLAN interface. The IP address serves as VXLAN gateway for devices that are accessible from the VXLAN-enabled VLAN.
Multicast and Broadcast over VXLAN
These sections describe multicast and broadcast over VXLANs. Multicast packet flooding describes broadcast and multicast transmission by associating a multicast group to a VTI through a configuration command.
Head-end Replication (HER) optimizes flooding of inter VTEP broadcast, unknown unicast and broadcast (BUM) traffic by using hardware and flood lists to perform replication on the supported platform.
Multicast Packet Flooding
Multicast packet flooding is supported with VXLAN bridging without MLAG. A VTI is associated with a multicast group through a configuration command.
VXLAN and Broadcast
When a VLAN receives or sends a broadcast packet the VTI is treated as a bridging domain L2 interface. The packet is sent from this interface on the multicast group associated with the VTI. The VTIs on remote VTEPs that receive this packet extract the original packet, which is then handled by the VLAN associated with the packet’s VNI. The VLAN floods the packet, excluding the VTI. When the broadcast results in a response, the resulting packet can be unicast back to the originating VTEP because the VXLAN address table obtained the host MAC to VTEP association from the broadcast packet.
VXLAN and Multicast
A VTI is treated as an L2 interface in the VLAN for handling multicast traffic, which is mapped from the VLAN to the multicast group associated with the VTI. All VTEPs join the configured multicast group for inter-VTEP communication within a VXLAN segment; this multicast group is independent of any other multicast groups that the hosts in the VLAN join.
The IP address space for the inter-host VXLAN communication may be sourced from a different VRF than the address space of the hosts in the VLAN. The multicast group for inter-VTEP transmissions must not be used for other purposes by any device in the VXLAN segment space.
Head-end replication uses a flood list to support broadcast, unknown unicast, and multicast (BUM) traffic over VXLAN. The flood list specifies a list of remote VTEPs. The switch replicates BUM data locally for bridging across the remote VTEPs specified by the flood list. This data flooding facilitates remote MAC address learning by forwarding data with unknown MAC addresses.
Head-end replication is required for VXLAN routing and to support VXLANs over MLAG.
VXLAN and MLAG
VXLAN over MLAG provides redundancy in hardware VTEPs. VTI configuration must be identical on each MLAG peer for them to act as a single VTEP. This also prevents the remote MAC from flapping between the remote VTEPs by ensuring that the rest of the network sees a host that is connected to the MLAG interface as residing behind a single VTEP.
Differences between VXLAN bridging and routing implementations over MLAG are applicable for the DCS-7050X series platform.
- VXLAN routing recirculates a packet twice, with the first iteration performing the routing action involving an L2 header rewrite, and the second recirculation performing VXLAN encap and decap operations. Recirculation is achieved by MAC loopback on dedicated loopback interfaces.
- The configuration for VXLAN routing on an MLAG VTEP includes separate Recirc-Channel configuration on both peers. The virtual IP, virtual MAC, and virtual VARP VTEP IP addresses are identical on both peers.
The following VTI elements must be configured identically on both MLAG peers:
- VLAN-VNI mappings
- VTEP IP address of the source loopback interface
- Flood VTEP list used for head-end replication
If OSPF is also in use, configure the OSPF router ID manually to prevent the switch from using the common VTEP IP address as the router ID.
The following rules are observed by MLAG switches so that they behave as a single VXLAN VTEP:
- Only the MLAG peer that receives a packet performs VXLAN encapsulation on it.
- Packets are not VXLAN encapsulated if they are received from the peer link.
- If a packet is decapsulated and sent over the peer link, it should not be flooded to active MLAG interfaces.
- If a packet is sent over the peer link to the CPU, it is not head-end replicated to other remote VTEPs.
- If a packet’s destination is the VTEP IP address, it is terminated by the MLAG
peer that receives it.
These commands complete the configuration required for a VXLAN routing deployment.
switch(config)#interface Vxlan1 switch(config-if-Vx1)#vxlan source-interface Loopback0 switch(config-if-Vx1)#vxlan udp-port 4789 switch(config-if-Vx1)#vxlan vlan 2417 vni 8358534 switch(config-if-Vx1)#vxlan flood vtep 188.8.131.52 184.108.40.206 switch(config-if-Vx1)#interface Vlan2417 switch(config-if-Vl2417)#ip address 220.127.116.11/24 switch(config-if-Vl2417)#interface Loopback0 switch(config-if-Lo0)#ip address 18.104.22.168/32 switch(config-if-Lo0)#ip routing switch(config)#interface Recirc-Channel627 switch(config-if-Re627)#switchport recirculation features vxlan switch(config-if-Re627)#interface Ethernet 1 switch(config-if-Et1)#traffic-loopback source system device mac switch(config-if-Et1)#channel-group recirculation 627 switch(config-if-Et1)#exit switch(config)#interface Ethernet 2 switch(config-if-Et2)#traffic-loopback source system device mac switch(config-if-Et2)#channel-group recirculation 627 switch(config-if-Et2)#
show running interface Loopback allows remote VTEP tunnels to be routed over L3 interfaces.
switch#show running interface Loopback298 interface Loopback298 ip address 22.214.171.124/32 switch# switch#show running interface Ethernet54/1.4095 interface Ethernet54/1.4095 mtu 9214 encapsulation dot1q vlan 267 ip address 126.96.36.199/24 switch# switch#show running interface Port-Channel1.4095 interface Port-Channel1.4095 mtu 9214 encapsulation dot1q vlan 1043 ip address 188.8.131.52/24 switch# switch#show running interface Vxlan1 interface Vxlan1 vxlan source-interface Loopback298 vxlan udp-port 4789 vxlan vlan 2156 vni 15613244 vxlan vlan 2393 vni 3610141 vxlan vlan 2156 flood vtep 184.108.40.206 220.127.116.11 vxlan vlan 2393 flood vtep 18.104.22.168 22.214.171.124 switch# switch#show port-channel Port Channel Port-Channel1: Active Ports: Ethernet51/1 switch# switch#show ip route VRF: default Codes: C - connected, S - static, K - kernel, O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1, E2 - OSPF external type 2, N1 - OSPF NSSA external type 1, . . . . . . S126.96.36.199/32 [1/0] via 188.8.131.52, Ethernet54/1.4095 S184.108.40.206/32 [1/0] via 220.127.116.11, Port-Channel1.4095
There are 2 remote VTEPs configured 18.104.22.168, and 22.214.171.124. The remote VTEP 126.96.36.199 is reachable through Ethernet54/1.4095 and remote VTEP 188.8.131.52 is reachable through port-channel1.4095.
Configuring Unconnected Ethernet Interfaces for Recirculation
On systems where bandwidth is not fully used by the front panel ports, unused bandwidth is used for recirculation.
The following example is applicable to the DCS-7050X series platform.
These commands expose unconnected Ethernet interfaces which are used for recirculation, in order to use them to replace or use along with front panel Ethernet interfaces.
switch(config)#service interface unconnected expose switch(config)#interface UnconnectedEthernet 2 switch(config-if-Ue2)#traffic-loopback source system device mac switch(config-if-Ue2)#channel-group recirculation 627
The following example enables display of the inactive interfaces using the show command.
switch(config)#service interface inactive expose
Running a show command generates the following output:
switch(config)#switch(config)#sho int et21/1-4 stat
Port Name Status Vlan Duplex SpeedType Flags Encapsulation Et21/1 connected 1 full 100G 100GBASE-CR4 Et21/2 inactive 1 full 25G100GBASE-CR4 Et21/3 inactive 1 full 25G100GBASE-CR4 Et21/4 inactive 1 full 25G100GBASE-CR4
On previous releases, Ethernet21/2,21/4 do not exist and the output would be the following:
switch(config)#switch(config)#sho int et21/1-4 stat
Port Name Status Vlan Duplex SpeedType Flags Encapsulation Et21/1 connected 1 full 100G 100GBASE-CR4 Et21/3 inactive 1 full 25G100GBASE-CR4
VXLAN Bridging and Routing Support
Describes the support of VXLAN Bridging and Routing on the R3 series of DCS 7280, 7500, and 7800 Arista switches.
- DCS-7500R3 Linecards
- DCS-7800R3 Linecards
Differences with DCS-7500R2 Implementation
- There is no need to configure the VXLAN-routing TCAM profile to enable VXLAN routing on the R3 Series switches. The command is still accepted for backward compatibility reasons.
- CPU bound traffic after VXLAN decapsulation (such as routing protocol packets) use the same CoPP queues used by the non VXLAN decapsulated packets. This is an improvement over the R2 series behavior where the CPU bound traffic after VXLAN decapsulation took a different CoPP queue that was shared with other IP Unicast packets.
Ther is no EVPN VxLAN Multicast (Type 6/7/8 NLRI) support.
VXLAN implementation requires two VXLAN tables and a MAC address table accommodation.
MAC Address Table VXLAN Support
MAC address table entries correlate MAC addresses with the port upon which packets arrive. In addition to Ethernet and port channels, the port column may specify a VTI for packets that arrive on a VLAN from a remote port through the VXLAN segment.
VTEP-MAC Address Table
VTEP-MAC address table entries correlate MAC address with the IP address of the VTEP from where packets bearing the MAC address arrive. The VTI uses this table to determine the destination address for packets that are sent to remote hosts.
The VNI-VLAN map displays the one-to-one correspondence between the VNIs assigned on the switch and the VLANs to which they are assigned. Each VNI can be assigned to only one VLAN; each VLAN can be assigned a maximum of one VNI. Each VNI-VLAN assignment constitutes a VXLAN segment.