印刷

VeloCloud SD-WAN 6.1 - Administration Guide - Azure Accelerated Networking Support for Virtual Edges 印刷

Azure Accelerated Networking Support for Virtual Edges

Azure accelerated networking support is available for Virtual Edges from the 5.4 release and later. This document provides detailed instructions on how to enable, disable, and verify accelerated networking for Virtual Edges, as well as providing support and host servicing details.

Accelerated Networking is Azure's implementation of single root I/O virtualization (SR-IOV), a standard that allows a physical PCIE device to appear as multiple virtual devices (virtual functions). When Accelerated Networking is enabled on an interface of a VeloCloud Edge on Azure, SR-IOV support for Mellanox ConnectX-4 and ConnectX-5 Network Interface Cards (NICs) is automatically enabled in the Edge Virtual Machine (VM).

Note: The SR-IOV support for Mellanox ConnectX-4 and ConnectX-5 NICs is only available for the VeloCloud Edge on Azure in the 5.4 release.

Azure Instance Support

The following table lists instance types that support the Accelerated Networking functionality.

Table 1. Azure Instance Support
Software Version Instance Types
Edge Software 5.4 or later Standard_D3_v2
Standard_D4_v2
Standard_D5_v2
Standard_D4_v5
Standard_D8_v5
Standard_D16_v5

Enable or Disable Azure Accelerated Networking

This section provides links for detailed instructions on the different ways to enable and disable Azure Accelerated Networking.

Accelerated Networking can be enabled or disabled via the following locations:
  • Azure portal
  • Azure CLI or Azure PowerShell
Select the following section links for instructions on how to enable and disable Accelerated Networking via the Azure portal and the Azure CLI or Azure PowerShell.

Enable Accelerated Networking

Accelerated Networking can be enabled via the Azure portal or via the Azure CLI. See this section for detailed instructions.

Azure requires that an existing Virtual Machine (VM) be stopped/deallocated before Accelerated Networking is enabled on any Network Interface Card (NIC). For additional information, see: https://learn.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli

There are two ways to enable Accelerated Networking, via the Azure portal or via the Azure CLI. For detailed instructions for both, see the sections below.

  1. Enable Accelerated Networking Via Azure Portal: To enable Accelerated Networking on an interface of an existing VM, follow the steps below, and see the image below for additional information.
    1. Stop/deallocate the VM.
    2. Change the setting of accelerated networking to Enabled.
    3. Restart the VM.
    Figure 1. Enable Accelerated Networking Via Azure Portal
    Important: To fully benefit from the Accelerated Networking feature, enable it on all interfaces of the VM.
    After the VM restarts, if the NIC model type allotted to the interface is not Mellanox ConnectX4 or ConnectX5, accelerated networking support for the interface is not activated by the Edge VM. As a troubleshooting effort, change the NIC model, stop/deallocate the VM, and then restart the VM.
    Note: You must stop the VM before you enable the Accelerated Networking support.
  2. Enable Accelerated Networking via Azure CLI: Follow the steps below to enable Accelerated Networking via the Azure CLI. For additional information, see https://learn.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli?tabs=windows#enable-accelerated-networking-on-individual-vms-or-vms-in-availability-sets.
    1. Deallocate resources of the VM. 
      az vm deallocate --resource-group <myResourceGroup> --name <myVm>

       

    2. Enable Accelerated Networking on the NIC. 
      az network nic update \ 
--name <myNic> \
--resource-group <myResourceGroup> \
--accelerated-networking true

       

    3. Restart the VM. 
      az vm start --resource-group <myResourceGroup> --name <myVm>

Disable Accelerated Networking

There are two options to disable Accelerated Networking, via the Azure portal or via the Azure CLI. See the sections below for detailed instructions.

Disable Accelerated Networking

To disable Accelerated Networking, choose one of the two options described in the sections below.

  1. Accelerated Networking Disabled Via Azure Portal: To disable Accelerated Networking on an interface of an existing Virtual Machine (VM):
    1. Stop/deallocate the VM.
    2. Change the setting of accelerated networking to disabled or false.
    3. Restart the VM.
  2. Accelerated Networking disabled via Azure CLI
    1. Deallocate resources of the VM. 
      az vm deallocate --resource-group <myResourceGroup> --name <myVm>

       

    2. Disable Accelerated Networking on the Network Interface Card (NIC). 
      az network nic update \
--name <myNic> \
--resource-group <myResourceGroup> \
--accelerated-networking false

       

    3. Restart the VM. 
      az vm start --resource-group <myResourceGroup> --name <myVm>

Verifying Accelerated Networking

This section discusses detailed instructions on how to verify Accelerated Networking.

To verify that Accelerated Networking is enabled on the Edge, log into the Edge Virtual Machine (VM) and run "lspci" from the command line as described below.

edge:b1-edge1:~# lspci
0000:00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled) (rev 03)
0000:00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 01)
0000:00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
0000:00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
0000:00:08.0 VGA compatible controller: Microsoft Corporation Hyper-V virtual VGA
0fcd:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)
8f67:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)
9b19:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)
9b46:00:02.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80)

This feature does not support the Mellanox ConnectX-3 as it is not managed by the mlx5_core driver. If the Edge VM is assigned the ConnectX-3 Network Interface Cards (NICs) by Azure, the Edge will behave as it behaves today without the Accelerated Networking support.

Azure Host Servicing

This section discusses detailed instructions for Azure Host Servicing.

The Azure Accelerated Networking support for Virtual Edges on Azure adds the DPDK failsafe/TAP/MLX PMD model. When Azure host maintenance is performed, the SR-IOV VFs might be temporarily removed and added back later.

Reference the following for additional information: https://learn.microsoft.com/en-us/azure/virtual-network/accelerated-networking-how-it-works#azure-host-servicing.

The events are logged by the kernel and can be viewed in the output of dmesg. They can also be viewed in /var/log/messages: 
edge:b3-edge1:~# egrep 'VF registering|VF unregistering' /var/log/messages
2023-07-25T22:06:11.903 INFO kern kernel:[ 11.091250] hv_netvsc 000d3a92-2dba-000d-3a92-2dba000d3a92 eth1: VF registering: eth5
2023-07-25T22:06:12.049 INFO kern kernel:[ 11.237233] hv_netvsc 000d3a92-245f-000d-3a92-245f000d3a92 eth2: VF registering: eth6
2023-07-25T22:06:12.208 INFO kern kernel:[ 11.396127] hv_netvsc 000d3a92-2178-000d-3a92-2178000d3a92 eth3: VF registering: eth7
2023-07-25T22:06:12.362 INFO kern kernel:[ 11.549624] hv_netvsc 000d3a92-218a-000d-3a92-218a000d3a92 eth4: VF registering: eth8
2023-07-25T22:30:09.188 INFO kern kernel:[ 1448.376507] hv_netvsc 000d3a92-2178-000d-3a92-2178000d3a92 eth3-hv: VF unregistering: eth7
2023-07-25T22:30:14.390 INFO kern kernel:[ 1453.577954] hv_netvsc 000d3a92-218a-000d-3a92-218a000d3a92 eth4-hv: VF unregistering: eth8
2023-07-25T22:30:19.380 INFO kern kernel:[ 1458.568168] hv_netvsc 000d3a92-2dba-000d-3a92-2dba000d3a92 eth1-hv: VF unregistering: eth5
2023-07-25T22:30:26.555 INFO kern kernel:[ 1465.742626] hv_netvsc 000d3a92-245f-000d-3a92-245f000d3a92 eth2-hv: VF unregistering: eth6

 

The HOTPLUG OUT events are reported on the VeloCloud Orchestrator, as shown in the image below.
Figure 2. HOTPLUG OUT Events

The Edge continues running during the Azure Host maintenance and the Paths stay up.

When the SR-IOV VFs are added back after Azure host maintenance is complete, the events can be viewed in the output of dmesg or in /var/log/messages. 
edge:b3-edge1:~# egrep 'VF registering|VF unregistering' /var/log/messages
...
2023-07-25T22:31:23.137 INFO kern kernel:[ 1522.324791] hv_netvsc 000d3a92-2dba-000d-3a92-2dba000d3a92 eth1-hv: VF registering: eth5
2023-07-25T22:31:28.381 INFO kern kernel:[ 1527.568576] hv_netvsc 000d3a92-2178-000d-3a92-2178000d3a92 eth3-hv: VF registering: eth6
2023-07-25T22:31:33.416 INFO kern kernel:[ 1532.604181] hv_netvsc 000d3a92-245f-000d-3a92-245f000d3a92 eth2-hv: VF registering: eth7
2023-07-25T22:31:38.468 INFO kern kernel:[ 1537.656531] hv_netvsc 000d3a92-218a-000d-3a92-218a000d3a92 eth4-hv: VF registering: eth8

 

The HOTPLUG IN events are reported on the VeloCloud Orchestrator.

Figure 3. HOTPLUG IN Events
..