Virtual Edge Deployment

1. Enable SR-IOV in the BIOS. Specific steps depend on the BIOS version. Log in to the BIOS console and look for SR-IOV Support/DMA. Verify support on the prompt by checking that Intel has the correct CPU flag.

   cat /proc/cpuinfo | grep vmx

2. Add the options on Boot (in /etc/default/grub).

   GRUB_CMDLINE_LINUX="intel_iommu=on"

   1. Run the following commands: update-grub and update-initramfs -u.
   2. Reboot.
   3. Enable iommu.

      velocloud@KVMperf3:~$ dmesg | grep -i IOMMU
       [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7 
       [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7 
       [ 0.000000] Intel-IOMMU: enabled
       ….
       velocloud@KVMperf3:~$ 

3. Based on the NIC chip set used, add a driver as follows:
   • For Intel 82599/82599ES cards in SR-IOV mode:
     1. Download and install ixgbe driver from the Intel website.
     2. Configure ixgbe config (tar and sudo make install).

        velocloud@KVMperf1:~$ cat /etc/modprobe.d/ixgbe.conf

     3. If the ixgbe configuration file does not exist, create the file as follows.

        options ixgbe max_vfs=32,32
        options ixgbe allow_unsupported_sfp=1
        options ixgbe MDD=0,0
        blacklist ixgbevf

     4. Run the update-initramfs -u command and reboot the Server.
     5. Use the modinfo command to verify successful installation.

        velocloud@KVMperf1:~$ modinfo ixgbe and ip link
         filename: /lib/modules/4.4.0-62-generic/updates/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
         version: 5.0.4
         license: GPL
         description: Intel(R) 10GbE PCI Express Linux Network Driver
         author: Intel Corporation, <该邮件地址已受到反垃圾邮件插件保护。要显示它需要在浏览器中启用 JavaScript。>
         srcversion: BA7E024DFE57A92C4F1DC93

   • For the Intel X710/XL710 cards in SR-IOV mode:
     1. Download and install i40e driver from the Intel website.
     2. Create the Virtual Functions (VFs).

        echo 4 > /sys/class/net/device name/device/sriov_numvfs

     3. To make the VFs persistent after a reboot, add the command from the previous step to the /etc/rc.d/rc.local file.
     4. Deactivate the VF driver.

        echo “blacklist i40evf” >> /etc/modprobe.d/blacklist.conf

     5. Run the update-initramfs -u command and reboot the Server.

lspci | grep -i Ethernet

01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function(rev

This topic describes the procedure for installing and activating the Virtual Edge on KVM using a cloud-init configuration file.

1. Use gunzip to extract the qcow2 file to the image location (for example, /var/lib/libvirt/images).
2. Create the Network pools for the device using SR-IOV and OpenVswitch.

   Using SR-IOV:

   The following is a sample network interface template specific to Intel X710/XL710 NIC cards using SR-IOV.

       <interface type='hostdev' managed='yes'>
           <mac address='52:54:00:79:19:3d'/>
           <driver name='vfio'/>
           <source>
               <address type='pci' domain='0x0000' bus='0x83' slot='0x0a' function='0x0'/>
           </source>
           <model type='virtio'/>
       </interface>

   Using OpenVSwitch:

   <network>
     <name>passthrough</name>
     <model type='virtio'/>
     <forward mode="bridge"/>
     <bridge name="passthrough"/>
     <virtualport type='openvswitch'/>
     <vlan trunk='yes'>
       <tag id='33' nativeMode='untagged'/>
       <tag id='200'/>
       <tag id='201'/>
       <tag id='202'/>
     </vlan>
   </network>
   
   <network>
     <name>passthrough</name>
     <model type='virtio'/>
     <forward mode="bridge"/>
   </network>
   
   <domain type='kvm'>
     <name>vedge1</name>
     <memory unit='KiB'>4194304</memory>
     <currentMemory unit='KiB'>4194304</currentMemory>
     <vcpu placement='static'>2</vcpu>
     <resource>
       <partition>/machine</partition>
     </resource>
     <os>
       <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
       <boot dev='hd'/>
     </os>
     <features>
       <acpi/>
       <apic/>
       <pae/>
     </features>
     <!-- Set the CPU mode to host model to leverage all the available features on the host CPU -->
     <cpu mode='host-model'>
       <model fallback='allow'/>
     </cpu>
     <clock offset='utc'/>
     <on_poweroff>destroy</on_poweroff>
     <on_reboot>restart</on_reboot>
     <on_crash>restart</on_crash>
     <devices>
       <emulator>/usr/bin/kvm-spice</emulator>
       <!-- Below is the location of the qcow2 disk image -->
       <disk type='file' device='disk'>
         <driver name='qemu' type='qcow2'/>
         <source file='/var/lib/libvirt/images/edge-VC_KVM_GUEST-x86_64-2.3.0-18-R23-20161114-GA-updatable-ext4.qcow2'/>
         <target dev='sda' bus='sata'/>
         <address type='drive' controller='0' bus='0' target='0' unit='0'/>
       </disk>
       <!-- If using cloud-init to boot up virtual edge, attach the 2nd disk as CD-ROM -->
       <disk type='file' device='cdrom'>
         <driver name='qemu' type='raw'/>
         <source file='/home/vcadmin/cloud-init/vedge1/seed.iso'/>
         <target dev='sdb' bus='sata'/>
         <readonly/>
         <address type='drive' controller='1' bus='0' target='0' unit='0'/>
       </disk>
       <controller type='usb' index='0'>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
       </controller>
       <controller type='pci' index='0' model='pci-root'/>
       <controller type='sata' index='0'>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
       </controller>
       <controller type='ide' index='0'>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
       </controller>
       <!-- The first two interfaces are for the default L2 interfaces, NOTE VLAN support just for SR-IOV and OpenvSwitch -->
       <interface type='network'>
         <model type='virtio'/>
         <source network='LAN1'/>
         <vlan><tag id='#hole2_vlan#'/></vlan>
         <alias name='LAN1'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x12' function='0x0'/>
       </interface>
       <interface type='network'>
         <model type='virtio'/>
         <source network='LAN2'/>
         <vlan><tag id='#LAN2_VLAN#'/></vlan>
         <alias name='hostdev1'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x13' function='0x0'/>
       </interface>
       <!-- The next two interfaces are for the default L3 interfaces. Note that additional 6 routed interfaces are supported for a combination of 8 interfaces total -->
       <interface type='network'>
         <model type='virtio'/>
         <source network='WAN1'/>
         <vlan><tag id='#hole2_vlan#'/></vlan>
         <alias name='LAN1'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x12' function='0x0'/>
       </interface>
       <interface type='network'>
         <model type='virtio'/>
         <source network='LAN2'/>
         <vlan><tag id='#LAN2_VLAN#'/></vlan>
         <alias name='hostdev1'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x13' function='0x0'/>
       </interface>
       <serial type='pty'>
         <target port='0'/>
       </serial>
       <console type='pty'>
         <target type='serial' port='0'/>
       </console>
       <input type='mouse' bus='ps2'/>
       <input type='keyboard' bus='ps2'/>
       <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'>
         <listen type='address' address='127.0.0.1'/>
       </graphics>
       <sound model='ich6'>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
       </sound>
       <video>
         <model type='cirrus' vram='9216' heads='1'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
       </video>
       <memballoon model='virtio'>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
       </memballoon>
     </devices>
   </domain>

    

3. Save the domain XML file that defines the VM (for example, vedge1.xml created in Step 2).
4. Launch the VM by performing the following steps:
   1. Create VM.

      virsh define vedge1.xml

   2. Start VM.

      virsh start vedge1

   Note: The name element in the domain XML file defines the VM as vedge1. Replace vedge1 with the specific name used in the name element.
5. When using SR-IOV mode, after launching the VM, set the following on the Virtual Functions (VFs) used:
   1. Set the spoofchk off.

      ip link set eth1 vf 0 spoofchk off

   2. Set the Trusted mode on.

      ip link set dev eth1 vf 0 trust on

   3. Set the VLAN, if necessary.

      ip link set eth1 vf 0 vlan 3500

   Note: Open vSwitch (OVS) mode bypasses the Virtual Functions configuration step.
6. Console into the VM.

   virsh list
   Id Name State
   ----------------------------------------------------
   25 test_vcg running
   velocloud@KVMperf2$ virsh console 25
   Connected to domain test_vcg
   Escape character is ^]

1. Ensure the NIC supports SR-IOV. Check the Hardware Compatibility List (HCL) at Arista Documentation.
   • Brand Name: Intel
   • I/O Device Type: Network
   • Features: SR-IOV

   

2. Go to the specific Arista host, select the Configure tab, and then choose Physical adapters.

   

3. Select Edit Settings. Change Status to Enabled and specify the number of required virtual functions. This number varies by the type of NIC card.

   

4. Reboot the hypervisor.
   A successful SR-IOV enablement displays the number of Virtual Functions (VFs) under the specified NIC, following an ESXi reboot.

   

   Note: To support VLAN tagging on SR-IOV interfaces, user must configure VLAN ID 4095 (Allow All) on the Port Group connected to the SR-IOV interface. For additional information, see VLAN Configuration.

1. Use the vSphere client to deploy an OVF template, and then select the Edge OVA file.

   

2. Select an OVF template from a URL or a Local file.

   

3. Select a name and location of the virtual machine.
4. Select a resource.
5. Verify the template details.

   

6. Select the storage location to store the files for the deployment template.

   

7. Configure the networks for each of the interfaces.
   Note: Skip this step when using a cloud-init file to provision the Virtual Edge on ESXi.

   

8. Customize the template by specifying the deployment properties. The following image highlights:
   1. From the Orchestrator UI, retrieve the URL/IP Address. Use this address in Step c.
   2. Create a new Virtual Edge for the Enterprise. After the Edge is created, copy the Activation Key. Use this Activation Key in Step c.

      

   3. On the Customize template page shown in the image below, enter the Activation Code and the Orchestrator URL/IP Address into the corresponding fields.

      

       

      

9. Review the configuration data.

   

10. Turn on the Virtual Edge.

    

    After the Edge powers up, it establishes connectivity to the Orchestrator.