Setting Up the Host and Launching PCI Pass-through

Set up a networking device to use PCI pass-through.

When sharing resources are not efficient, or packets are consumed by a virtualized switch before reaching the VM (vEOS), implementing PCI Pass-through for NIC provides dedicated and non-filtered network resources to the VM.

1. Identify Available Physical Functions.

Similar to the SR-IOV, identify an available physical function (a NIC in this scenario) and its identifier. Use the lspci | grep Ethernet Linux command to display the available physical functions.

In this example, 82:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection is the physical function and 82:00.0 is the device identification code.

# lspci | grep Ethernet
01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
81:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
82:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
82:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
83:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
83:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

2. Verify Available Physical Functions.

Verify the available physical functions by using the virsh Linux commands.

[arista@solution]$ virsh nodedev-list | grep 82_00_0
pci_0000_82_00_0
[arista@solution]$ virsh nodedev-dumpxml pci_0000_82_00_0
<device>
<name>pci_0000_82_00_0</name>
<path>/sys/devices/pci0000:80/0000:80:02.0/0000:82:00.0</path>
<parent>pci_0000_80_02_0</parent>
<driver>
<name>vfio-pci</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>130</bus>
<slot>0</slot>
<function>0</function>
<product id='0x10fb'>82599ES 10-Gigabit SFI/SFP+ Network Connection</product>
<vendor id='0x8086'>Intel Corporation</vendor>
<capability type='virt_functions' maxCount='64'/>

In this example, the domain is 0 (Hex domain=0x0), the bus is 130 (Hex bus=0x82), the slot is 0 (Hex slot=0x0), and function is 0 (Hex function=0x0).

With the domain, bus, slot, and function information, construct the device entry and add it into the VMs XML configuration.

 <devices>
...
  <hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x82' slot='0x00' function='0x0'/>
</source>
</hostdev>

3. Verify the NIC was detected by the VM.

When starting the VM (vEOS in this case), the VM should detect NIC.

switch#bash
 
Arista Networks EOS shell
 
[admin@veos1 ~]$ lspci | grep Ethernet
00:03.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
[admin@veos ~]$ 

4. Verify Driver Requirements.

If the NIC is supported by the vEOS and any other driver requirements are met, the corresponding ethernet interfaces are available to use on the vEOS. Use the show interface command to display the available vEOS Ethernet interfaces.

switch#show interfacestatus 
Port Name Status Vlan Duplex SpeedType Flags
 Et1connectedrouted full10G10/100/1000 
 Ma1connectedrouted a-fulla-1G 10/100/1000 
 
switch#bash
bash-4.3# ethtool -i et1
driver: ixgbe
version: 4.2.1-k
firmware-version: 0x18b30001
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no 

Example Deployment

vEOS can use passthrough I/O interfaces where the network I/O does not hit the hypervisor. In this model, the VM owns the entire network card, thus fully bypassing the hypervisor.

Setting up SR-IOV is initially more involved. Arista recommends starting out with LinuxBridge.

  • SR-IOV has the following advantages over LinuxBridge Higher Performance ~ 2x
  • Better latency and jitter characteristics
  • vEOS directly receives physical port state indications from the virtual device.
Figure 1. Linux PCI Passthrough-based Deployment