Auto VPN

 

AutoVPN is a feature that facilitates the easy setup and configuration of an enterprise WAN network, which could encompass multiple branches, transit hubs, data centers, and cloud deployments. It offers the following capabilities:

  • Automatic endpoint discovery
  • Secure IPsec key exchange and key rotation between endpoints without requiring a full mesh configuration.
  • Dynamic adaptation to WAN IP changes.

Auto VPN Configuration

 

Pre-configuration Requirements

Since many of the Caravan features are currently unavailable in the standard CloudEOS release image, users must activate specific toggles on the CloudEOS router to enable these features behind the scenes.
  • Please ensure that the routing model used is multi-agent by configuring it using the following command: service routing protocols model multi-agent. This will require a reboot to take effect.
  • Below are the steps to enable the right set of toggles:
    1. Create a file under /mnt/flash and name it toggle_override
    2. If this file already exists, you can directly modify it.
      Note: The same toggle needs to be enabled on both the edges and the Pathfinder.
    3. Add the below mentioned lines in the file to enable the required features and save the file (a reboot will be needed for the features to be enabled).
      Avt=1
      BgpDps=1
      SfeDpi=1
      DpsPathMtu=1
      ClassificationDpi=1
      Stun=1
      IpsecKeyController=1
      DpsPatSupport=1
      BgpLsProducerDps=1
      BgpLsConsumerDps=1
      ArBgpAddPathSendCapLinkState=1
      SrTeDpsPolicy=1

Security Considerations

In typical deployments, there might be a Firewall (FW) situated in the path between an edge device and the Pathfinder or another edge device. To ensure that DPS/Auto-VPN connections can be established successfully, certain ports need to be opened in the FW's security settings. These ports facilitate the necessary communication for the DPS/Auto-VPN functionality.

To enable DPS/Auto-VPN connections through the Firewall, the following ports should be opened:

  • IKE: Port 500 (UDP)
  • IPsec: Port 4500 (UDP)
  • STUN: Port 3478 (UDP)
  • ESP: Protocol 50
  • If no encryption: Port 4793

 

EOS Configurations at the Pathfinder

Below are the steps for EOS configuration at the Pathfinder with examples:

  1. To configure loopback0, utilize the command interface Loopback. This command will set the switch to the interface loopback mode.
    switch(config)#interface loopback 0
    switch(config-if-Lo0)#
    switch(config-if-Lo0)#ip address 192.192.99.1/32
  2. To configure interface VXLAN 1, use the command interface vxlan. This command will set the switch to the interface VXLAN mode for VXLAN 1.
    switch(config)#interface vxlan 1
    switch(config-if-Vx1)#vxlan source-interface loopback 0
    switch(config-if-Vx1)#vxlan udp-port 4789
    switch(config-if-Vx1)#vxlan vrf default vni 101
  3. To configure STUN, utilize the stun command. This command will set the switch to the STUN mode.
    switch(config)#stun 
    switch(config-stun)#server 
    switch(config-stun-server)#local-interface ethernet 1
    switch(config-stun-server)#router path-selection 
    switch(config-dynamic-path-selection)#peer dynamic source stun 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#local interface ethernet 1
    switch(config-internet-interface-Ethernet1)#
  4. The ipsec command is available for IPsecurity configuration, but it is important to note that it is optional. IPsec is not required for DPS/AutoVPN to function correctly.
    switch(config)#ip security 
    switch(config-ipsec)#ike policy ikepolicyCDApp 
    switch(config-ipsec-ike)#local-id 192.192.99.1
    switch(config-ipsec-ike)#sa policy sapolicyCDApp 
    switch(config-ipsec-sa)#profile profileCDApp 
    switch(config-ipsec-profile)#ike-policy ikepolicyCDApp 
    switch(config-ipsec-profile)#sa sapolicyCDApp 
    switch(config-ipsec-profile)#connection start 
    switch(config-ipsec-profile)#shared-key 7 <pre-shared key in clear text shown by the running config> 
    switch(config-ipsec-profile)#dpd 10 50 clear 
    switch(config-ipsec-profile)#mode transport 
    switch(config-ipsec-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#ipsec profile profileCDApp 
    autovpn-transit1(config-path-group-internet-0)#
  5. Create a DPS1 interface using the interface dps command.
    switch(config)#interface dps 1
    switch(config-if-Dp1)#
  6. Configure the Load Balance policies and the DPS policies as shown in the below example.
    
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#load-balance policy dps-lb-policy-default 
    switch(config-load-balance-policy-dps-lb-policy-default)#path-group internet
    ------------------------------------------------------------------------------------------------
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#policy dps-policy-default 
    switch(config-policy-dps-policy-default)#default-match 
    switch(config-policy-default-rule-dps-policy-default)#load-balance dps-lb-policy-default 
    switch(config-policy-default-rule-dps-policy-default)#vrf default
    switch(config-vrf-default)#path-selection-policy dps-policy-default 
    switch(config-vrf-default)#
  7. To configure BGP (Border Gateway Protocol), use the router bgp command. The BGP configuration is essential for distributing IPsec keys, DPS information, and route updates using BGP path attributes and extended communities.
    switch(config)#router bgp 65000
    switch(config-router-bgp)#router-id 192.192.99.1
    switch(config-router-bgp)#maximum-paths 16
    switch(config-router-bgp)#neighbor autovpnedges peer group
    switch(config-router-bgp)#neighbor autovpnedges remote-as 65000
    switch(config-router-bgp)#neighbor autovpnedges update-source loopback 0
    switch(config-router-bgp)#neighbor autovpnedges route-reflector-client 
    switch(config-router-bgp)#neighbor autovpnedges send-community extended 
    switch(config-router-bgp)#neighbor autovpnedges maximum-routes 12000
    switch(config-router-bgp)#neighbor 192.192.101.1 peer group autovpnedges
    switch(config-router-bgp)#neighbor 192.192.101.1 remote-as 65000
    switch(config-router-bgp)#neighbor 192.192.101.1 peer group autovpnedges
    switch(config-router-bgp)#neighbor 192.192.101.1 remote-as 65000
    switch(config-router-bgp)#neighbor 192.192.101.1 peer group autovpnedges
    switch(config-router-bgp)#neighbor 192.192.101.1 remote-as 65000
    switch(config-router-bgp)#
    switch(config-router-bgp)#address-family ipv4
    switch(config-router-bgp-af)#no neighbor autovpnedges activate 
    switch(config-router-bgp-af)#address-family path-selection 
    switch(config-router-bgp-af)#neighbor autovpnedges activate 
    switch(config-router-bgp-af)#neighbor autovpnedges additional-paths receive 
    switch(config-router-bgp-af)#neighbor autovpnedges additional-paths send any 
    switch(config-router-bgp-af)#
  8. To configure Terminattr, follow these steps. However, please note that this configuration is optional and only necessary if the connection to Cloud Vision is required.
    daemon TerminAttr
     exec /usr/bin/TerminAttr -cvaddr=apiserver.cv-staging.corp.arista.io:443 -cvcompression=gzip -taillogs -cvauth=token-secure,/tmp/token-secure -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata,flowtracking/hardware -ingestexclude=/Sysdb/cell/1/agent,/Sysdb/cell/2/agent -disableaaa
     no shutdown
    Note: Please refer the EOS guide https://www.arista.com/en/um-eos for more information about the commands.

EOS Configurations at the Transit Router - AT1

Below are the steps for EOS configuration at the Transit Router - AT1 with examples:

  1. To configure loopback0, utilize the command interface Loopback. This command will set the switch to the interface loopback mode.
    switch(config)#interface loopback 0
    switch(config-if-Lo0)#
    switch(config-if-Lo0)#ip address 192.192.102.1/32
  2. To configure interface VXLAN 1, use the command interface vxlan. This command will set the switch to the interface VXLAN mode for VXLAN 1.
    switch(config)#interface vxlan 1
    switch(config-if-Vx1)#vxlan source-interface loopback 0
    switch(config-if-Vx1)#vxlan udp-port 4789
    switch(config-if-Vx1)#vxlan vrf default vni 101
  3. To configure STUN, use the stun command. This command will set the switch to the STUN mode.
    switch(config)#stun 
    switch(config-stun)#client 
    switch(config-stun-client)#server-profile profile1 
    switch(config-stun-server-profile)#ip address 54.215.30.249
    switch(config-stun-server-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#local interface ethernet 1
    switch(config-internet-interface-Ethernet1)#stun server-profile profile1 
    switch(config-internet-interface-Ethernet1)#peer static router-ip 192.192.99.1
    switch(config-peer-router-ip-192.192.99.1-internet)#ipv4 address 54.215.30.249
    switch(config-peer-router-ip-192.192.99.1-internet)#
  4. To configure dynamic DPS, use the command peer dynamic.
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#peer dynamic 
    switch(config-peer-dynamic-internet)#
  5. The ipsec command is available for IPsecurity configuration.
    switch(config)#ip security 
    switch(config-ipsec)#ike policy ikepolicyCDApp 
    switch(config-ipsec-ike)#local-id 192.192.101.1
    switch(config-ipsec-ike)#sa policy sapolicyCDApp 
    switch(config-ipsec-sa)#profile profileCDApp 
    switch(config-ipsec-profile)#ike-policy ikepolicyCDApp 
    switch(config-ipsec-profile)#sa-policy sapolicyCDApp 
    switch(config-ipsec-profile)#connection start 
    switch(config-ipsec-profile)#shared-key 7
    switch(config-ipsec-profile)#dpd 10 50 clear 
    switch(config-ipsec-profile)#mode transport
    switch(config-ipsec-profile)#key controller
    switch(config-ipsec-profile)#profile profileCDApp
    switch(config-ipsec-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#ipsec profile profileCDApp 
    autovpn-transit1(config-path-group-internet-0)#
  6. Create a DPS1 interface using the interface dps command.
    switch(config)#interface dps 1
    switch(config-if-Dp1)#
  7. Configure the Load Balance policies and the DPS policies as shown in the below example.
    
    Load Balance Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#load-balance policy dps-lb-policy-default 
    switch(config-load-balance-policy-dps-lb-policy-default)#path-group internet
    ------------------------------------------------------------------------------------------------
    DPS Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#policy dps-policy-default 
    switch(config-policy-dps-policy-default)#default-match 
    switch(config-policy-default-rule-dps-policy-default)#load-balance dps-lb-policy-default 
    switch(config-policy-default-rule-dps-policy-default)#vrf default
    switch(config-vrf-default)#path-selection-policy dps-policy-default 
    switch(config-vrf-default)#
  8. To configure BGP (Border Gateway Protocol), use the router bgp command. The BGP configuration is essential for distributing IPsec keys, DPS information, and route updates using BGP path attributes and extended communities.
    switch(config)#router bgp 65000 
    switch(config-router-bgp)#router-id 192.192.101.1
    switch(config-router-bgp)#maximum-paths 16
    switch(config-router-bgp)#neighbor pathfinder peer group 
    switch(config-router-bgp)#neighbor pathfinder remote-as 65000
    switch(config-router-bgp)#neighbor pathfinder update-source loopback 0
    switch(config-router-bgp)#neighbor pathfinder send-community 
    switch(config-router-bgp)#neighbor pathfinder maximum-routes 12000
    switch(config-router-bgp)#neighbor 192.192.99.1 peer group pathfinder 
    switch(config-router-bgp)#address-family ipv4
    switch(config-router-bgp-af)#no neighbor pathfinder activate 
    switch(config-router-bgp-af)#address-family path-selection 
    switch(config-router-bgp-af)#neighbor pathfinder activate 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths receive 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths send any 
    switch(config-router-bgp-af)#
    Note: It is assumed that BGP neighborship configuration has already been done at the Pathfinder.
  9. To configure Terminattr, follow these steps.
    daemon TerminAttr
     exec /usr/bin/TerminAttr -cvaddr=apiserver.cv-staging.corp.arista.io:443 -cvcompression=gzip -taillogs -cvauth=token-secure,/tmp/token-secure -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata,flowtracking/hardware -ingestexclude=/Sysdb/cell/1/agent,/Sysdb/cell/2/agent -disableaaa
     no shutdown
    !
    Note: Please refer the EOS guide https://www.arista.com/en/um-eos for more information about the commands.

Verification Commands between the Pathfinder and Transit Router

The following verification commands are used at the Pathfinder and at the Transit/Edge.

Pathfinder Verification Commands

  • The show stun server bindings command is used to verify the STUN configuration at the Pathfinder.
    switch(config)#show stun server bindings
    Current System Time: Thu Feb 16 08:50:16 2023
    Transaction ID Public AddressAttributes Timeout
    ------------------------ --------------------- ---------- --------
    00000001c0c0650100000000 54.241.134.3:45003 0:09:34
  • The show path-selection paths command is used to verify the DPS configuration.
    switch(config)#show path-selection paths
    PeerPath Group Source DestinationPath Name TypeTC Route State Telemetry StateMTU
    ------------- ---------- -------- ------------ --------- ------- -- ----------------- ---------------- ----
    192.192.101.1 internet 10.0.0.5 54.241.134.3 path1 dynamic 0IPsec established active (0:04:19) 1414

    The Telemetry state comes up and the Route state become IPsec established after the IPsec profile is applied to Router path selection

    As soon as the load-balance and DPS policies are created and applied to a path-group (Internet) that also exists at the edge, the DPS/IPsec dynamic tunnels (Auto-VPN) come up
    Note: It is assumed that BGP neighborship configuration has already been done at the edges.
    switch(config)#show monitor telemetry path characteristics
    PeerPath Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
    ------------- ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
    192.192.101.1 internet path1 0active (0:05:00) 0.480.0130.00.01414
  • The show ip security connection command is used to verify the IPSec configuration at the Pathfinder.
    switch(config)#show ip security connection
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path110.0.0.5 54.241.134.3 Established3 minutes76428 bytes203106 bytes 41 minutes
     487 pkts 1058 pkts

Transit/Edge Verification Commands

  • The show stun client translations command is used to verify the STUN configuration at the Transit.
    switch(config)#show stun client translations
    Current System Time: Thu Feb 16 08:32:47 2023
    Agent Transaction ID Source AddressPublic AddressLast
     Refreshed
    ----- ------------------------ --------------------- --------------------- ------------
    dps 00000001c0c0650100000000 10.0.1.12:450054.241.134.3:4500 0:00:01 ago
    
  • DPS Verification Commands
    • DPS status when no IPsec profile is configured:
      switch(config)#show path-selection paths
      Peer Path Group SourceDestination Path Name Type TC Route State Telemetry State MTU
      ------------ ---------- --------- ------------- --------- ------ -- ----------- --------------- ---
      192.192.99.1 Internet 10.0.1.12 54.215.30.249 path1 static 0resolvedinactive0
    • DPS status when IPsec profile is configured:
      switch(config)#show path-selection paths
      Peer Path Group SourceDestination Path Name Type TC Route State Telemetry StateMTU
      ------------ ---------- --------- ------------- --------- ------ -- ----------------- ---------------- ----
      192.192.99.1 internet 10.0.1.12 54.215.30.249 path1 static 0IPsec established active (0:06:30) 1414
      
      switch(config)#show monitor telemetry path characteristics
      Peer Path Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
      ------------ ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
      192.192.99.1 internet path1 0active (0:13:18) 0.477 0.0030.00.01414
  • IPsec Verification Command
    switch(config)#show ip security connection
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path110.0.1.1254.215.30.249Established10 minutes 213124 bytes 602096 bytes 35 minutes
     1333 pkts2918 pkts

EOS Configurations at the Edge Router - AE2

Below are the steps for EOS configuration at the Edge Router - AE2 with examples:

  1. To configure loopback0, utilize the command interface Loopback. This command will set the switch to the interface loopback mode.
    switch(config)#interface loopback 0
    switch(config-if-Lo0)#
    switch(config-if-Lo0)#ip address 192.192.102.1/32
  2. To configure interface VXLAN 1, use the command interface vxlan. This command will set the switch to the interface VXLAN mode for VXLAN 1.
    switch(config)#interface vxlan 1
    switch(config-if-Vx1)#vxlan source-interface loopback 0
    switch(config-if-Vx1)#vxlan udp-port 4789
    switch(config-if-Vx1)#vxlan vrf default vni 101
  3. To configure STUN, use the stun command. This command will set the switch to the STUN mode.
    switch(config)#stun 
    switch(config-stun)#client 
    switch(config-stun-client)#server-profile profile1 
    switch(config-stun-server-profile)#ip address 54.215.30.249
    switch(config-stun-server-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#local interface ethernet 1
    switch(config-internet-interface-Ethernet1)#stun server-profile profile1 
    switch(config-internet-interface-Ethernet1)#peer static router-ip 192.192.99.1
    switch(config-peer-router-ip-192.192.99.1-internet)#ipv4 address 54.215.30.249
    switch(config-peer-router-ip-192.192.99.1-internet)#
  4. To configure dynamic DPS, use the command peer dynamic.
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#peer dynamic 
    switch(config-peer-dynamic-internet)#
  5. The ipsec command is available for IPsecurity configuration.
    switch(config)#ip security 
    switch(config-ipsec)#ike policy ikepolicyCDApp 
    switch(config-ipsec-ike)#local-id 192.192.102.1
    switch(config-ipsec-ike)#sa policy sapolicyCDApp 
    switch(config-ipsec-sa)#profile profileCDApp 
    switch(config-ipsec-profile)#ike-policy ikepolicyCDApp 
    switch(config-ipsec-profile)#sa-policy sapolicyCDApp 
    switch(config-ipsec-profile)#connection start 
    switch(config-ipsec-profile)#shared-key 7
    switch(config-ipsec-profile)#dpd 10 50 clear 
    switch(config-ipsec-profile)#mode transport
    switch(config-ipsec-profile)#key controller
    switch(config-ipsec-profile)#profile profileCDApp
    switch(config-ipsec-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#ipsec profile profileCDApp 
    autovpn-transit1(config-path-group-internet-0)#
  6. Create a DPS1 interface using the interface dps command.
    switch(config)#interface dps 1
    switch(config-if-Dp1)#
  7. Configure the Load Balance policies and the DPS policies as shown in the below example.
    
    Load Balance Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#load-balance policy dps-lb-policy-default 
    switch(config-load-balance-policy-dps-lb-policy-default)#path-group internet
    ------------------------------------------------------------------------------------------------
    DPS Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#policy dps-policy-default 
    switch(config-policy-dps-policy-default)#default-match 
    switch(config-policy-default-rule-dps-policy-default)#load-balance dps-lb-policy-default 
    switch(config-policy-default-rule-dps-policy-default)#vrf default
    switch(config-vrf-default)#path-selection-policy dps-policy-default 
    switch(config-vrf-default)#
  8. To configure BGP (Border Gateway Protocol), use the router bgp command. The BGP configuration is essential for distributing IPsec keys, DPS information, and route updates using BGP path attributes and extended communities.
    switch(config)#router bgp 65000 
    switch(config-router-bgp)#router-id 192.192.102.1
    switch(config-router-bgp)#maximum-paths 16
    switch(config-router-bgp)#neighbor pathfinder peer group 
    switch(config-router-bgp)#neighbor pathfinder remote-as 65000
    switch(config-router-bgp)#neighbor pathfinder update-source loopback 0
    switch(config-router-bgp)#neighbor pathfinder send-community 
    switch(config-router-bgp)#neighbor pathfinder maximum-routes 12000
    switch(config-router-bgp)#neighbor 192.192.99.1 peer group pathfinder 
    switch(config-router-bgp)#address-family ipv4
    switch(config-router-bgp-af)#no neighbor pathfinder activate 
    switch(config-router-bgp-af)#address-family path-selection 
    switch(config-router-bgp-af)#neighbor pathfinder activate 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths receive 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths send any 
    switch(config-router-bgp-af)#
    Note: It is assumed that BGP neighborship configuration has already been done at the Pathfinder.
  9. To configure Terminattr, follow these steps.
    daemon TerminAttr
     exec /usr/bin/TerminAttr -cvaddr=apiserver.cv-staging.corp.arista.io:443 -cvcompression=gzip -taillogs -cvauth=token-secure,/tmp/token-secure -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata,flowtracking/hardware -ingestexclude=/Sysdb/cell/1/agent,/Sysdb/cell/2/agent -disableaaa
     no shutdown
    !
    Note: Please refer the EOS guide https://www.arista.com/en/um-eos for more information about the commands.

Verification Commands between the Transit Nodes

The following verification commands are used between the Transit Nodes.

At the Transit Node(AT1)

  • DPS Verification Commands
    switch(config)#show path-selection paths
    PeerPath Group SourceDestination Path Name TypeTC Route State Telemetry StateMTU
    ------------- ---------- --------- ------------- --------- ------- -- ----------------- ---------------- ----
    192.192.99.1internet 10.0.1.12 54.215.30.249 path1 static0IPsec established active (0:39:21) 1414
    192.192.102.1 internet 10.0.1.12 13.56.168.187 path2 dynamic 0IPsec established active (0:05:01) 1414
    switch(config)#show monitor telemetry path characteristics
    PeerPath Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
    ------------- ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
    192.192.99.1internet path1 0active (0:41:42) 0.504 0.01 0.00.01414
    192.192.102.1 internet path2 0active (0:07:22) 0.082 0.0050.00.01414
    The commands in bold above (path2) provides the details of the AutoVPN (DPS+IPsec dynamic tunnel) path between the transit router and the edge node (AE2).
  • IPsec Verification Command
    switch(config)#show ip security connection path name Path2 <<< case sensitive >>>
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path210.0.1.1213.56.168.187Established9 minutes179200 bytes 419134 bytes 33 minutes
     1240 pkts2650 pkts

At the Edge Node (AE2)

  • DPS Verification Commands
    switch(config)#show path-selection paths
    PeerPath Group SourceDestination Path Name TypeTC Route State Telemetry StateMTU
    ------------- ---------- --------- ------------- --------- ------- -- ----------------- ---------------- ----
    192.192.99.1internet 10.0.2.12 54.215.30.249 path1 static0IPsec established active (0:17:12) 1414
    192.192.101.1 internet 10.0.2.12 54.241.134.3path2 dynamic 0IPsec established active (0:11:49) 1414
    
    switch(config)#show monitor telemetry path characteristics
    PeerPath Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
    ------------- ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
    192.192.99.1internet path1 0active (0:17:37) 0.478 0.0070.00.01414
    192.192.101.1 internet path2 0active (0:12:13) 0.085 0.0080.00.01414
    The commands in bold above (path2) provides the details of the AutoVPN (aka DPS+IPsec dynamic tunnel) path b/w the transit router and the edge node (AE2)
  • IPsec Verification Command
    switch(config)#show ip security connection path name Path2
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path210.0.2.1254.241.134.3 Established13 minutes 247100 bytes 617770 bytes 30 minutes
     1659 pkts3570 pkts

EOS Configurations at the Edge Router - AE3

Below are the steps for EOS configuration at the Edge Router - AE3 with examples:

  1. To configure loopback0, utilize the command interface Loopback. This command will set the switch to the interface loopback mode.
    switch(config)#interface loopback 0
    switch(config-if-Lo0)#
    switch(config-if-Lo0)#ip address 192.192.103.1/32
  2. To configure interface VXLAN 1, use the command interface vxlan. This command will set the switch to the interface VXLAN mode for VXLAN 1.
    switch(config)#interface vxlan 1
    switch(config-if-Vx1)#vxlan source-interface loopback 0
    switch(config-if-Vx1)#vxlan udp-port 4789
    switch(config-if-Vx1)#vxlan vrf default vni 101
  3. To configure STUN, use the stun command. This command will set the switch to the STUN mode.
    switch(config)#stun 
    switch(config-stun)#client 
    switch(config-stun-client)#server-profile profile1 
    switch(config-stun-server-profile)#ip address 54.215.30.249
    switch(config-stun-server-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#local interface ethernet 1
    switch(config-internet-interface-Ethernet1)#stun server-profile profile1 
    switch(config-internet-interface-Ethernet1)#peer static router-ip 192.192.99.1
    switch(config-peer-router-ip-192.192.99.1-internet)#ipv4 address 54.215.30.249
    switch(config-peer-router-ip-192.192.99.1-internet)#
  4. To configure dynamic DPS, use the command peer dynamic.
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#peer dynamic 
    switch(config-peer-dynamic-internet)#
  5. The ipsec command is available for IPsecurity configuration.
    switch(config)#ip security 
    switch(config-ipsec)#ike policy ikepolicyCDApp 
    switch(config-ipsec-ike)#local-id 192.192.103.1
    switch(config-ipsec-ike)#sa policy sapolicyCDApp 
    switch(config-ipsec-sa)#profile profileCDApp 
    switch(config-ipsec-profile)#ike-policy ikepolicyCDApp 
    switch(config-ipsec-profile)#sa-policy sapolicyCDApp 
    switch(config-ipsec-profile)#connection start 
    switch(config-ipsec-profile)#shared-key 7
    switch(config-ipsec-profile)#dpd 10 50 clear 
    switch(config-ipsec-profile)#mode transport
    switch(config-ipsec-profile)#key controller
    switch(config-ipsec-profile)#profile profileCDApp
    switch(config-ipsec-profile)#router path-selection 
    switch(config-dynamic-path-selection)#path-group internet 
    switch(config-path-group-internet-0)#ipsec profile profileCDApp 
    autovpn-transit1(config-path-group-internet-0)#
  6. Create a DPS1 interface using the interface dps command.
    switch(config)#interface dps 1
    switch(config-if-Dp1)#
  7. Configure the Load Balance policies and the DPS policies as shown in the below example.
    
    Load Balance Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#load-balance policy dps-lb-policy-default 
    switch(config-load-balance-policy-dps-lb-policy-default)#path-group internet
    ------------------------------------------------------------------------------------------------
    DPS Policies
    switch(config)#router path-selection 
    switch(config-dynamic-path-selection)#policy dps-policy-default 
    switch(config-policy-dps-policy-default)#default-match 
    switch(config-policy-default-rule-dps-policy-default)#load-balance dps-lb-policy-default 
    switch(config-policy-default-rule-dps-policy-default)#vrf default
    switch(config-vrf-default)#path-selection-policy dps-policy-default 
    switch(config-vrf-default)#
  8. To configure BGP (Border Gateway Protocol), use the router bgp command. The BGP configuration is essential for distributing IPsec keys, DPS information, and route updates using BGP path attributes and extended communities.
    switch(config)#router bgp 65000 
    switch(config-router-bgp)#router-id 192.192.103.1
    switch(config-router-bgp)#maximum-paths 16
    switch(config-router-bgp)#neighbor pathfinder peer group 
    switch(config-router-bgp)#neighbor pathfinder remote-as 65000
    switch(config-router-bgp)#neighbor pathfinder update-source loopback 0
    switch(config-router-bgp)#neighbor pathfinder send-community 
    switch(config-router-bgp)#neighbor pathfinder maximum-routes 12000
    switch(config-router-bgp)#neighbor 192.192.99.1 peer group pathfinder 
    switch(config-router-bgp)#address-family ipv4
    switch(config-router-bgp-af)#no neighbor pathfinder activate 
    switch(config-router-bgp-af)#address-family path-selection 
    switch(config-router-bgp-af)#neighbor pathfinder activate 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths receive 
    switch(config-router-bgp-af)#neighbor pathfinder additional-paths send any 
    switch(config-router-bgp-af)#
    Note: It is assumed that BGP neighborship configuration has already been done at the Pathfinder.
  9. To configure Terminattr, follow these steps.
    daemon TerminAttr
     exec /usr/bin/TerminAttr -cvaddr=apiserver.cv-staging.corp.arista.io:443 -cvcompression=gzip -taillogs -cvauth=token-secure,/tmp/token-secure -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata,flowtracking/hardware -ingestexclude=/Sysdb/cell/1/agent,/Sysdb/cell/2/agent -disableaaa
     no shutdown
    !
    Note: Please refer the EOS guide https://www.arista.com/en/um-eos for more information about the commands.

Verification Commands between the Edge AE3 and Pathfinder as well as other Transit Nodes

The following verification commands are used between the Edge AE3 and Pathfinder as well as other Transit Nodes.

At the Pathfinder

  • The show stun server bindings command is used to verify the STUN configuration. Theshow stun server bindings command shows the IP/Port bindings for both the edge nodes (AE2, AE3) and the transit node.
    switch(config)#show stun server bindings
    Current System Time: Thu Feb 16 09:55:08 2023
    Transaction ID Public AddressAttributes Timeout
    ------------------------ --------------------- ---------- --------
    00000001c0c0650100000000 54.241.134.3:45003 0:07:47
    00000001c0c0660100000000 13.56.168.187:4500 3 0:09:44
    00000001c0c0670100000000 54.177.169.176:45003 0:08:40
  • DPS Verification Commands
    switch(config)#show path-selection paths
    PeerPath Group Source DestinationPath Name TypeTC Route State Telemetry StateMTU
    ------------- ---------- -------- -------------- --------- ------- -- ----------------- ---------------- ----
    192.192.101.1 internet 10.0.0.5 54.241.134.3 path1 dynamic 0IPsec established active (1:09:21) 1414
    192.192.102.1 internet 10.0.0.5 13.56.168.187path2 dynamic 0IPsec established active (0:40:21) 1414
    192.192.103.1 internet 10.0.0.5 54.177.169.176 path3 dynamic 0IPsec established active (0:11:25) 1414
    
    switch(config)#show monitor telemetry path characteristics
    PeerPath Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
    ------------- ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
    192.192.101.1 internet path1 0active (1:09:45) 0.476 0.0110.00.01414
    192.192.102.1 internet path2 0active (0:40:45) 0.491 0.0050.00.01414
    192.192.103.1 internet path3 0active (0:11:48) 0.479 0.0150.00.01414
  • IPsec Verification Command
    switch(config)#show ip security connection
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path110.0.0.5 54.241.134.3 Established1 hour 1319232 bytes3135419 bytesN/A
     8976 pkts19212 pkts
    Path210.0.0.5 13.56.168.187Established41 minutes 776744 bytes 1874914 bytes1 minute
     5266 pkts11304 pkts
    Path310.0.0.5 54.177.169.176 Established12 minutes 242904 bytes 618431 bytes 30 minutes

At the Edge Node (AE3)

  • The show stun client translations command is used to verify the STUN configuration.
    switch(config)#show stun client translations
    Current System Time: Thu Feb 16 09:50:30 2023
    Agent Transaction ID Source AddressPublic AddressLast
     Refreshed
    ----- ------------------------ --------------------- --------------------- ------------
    dps 00000001c0c0670100000000 10.0.3.13:450054.177.169.176:4500 0:01:51 ago
  • DPS Verification Commands
    switch(config)#show path-selection paths
    PeerPath Group SourceDestination Path Name TypeTC Route State Telemetry StateMTU
    ------------- ---------- --------- ------------- --------- ------- -- ----------------- ---------------- ----
    192.192.99.1internet 10.0.3.13 54.215.30.249 path1 static0IPsec established active (0:02:23) 1414
    192.192.101.1 internet 10.0.3.13 54.241.134.3path2 dynamic 0IPsec established active (0:00:53) 1414
    192.192.102.1 internet 10.0.3.13 13.56.168.187 path3 dynamic 0IPsec established active (0:00:53) 1414
    
    switch(config)#show monitor telemetry path characteristics
    PeerPath Group Path Name TC Tx State Latency(ms) Jitter(ms) Throughput(Mbps) Loss Rate(%) MTU
    ------------- ---------- --------- -- ---------------- ----------- ---------- ---------------- ------------ ----
    192.192.99.1internet path1 0active (0:03:12) 0.476 0.0040.00.01414
    192.192.101.1 internet path2 0active (0:01:42) 0.079 0.0010.00.01414
    192.192.102.1 internet path3 0active (0:01:42) 0.089 0.0070.00.01414
    

    The highlighted commands (path2 and path3) furnish information about the AutoVPN (DPS+IPsec dynamic tunnel) route connecting the transit router to the respective edge node (AE3), as well as between the two edge nodes (AE2 and AE3).

    The Path1 is the static DPS tunnel between the edge (AE3) and the Pathfinder

  • IPsec Verification Command
    switch(config)#show ip security connection
    Tunnel Source Dest Status Uptime InputOutput Rekey Time
    Path110.0.3.1354.215.30.249Established5 minutes130360 bytes 394377 bytes 43 minutes
     766 pkts 1680 pkts
    Path210.0.3.1354.241.134.3 Established4 minutes80312 bytes204900 bytes 40 minutes
     534 pkts 1152 pkts
    Path310.0.3.1313.56.168.187Established4 minutes80312 bytes204838 bytes 38 minutes

 

Pathfinder Redundancy

For optimal reliability, it is advised that each area, whether it's a region or a zone, employs multiple pathfinders (usually two). These pathfinders establish connections with all edge nodes within that area. This redundancy setup safeguards against potential disruptions. If a link between an edge node and one of the pathfinders fails, or if one of the pathfinders becomes unavailable, there is a backup in place.

How does Pathfinder redundancy operate?

  • Pathfinder fulfills three main functions:
    1. It operates as a "traditional" BGP route-reflector for the address families "path-selection" and "evpn."
    2. It functions as a path-computation engine for calculating end-to-end paths, also known as multi-hop paths:
      1. Input: Path characteristics communicated by edge devices via BGP-LS messages.
      2. Output: Multi-hop (MH) paths conveyed to edge devices using BGP-SR-TE messages.
    3. It offers a STUN service to the edge routers.
  • The redundancy of pathfinders is established following BGP best practices:

The redundancy of pathfinders is established following BGP best practices:

  • Deploy a minimum of two Pathfinders.
  • Serve as a route reflector (RR) for edge devices.
  • Act as RR for each other.

This configuration ensures robustness and resilience within the network, as multiple pathfinders collaborate to maintain efficient and reliable paths for data transmission.

Configuration to achieve Pathfinder Redundancy (for AutoVPN only)

  1. Bring up the 2nd pathfinder following the similar configuration (below) as for Pathfinder1.
    daemon TerminAttr
     exec /usr/bin/TerminAttr -cvaddr=apiserver.cv-staging.corp.arista.io:443 -cvcompression=gzip -taillogs -cvauth=token-secure,/tmp/token-secure -smashexcludes=ale,flexCounter,hardware,kni,pulse,strata,flowtracking/hardware -ingestexclude=/Sysdb/cell/1/agent,/Sysdb/cell/2/agent -disableaaa
     no shutdown
    !
    router path-selection
     peer dynamic source stun
     !
     path-group Internet
    local interface Ethernet1
    ipsec profile profileCDApp
    !
     load-balance policy dps-lb-policy-default 
    path-group Internet
     !
     policy dps-policy-default
     	default-match
     		load-balance dps-lb-policy-default
     !
     vrf default
    path-selection-policy dps-policy-default
     !
    ip security 
     ike policy ikepolicyCDApp
    local-id 192.192.99.2
     !
     sa policy sapolicyCDApp 
     !
     profile profileCDApp
    ike-policy ikepolicyCDApp
    sa-policy sapolicyCDApp
    connection start
    shared-key 7 <pre-shared key in clear text> 
    dpd 10 50 clear
    mode transport
     !
    
    
    interface Loopback0
     ip address 192.192.99.2/32 
    !
    interface Vxlan1
     vxlan source-interface Loopback0
     vxlan udp-port 4789
     vxlan vrf default vni 101
    ! 
    interface Dps1
    !
    router bgp 65000
     router-id 192.192.99.2
     maximum-paths 16
     neighbor autovpnEdges peer group
     neighbor autovpnEdges remote-as 65000
     neighbor autovpnEdges update-source Loopback0
     neighbor autovpnEdges route-reflector-client
     neighbor autovpnEdges send-community extended
     neighbor autovpnEdges maximum-routes 12000
     neighbor 192.192.101.1 peer group autovpnEdges
     neighbor 192.192.101.1 remote-as 65000
     neighbor 192.192.102.1 peer group autovpnEdges 
     neighbor 192.192.102.1 remote-as 65000
     neighbor 192.192.103.1 peer group autovpnEdges 
     neighbor 192.192.103.1 remote-as 65000
     !
     address-family ipv4
    no neighbor autovpnEdges activate
     !
     address-family path-selection
    neighbor autovpnEdges activate
    neighbor autovpnEdges additional-paths receive
    neighbor autovpnEdges additional-paths send any
    !
    
    stun
     server
     local-interface Ethernet1
    ! 
  2. Add the Pathfinder1 as BGP neighbor.
    router bgp 65000
     neighbor PATHFINDERS peer group<< adding the two PFs as BGP neighbors >>
     neighbor PATHFINDERS remote-as 65000
     neighbor PATHFINDERS update-source Loopback0
     neighbor PATHFINDERS route-reflector-client
     neighbor PATHFINDERS send-community extended
     neighbor 192.168.99.1 peer group PATHFINDERS
     
     address-family ipv4
    no neighbor autovpnEdges activate 
     !
    
     address-family path-selection
    neighbor PATHFINDERS activate
    neighbor PATHFINDERS additional-paths receive
    neighbor PATHFINDERS additional-paths send any 
    !
  3. Add the same BGP configuration as above on Pathfinder 1 to complete the BGP neighborship between the two Pathfinders.
  4. Create Static DPS connection between the two Pathfinders. A Static DPS connection is created between the two pathfinders for exchanging BGP address families.
    router path-selection
    path-group Internet
    peer static router-ip 192.168.99.1<<<Lo0 IP address of the Pathfinder 1>>>
    ipv4 address 35.211.63.51
    !
    Note: Add the same configuration (while changing the IP to that of PF2 Lo) as above on Pathfinder 1 as well.
  5. Add configuration on all the edges nodes to add PF2 as the second pathfinder.
    1. DPS configuration and STUN profile.
      router path-selection
       path-group Internet 
      local interface Ethernet1
       stun server-profile profile1 profile2 <<< profile2 is the STUN server profile for PF2 >>>
      ! 
      peer static router-ip 192.192.99.1
       ipv4 address 54.x.y.z<<< public IP address for the Pathfinder >>>
      peer static router-ip 192.192.99.2
       ipv4 address 35.x.y.z
      !
    2. BGP configuration.
      router bgp 65000
      neighbor 192.168.99.2 peer group PATHFINDERS<<< adding 2nd Pathfinder at the edge node >>>
      !
    3. STUN configuration.
      stun
       client
      server-profile profile1
       ip address 54.x.y.z
      server-profile profile2<<< adding the stun server profile for the 2nd Pathfinder >>>
       ip address 35.x.y.z
      !

Auto VPN Command

 

stun

The stun command activates the STUN configuration mode for the switch.

The no stun and default stun commands restore the default setting by deleting the corresponding stun command from running-config.

Command Mode

Global Configuration

Command Syntax

stun

no stun

default stun

Parameters
  • client: Configure STUN client
  • server: Configure STUN server

Example

The stun command activates the STUN configuration mode for the switch.
switch(config)#stun
switch(config-stun)#