◄
►

Configure Dynamic Routing with OSPF or BGP

This section discusses how to configure dynamic routing with OSPF or BGP.

Edge learns routes from adjacent routers through OSPF and BGP. It sends the learned routes to the Gateway/Controller. The Gateway/Controller acts like a route reflector and sends the learned routes to other Edge. The Overlay Flow Control (OFC) enables enterprise-wide route visibility and control for ease of programming and for full and partial overlay.

Arista supports Inbound/Outbound filters to OSPF neighbors, OE1/OE2 route types, MD5 authentication. Routes learned through OSPF will be automatically redistributed to the controller hosted in the cloud or on-premise. Support for BGP Inbound/Outbound filters and the filter can be set to Deny, or optionally you can Add/Change the BGP attribute to influence the path selection, i.e. RFC 1998 community, MED, and local preference.

Note: For information about OSPF and BGP Redistribution, see OSPF/BGP Redistribution.

Activate OSPF for Profiles

Open Shortest Path First (OSPF) can be enabled only on a LAN interface as an active or passive interface. The Edge will only advertise the prefix associated with that LAN switch port. To get full OSPF functionality, you must use it in routed interfaces.

OSPF is an interior gateway protocol (IGP) that operates within a single autonomous system (AS).
Note: OSPF is configurable only on the Global Segment.
OSPFv3 is introduced in the 5.2 release and provides support for the following:
  • Support for OSPFv3 is introduced in the SD-WAN Edge for IPv6 underlay routing in addition to existing BGPv6 support. The following is supported:
    • Underlay IPv6 route learning.
    • Redistribution of OSPFv3 routes into overlay/BGP and vice-versa.
    • Support for Overlay Flow Control (OFC).
  • OSPFv3 is implemented with feature parity to OSPFv2 with the following exceptions:
    • Point to Point (P2P) is not supported.
    • BFDv6 with OSPFv3 is not supported.
    • md5 authentication is not available, as OSPFv3 header does not support it.
This section discusses how to configure dynamic routing with OSPFv2 and OSPFv3 along with Route Summarization.
Note: OSPFv2 supports only IPv4. OSPFv3 supports only IPv6 and is available starting with the 5.2 release.
Note: Route Summarization is available starting with the 5.2 release.

To activate OSPF, perform the steps in the procedure below:

  1. In the SD-WAN service of the Enterprise Portal, select Configure > Profiles .

    The Profile page displays.

    Note: Depending upon your login permissions, you might need to select a Customer or Partner first, then select the Configure tab as indicated in next step.
  2. Select a Profile from the list of available Profiles.
  3. Go to the Routing & NAT section in the UI and select the arrow next to OSPF.
  4. In the OSPF Areas section, configure the Redistribution Settings for OSPFv2/v3, BGP Settings, and if applicable, Route Summarization.
    Figure 1. Configure OSPF for Profiles
    Note: OSPFv2 supports only IPv4. OSPFv3 supports only IPv6 and is only available in the 5.2 release.
    Table 1. OSPF Area Option Descriptions
    Option Description
    Redistribution Settings
    Default Route Choose an OSPF route type (O1 or O2) to be used for default route. Default selection for this configuration is "None".
    Advertise Choose either Always or Conditional. (Choosing Always means to Advertise the default route always. Choosing Conditional means to redistribute default route only when Edge learns via overlay or underlay). The “Overlay Prefixes” option must be checked to use the Conditional default route.
    Overlay Prefixes If applicable, check the Overlay Prefixes check box.
    BGP Settings
    BGP To enable injection of BGP routes into OSPF, select the BGP check box. BGP routes can be redistributed into OSPF, so if this is applicable, enter or choose the configuration options as follows:
    Set Metric In the Set Metric text box, enter the metric. (This is the metric that OSPF would put in its external LSAs that it generates from the redistributed routes). The default metric is 20.
    Set Metric Type From the Set Metric Type drop-down menu, choose a metric type. (This is either type E1 or E2 (OSPF External-LSA type)); the default type is E2).

     

  5. In OSPF Areas, select +Add and configure the options, as described in the table below. Add additional areas, if necessary, by selecting +Add. The fields in the table below cannot be overridden at the Edge level.
    Table 2. OSPF Area Add Option Descriptions
    Option Description
    Area ID Select inside the Area ID text box, enter an OSPF area ID.
    Name Select inside the Name text box, enter a descriptive name for your area.
    Type By default, the Normal type is selected. Only Normal type is supported at this time.

     

  6. Next, configure the Interface Settings for OSPF. For configuration details, see either Configure Interface Settings for Profile or Configure Interface Settings for Edges.
    Note: OSPF has to be activated at the Profile level first before you can configure it on Edge interfaces.
  7. If applicable, configure Route Summarization as follows:
    Note: The Route Summarization feature is available starting with the 5.2 release, for an overview and use case for this feature, see Route Summarization.
    1. Navigate to Route Summarization.
    2. Select +Add in the Route Summarization area. A new row is added to the Route Summarization area. Configure Route Summarization, as described in the table.
      Figure 2. Configuring Route Summarization

       

      Table 3. Route Summarization Option Descriptions
      Option Description
      Subnet Enter the IP subnet.
      No Advertise When No Advertise is set, all the external routes (Type-5) that are under this supernet are summarized and have chosen not to advertise it. This means it effectively blocks the whole supernet from advertising to its peer.
      Tag Enter the router Tag value (1-4294967295).
      Metric Type Enter the Metric Type (E1 or E2).
      Metric Enter the advertised metric for this route ((0-16777215).
    3. Add additional routes, if necessary, by selecting +Add. Route Summarization Clone or Delete to copy or delete a Route Summarization.
  8. Select Save Changes.

Route Filters

There are two different types of routing:
  • Inbound Routing includes preferences that can be learned or ignored from OSPF and installed into the Overlay Flow Control.
  • Outbound Routing indicates what prefixes can be redistributed into the OSPF.

Activate OSPF for Edges

Open Shortest Path First (OSPF) can be enabled on a LAN (routed and switched) or a WAN interface. But only a LAN interface can be activated as an active or passive interface. The Edge will only advertise the prefix associated with that LAN switch port. To get full OSPF functionality, you must use it in routed interfaces. After you configure the OSPF settings at the Profile level, all the Edges associated with the Profile will inherit the OSPF configuration from the Profile. However, you cannot override the OSPF configuration settings at the Edge level.

Note: Edges running lower versions (6.0 and below) will not process OSPF configuration in non-global segments even though OSPF configuration is allowed at the Profile level in the Orchestrator.

If needed, you can view the OSPF configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise Portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. Go to the Routing & NAT section and select the arrow next toOSPF.
  4. In the OSPF section, you can view all the inherited OSPF configuration such as OSPF areas, Redistribution settings for OSPFv2/v3, BGP settings, and Route Summarization.
    Figure 3. Configure OSPF for Edges

Configure BGP

You can configure the BGP per segment for a Profile or an Edge. Configuring BGP is available for Underlay Neighbors and Non SD-WAN Neighbors.

Arista supports 4-Byte ASN BGP as follows:
  • As the ASN of Edges.
  • Peer to a neighbor with 4-Byte ASN.
  • Accept 4-Byte ASNs in route advertisements.
See the following sections for configuring BGP for Underlay Neighbors and Non SD-WAN Neighbors:

Configure BGP from Edge to Underlay Neighbors for Profiles

You can configure the BGP per segment at the Profile level as well as at the Edge level. This section provides steps on how to configure BGP with Underlay Neighbors.

Arista supports 4-Byte ASN BGP. See Configure BGP, for additional information.

Note: Route Summarization is new for the 5.2 release. For an overview, use case, and black hole routing details for Route Summarization, see section titled Route Summarization. For configuration details, see the steps below.

To configure BGP:

  1. In the SD-WAN service of the Enterprise Portal, select the Configure tab.
  2. From the left menu, select Profiles. The Profile page displays.
  3. Select a Profile from the list of available Profiles (or Add a Profile if necessary).
  4. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  5. In the BGP area, toggle the radio button from Off to On.
    Figure 4. Configure BGP

     

  6. In the BGP area, enter the local Autonomous System Number (ASN) number in the appropriate text field.
  7. Configure the BGP Settings, as described in the following table below.
    Table 4. BGP Option Descriptions
    Option Description
    Router ID Enter the global BGP router ID. If you do not specify any value, the ID is automatically assigned. If you have configured a loopback Interface for the Edge, the IP address of the loopback Interface will be assigned as the router ID.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Uplink Community Enter the community string to be treated as uplink routes.

    Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes.

    Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
    Enable Graceful Restart check box Note when selecting this check box:

    The local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.

     

  8. Select +Add in the Filter List area to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 5. Add Filter List
  9. In the appropriate text fields, set the rules for the filter, as described in the following table.
    Table 5. Filter Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to be matched with the filter:
    • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the

    corresponding prefix IP address in the Value field.

    • Community: Choose to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the Prosecutes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when Thebes routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None: The attributes of the matching routes remain the same.
    • Local Preference: The matching traffic is routed to the path with the specified local preference.
    • Community: The matching routes are filtered by the specified community string. You can also select the Community Additivecheck box to enable the additive option, which appends the community value to existing communities.
    • Metric: The matching traffic is routed to the path with the specified metric value.

     

  10. Select the plus (+) icon to add more matching rules for the filter. Repeat the procedure to create more BGP filters.
    The configured filters are displayed in the Filter List area.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
  11. Scroll down to the Neighbors area and select +Add.
    Figure 6. Add BGP Neighbors

    Configure the following settings for the IPv4 addressing type, as described in the table below.

    Table 6. Neighbor Option Descriptions
    Option Description
    Neighbor IP Enter the IPv4 address of the BGP neighbor
    ASN Enter the ASN of the neighbor
    Inbound Filter Select an Inbound filer from the drop-down list
    Outbound Filter Select an Outbound filer from the drop-down list
    Note: When overriding and configuring BGP neighbors at the Edge level, any Profile-level filters associated with the neighbors will be removed when you switch the Edge from one profile to another. So at the Edge level, you must make sure to reassociate the filters with the BGP neighbors after switching the Edge profile.

    Additional Options – Select the view all button to configure the following additional settings:

    Figure 7. Configure Additional Options
    Table 7. Additional Option Descriptions
    Option Description
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. The range is from 1 to 255 and the default value is 1.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is inherent by default and this field is not configurable.
    Local IP Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address forth outgoing packets. If you do not enter any value, the IP address of the physical Interface is used as the source IP address.
    Note: For eBGP, this field is available only when Max- hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Uplink Used to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes leant over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, also check "Overlay Prefix Over Uplink" in the Advanced Settings area.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.

    The Default Originate Filter option allows you to control how the default route is advertised. You can choose "None" to advertise the default route without any modification, "Same as Outbound" to use the same filter as the outbound filter or select a specific filter from the list. The chosen filter is then applied to the default route, modifying its parameters accordingly.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.

     

  12. Select the Plus (+) icon to add more BGP neighbors.
    Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For additional information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  13. Scroll down to Advanced Settings and select the down arrow to open the Advanced Settings section.
    Figure 8. Configure Advanced Settings
  14. Configure the following advanced settings, as indicated in the following table, which are globally applied to all the BGP neighbors with IPv4 addresses.
    Figure 9. Advanced Settings Configuration
    Table 8. Advanced Settings Option Descriptions
    Option Description
    Overlay Prefix Select the check box to redistribute the prefixes learned from the overlay.
    For example, when a Spoke is connected to primary and secondary Hub or Hub Cluster, the Spoke's subnets are redistributed by primary and secondary Hub or Hub Cluster to their neighbor with metric (MED) 33 and 34 respectively. You must configure "bgp always-compare-med" in the neighbor router for symmetric routing.
    Note: Prior to 5.1, the advertised MED values were starting from eight. From release 5.1 and later, the MED values advertised by HUB starts from 33.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to deactivate AS-PATH Carry Over. In certain topologies, deactivating AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Warning: When the AS-PATH Carry Over is deactivated, tune your network to avoid routing loops.
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    The Default Originate Filter option allows you to control how the default route is advertised. You can choose "None" to advertise the default route without any modification, "Same as Outbound" to use the same filter as the outbound filter or select a specific filter from the list. The chosen filter is then applied to the default route, modifying its parameters accordingly.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address in IPv4 format that BGP will be advertising to the peers. Select the plus + icon to add more network addresses.

    When you enable the Default Route option, the BGP routes are advertised based on the Default Route selection globally and per BGP neighbor, as shown in the following table:

    Table 9. Default Route Selection
    Global Per BGP Neighbor Advertising Options
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.

     

  15. Select the IPv6 tab to configure the BGP settings for IPv6 addresses. Enter a valid IPv6 address of the BGP neighbor in the Neighbor IP field. The BGP peer for IPv6 supports the following address format:
    • Global unicast address (2001:CAFE:0:2::1)
    • Unique Local address (FD00::1234:BEFF:ACE:E0A4)
  16. Configure the other settings as required.
    Note: The Local IP address configuration is not available for IPv6 address type.
  17. Select Advanced to configure the following advanced settings, which are globally applied to all the BGP neighbors with IPv6 addresses.
    Table 10. Advanced Settings Option Descriptions
    Option Description
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    When you select the Default Route option, the Advertise option is available as Conditional.

    The Default Originate Filter option allows you to control how the default route is advertised. You can choose "None" to advertise the default route without any modification, "Same as Outbound" to use the same filter as the outbound filter or select a specific filter from the list. The chosen filter is then be applied to the default route, modifying its parameters accordingly.
    Networks Enter the network address in IPv6 format that BGP will be advertising to the peers. Select the Plus + icon to add more network addresses.

     

  18. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 10. Add Route Summarization
    1. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
    2. Under the AS Set column, select the Yes check box if applicable.
    3. Under the Summary Only column, select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  19. Select Save Changes.
    Note: When you configure BGP settings for a profile, the configuration settings are automatically applied to the SD-WAN Edges that are associated with the profile.

    You can also configure BGP for Non SD-WAN Destination Neighbors in an Edge. For additional information, see Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors.

Configure BGP from Edge to Underlay Neighbors for Edges

You can override the inherited Profile settings at the Edge level when configuring BGP from the Edge to Underlay Neighbors.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges . The Edges page displays the existing Edges.
  2. Select the link to an Edge or select the View link in the Device column of the Edge.
  3. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  4. The BGP settings configured for the associated Profile are displayed. If required, you can select the Override check box and modify the BGP Settings.
    Note: When overriding and configuring BGP neighbors at the Edge level, any Profile-level filters associated with the neighbors will be removed when you switch the Edge from one profile to another. So at the Edge level, you must make sure to reassociate the filters with the BGP neighbors after switching the Edge profile.
  5. In addition to the BGP settings configured for a Profile, you can select an Edge Interface configured in the segment as the source Interface for BGP. For the IPv4 address type, you can select only the Loopback Interface as Source Interface and for the IPv6 address type, you can select any Edge Interface as the Source Interface.
    This field is available:
    • Only when you choose to override the BGP Settings at the Edge level.
    • For eBGP, only when Max-hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Important:
    • You cannot select an Edge Interface if you have already configured a local IP address in the Local IP field.
    • You cannot configure a local IP address if you have selected an Edge Interface in the Source Interface drop-down list.
  6. Select Save Changes in the Device screen to save the modified configuration.

Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors

The Non SD-WAN BGP Neighbors configuration is not applicable at Profile level. You can configure the NSD Neighbors only at the Edge level.

BGP is used to establish the BGP neighborship over the IPSec tunnels to the Non SD-WAN Sites. Direct IPSec tunnels are used for establishing a secure communication between the SD-WAN Edge and the Non SD-WAN Destination (NSD). In previous releases, Arista supported NSD tunnels from the SD-WAN Edge with the ability to add NVS static routes. In the 4.3 release, this functionality is extended to support BGP over IPSec to the NSD endpoint for a route-based VPN.

VeloCloud SD-WAN supports 4-Byte ASN BGP. See Configure BGP, for additional information.
Note: The Azure vWAN Automation from Edge feature is not compatible with BGP over IPSec. This is because only static routes are supported when automating connectivity from an Edge to an Azure vWAN.

Use Case 1: BGP Over IPSec from an Edge to an Azure VPN

Each Azure VPN gateway allocates one set of public Virtual Public IPs (VIP) for a branch Edge to form IPSec tunnels. Similarly, Azure also allocates one internal private subnet and assigns one internal IP per VIP. This internal tunnel-ip (peer tunnel-ip) will be used for creating BGP peering with the Azure Gateway.

Azure has a restriction that the BGP peer IP (Edge's local tunnel-ip) shouldn't be in the same connected subnet or 169.x.x.x subnet, and therefore we need to support multi-hop BGP on the Edge. In BGP terminology, the local tunnel-ip maps to BGP source address and peer tunnel-ip maps to neighbor/peer address. We need to form a mesh of BGP connections- one per NSD tunnel so that the return traffic from the NVS could be load-balanced (flow-based)- design on the Azure Gateway side. In the below diagram for the physical Edge, we have two public WAN links and so four tunnels to an Azure Gateway. Each tunnel is associated with one BGP connection uniquely identified by the local tunnel_ip and remote peer tunnel_ip. On the Virtual Edge, the only difference is that we have one public WAN link and a maximum of two tunnels and two BGP sessions to the Azure Gateway.

Figure 11. BGP Over IPSec from an Edge to an Azure VPN
Note: When an SD-WAN Edge is connected to the same Azure end-point using multiple WAN links, there is a maximum of two NSD-BGP neighbors that could be configured (since remote end has only two public_ips and two NSD-BGP peer_ips). Both NSD-BGP neighbors can be configured on the same link (primary/secondary tunnel), or tunnels on different links. If a customer attempts to configure more than two NSD-BGP neighbors and configure the same NSD-BGP peer_ip on more than one tunnel, the last configured BGP nbr_ip + local_ip would be on the SD-WAN Edge and Free Range Routing (FRR).

Use Case 2: BGP Over IPSec from Edge to AWS VPN/Transit Gateway

Unlike Azure, AWS VPN Gateway allocates one set of public VIPs per link to a branch Edge. The total sets of public IPs allocated to a branch Edge from an AWS Gateway will be equal to the number of Edge public WAN links that will connect to the AWS VPN Gateway. Similarly, a /30 internal/private subnet would be allocated per tunnel, which are used for BGP peering on that tunnel. These IPs could be manually overridden in AWS Gateway configuration to ensure they are unique across different availability zones.

Similar to the Azure use-case, the Edge will form a mesh of BGP connections- one per tunnel to the AWS gateway. This will allow load-balancing of the return traffic from the AWS VPN Gateway- design on the AWS side. In the diagram below, for the physical Edge, the AWS Gateway allocates one set of public IPs and one set of tunnel-ips (/30) for each Edge WAN link. There are a total of four tunnels, but terminate in different public IPs on the AWS Gateway and four BGP connections.

Figure 12. BGP Over IPSec from Edge to AWS VPN/Transit Gateway

Use Case 3: Edge Connecting to Both AWS and Azure VPN Gateways (Hybrid Cloud)

One branch Edge could be connected to both Azure Gateway and AWS Gateway for redundancy purposes or some workloads/apps hosted in one cloud provider while other workloads/apps hosted in a different cloud provider. Regardless of the use-case, the Edge always establishes one BGP session per tunnel and propagates the routes between SD-WAN and IaaS. The diagram below is an example of one branch Edge connected to both Azure and AWS clouds.

Figure 13. Edge Connecting to Both AWS and Azure VPN Gateways

Use Case 4: Hub Cluster Connecting to Azure/AWS Transit Gateways

The Hub cluster members can form IPSec tunnels to the Azure/AWS transit Gateways and leverage the transit Gateways as Layer 3 for routing traffic between different VPCs. Without the native BGP over IPSec functionality on Hub, the Hub needs to connect to an L3 router (Cisco CSR widely used here) using native BGP and the L3 router forming a mesh of BGP over IPSec tunnels with different VPCs. L3 router serves as a transit end-point between different VPCs. Usecase-1 (left diagram below): Use Hub as a transit node between different VPCs in different Availability Zones (AZ) so that one VPC can talk to another VPC. Usecase-2 (right diagram below): Connect all Hubs in the cluster directly to a cloud transit gateway and can use the cloud gateway as a PE(L3) router for routes distribution between cluster members. In both use-cases, without the support for BGP over IPSec on Hub, hub connects to an L3 router like CSR using native BGP and CSR peers with transit/VPC gateway using BGP over IPSec.

Figure 14. Hub Cluster Connecting to Azure/AWS Transit Gateways

Use Case 5: Support Transit Functionality in Cloud Providers without Native Support

Some cloud providers like Google Cloud and AliCloud do not have native support for transit functionality (no transit Gateways), and with the support for BGP over IPSec, can rely on SD-WAN Edge/Hub deployed in the cloud to achieve the transit functionality between different VPCs/VNETs. Without the BGP over IPSec support, you must use an L3 router like CSR (solution (2)) to achieve the transit functionality.

Note: Prior to the 4.3 release, for customers who have reachability to the same NVS-Static destination via NVS-From-Gateway and NVS-From-Edge, the traffic from other branch SD-WAN Edges will prefer the path via NVS-Gateway. When customers upgrade their network to the 4.3 release or later, this traffic path from other branch- SD-WAN Edges will prefer the path via the NVS-Edge. Therefore, customers must update the NVS-Static-Destination’s metric of the NSD-Edge and the NSD-Gateway as per their traffic path preference.

To enable BGP with Non SD-WAN neighbors:

  1. In the SD-WAN service of the Enterprise Portal, select Configure.
  2. From the left menu, select Edges. The Edges page displays.
  3. Select an Edge from the list of available Edges.
  4. Go to the Routing & NAT section in the UI and select the arrow next to BGP.
  5. In the BGP area, check the Override check box and toggle the radio button from Off to On.
    Figure 15. Override BGP Settings
  6. In the BGP Editor window, enter the local Autonomous System Number (ASN) and then configure the following in the BGP Settings section.
    Table 11. BGP Settings Field Descriptions
    Option Description
    Router ID Enter the global BGP router ID. If you do not specify any value, the ID is automatically assigned. If you have configured a loopback Interface for the Edge, the IP address of the loopback Interface will be assigned as the router ID.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Uplink Community Enter the community string to be treated as uplink routes.

    Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes.

    Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
    Enable Graceful Restart check box Please note when selecting this check box:

    The local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.
  7. Select +Add in the Filter List area to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 16. Add Filter
    In the appropriate text fields, set the rules for the filter, as described in the following table, and select OK.
    Table 12. Add Filter Field Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to be matched with the filter:
    • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
    • Community: Choose to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None: The attributes of the matching routes remain the same.
    • Local Preference: The matching traffic is routed to the path with the specified local preference.
    • Community: The matching routes are filtered by the specified community string. You can also select the Community Additive check box to enable the additive option, which appends the community value to existing communities.
    • Metric: The matching traffic is routed to the path with the specified metric value.
    • AS-Path-Prepend: Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.
    To add more matching rules to the filter, select the Plus ( +) icon.
    The configured filters are displayed in the BGP Editor window.
  8. Configure Underlay Neighbors for IPv4 and IPv6 addresses, as required. For additional information, see Configure BGP from Edge to Underlay Neighbors for Profiles.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
  9. In the NSD Neighbors section, configure the following settings, as described in the following table.
    Table 13. NSD Neighbors Field Descriptions
    Option Description
    NSD Name Select the NSD Name from the drop-down list. The NSDs already configured in the Branch to Non SD-WAN Destination via Edge area of the Orchestrator are displayed in the drop-down list.
    Link Name Choose the name of the WAN link associated with the NSD neighbor.
    Tunnel Type Choose the tunnel type of the Peer as Primary or Secondary.
    Neighbor IP Enter the IP address of the NSD neighbor.
    ASN Enter the ASN for the NSD neighbor.
    Inbound Filter Select an Inbound filer from the drop-down list.
    Outbound Filter Select an Outbound filer from the drop-down list.
    Additional Options – Select the view all link to configure the following additional settings:
    Uplink Used to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes leant over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, select the Overlay Prefix Over Uplink check box in the Advanced Settings.
    Local IP Local IP is mandatory for configuring Non SD-WAN Neighbors. Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
    Note: When upgrading to the 5.1 release, any max-hop value of 1 will automatically be updated to a max-hop value of 2.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is deactivated by default and this field is not configurable.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Note: Single-hop BFD session is not supported for BGP over IPSec with NSD Neighbors. However, multi-hop BFD is supported. Local IP is mandatory for NSD-BGP sessions on the SD-WAN Edge. The SD-WAN Edge handles only the connected Interface IPs as a single-hop BFD.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.
    Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and interface. Until the recursive resolution happens, the recursive routes point to an intermediate interface. For additional information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  10. Select Advanced to configure the following settings.
    Note: Advanced Settings are shared across both the underlay BGP neighbors and NSD BGP neighbors.
    Table 14. Advanced BGP Settings Descriptions
    Option Description
    Overlay Prefix Select the check box to redistribute the prefixes learned from the overlay.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to turn off AS-PATH Carry Over. In certain topologies, turning off AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Warning: When the AS-PATH Carry Over is turned off, tune your network to avoid routing loops.
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    When you select the Default Route option, the Advertise option is available as Conditional.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address that BGP will be advertising to the peers. Select the Plus ( +) Icon to add more network addresses.

    When you enable the Default Route option, the BGP routes are advertised based on the Default Route selection globally and per BGP neighbor, as shown in the following table.

    Table 15. Default Route Selection
    Global Per BGP Neighbor Advertising Options
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.
  11. Select OK to save the configured filters and NSD Neighbors.
  12. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 17. Add Route Summarization
    1. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
    2. Under the AS Set column, select the Yes check box if applicable.
    3. Under the Summary Only column, select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  13. Select Save Changes.

You can also configure BGP from Edge to underlay neighbors. For additional information, see Configure BGP from Edge to Underlay Neighbors for Edges.

Configure BGP Over IPsec from Gateways

Ensure that you have configured the following:

 

Note: It is recommended to turn on Distributed Cost Calculation for best performance and scaling when using BGP over IPsec via Gateway. The Distributed Cost Calculation is supported starting from Release 3.4.0.

For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.

You can configure BGP Settings for Gateways over IPsec tunnels.

Only eBGP is supported with BGP over IPsec.
Note: It is recommended to use eBGP between SDWAN Gateway and NSD sites. If iBGP is used, applying local preference does not work with outbound filter. In that case, customer must choose metric or AS path prepend options to achieve desirable routing.

Arista VeloCloud SD-WAN allows Enterprise users to define and configure a Non SD-WAN Destination instance in order to establish a secure IPsec tunnel to a Non SD-WAN Destination through an SD-WAN Gateway.

Note: For the 5.2 release, when multiple NSDs are configured for the same segment, the same set of summary route configurations must be present across all NSDs.
Note: The Azure vWAN Automation from Gateway feature is not compatible with BGP over IPsec. This is because only static routes are supported when automating connectivity from a Gateway to an Azure vWAN.
Important: DCC is mandatory for ECMP to work properly.
  1. Go to Configure > Network Services , and then under Non SD-WAN Destinations, expand Non SD-WAN Destinations via Gateway.
    Note: The New NSD via Gateway option appears only when there are no items in the table. Follow Steps 2 and 3 to create a new Non SD-WAN Destination.
    Figure 18. Configure Non SD-WAN Destinations
  2. Select +New to create a new Non SD-WAN Destination. The Non SD-WAN Destinations via Gateway dialog appears.
    Figure 19. Create New Non SD-WAN Destinations via Gateway
  3. In the Non SD-WAN Destinations via Gateway area, configure the required fields as described in the following table.
    Table 16. Non SD-WAN Destinations via Gateway Field Descriptions
    Option Description
    Name Enter a name for the Non SD-WAN Destination in the text box.
    Type Select an IPsec tunnel type from the drop-down menu.
    Tunnel Mode Active/ Hot-Standby mode supports to set up a maximum of 2 tunnel endpoints or Gateways.
    Active/Active mode supports to set up a maximum of 4 tunnel endpoints or Gateways. All Active tunnels can send and receive traffic through ECMP.
    VPN Gateway 1 Enter a valid IP address
    VPN Gateway 2 Enter a valid IP address. This field is optional
    The NSD is created and it appears in the Non SD-WAN Destinations via Gateway table, as shown in the following screenshot.
    Figure 20. List of configured NSDs via Gateway
  4. In the Non SD-WAN Destination via Gateway area, slide the grey bar to the far right to the BGP column.
    1. Select the Edit link under the BGP column, the Edit BGP dialog displays.

      If the Edit link does not display under the BGP column, see Configure a Tunnel Between a Branch and a Non SD-WAN Destinations via Edge to enable an Edge to Non SD-WAN via Gateway.

    2. Toggle the BGP Activated radio button to the right to turn it green.
    3. Select +Add to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
      Table 17. Create Filter Field Descriptions
      Option Description
      Filter Name Enter a descriptive name for the BGP filter.
      Match Type and Value Choose the type of the routes to be matched with the filter:
      • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the

      corresponding prefix IP address in the Value field.

      • Community: Choose to match with a community and enter the community string in the Value field.
      Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
      Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
      Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
      • None: The attributes of the matching routes remain the same.
      • Local Preference: The matching traffic is routed to the path with the specified local preference.
      • Community: The matching routes are filtered by the specified community string. You can also select the Community Additive check box to enable the additive option, which appends the community value to existing communities.
      • Metric: The matching traffic is routed to the path with the specified metric value.
      • AS-Path-Prepend: Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.
    4. Select the plus (+) icon to add more matching rules for the filter. Repeat the procedure to create more filters.

      The configured filters are displayed in the Filter List area.

      Figure 21. BGP Filter List
  5. In the Edit BGP window, configure the BGP settings for the Primary and Secondary Gateways.
    Figure 22. Edit BGP Neighbors Gateway Settings
    Note: These BGP neighbors are assigned to their respective tunnels exclusively for neighborship establishment and subsequent control exchanges, ensuring these communication occurs solely over the designated tunnels.
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Note: For a customer deployment where a Non SD-WAN Destination (NSD) via Gateway is configured to use redundant tunnels, if the Primary and Secondary Gateways advertise a prefix with an equal AS path to the Primary and Secondary NSD tunnels, the Primary NSD tunnel will prefer a redundant Gateway path over the Primary Gateway. The impact of the Primary NSD over Gateway tunnel preferring the redundant Gateway path over the Primary Gateway is experienced only for return traffic to the Gateway from the NSD.

    If you do not want your BGP router to prefer the redundant Gateway, the workaround is to configure AS-PATH prepend and set the metric filter to a higher (3 or more) metric for the advertised prefix in the redundant Gateway. Doing this ensures the NSD's primary tunnel chooses the Primary Gateway for return traffic.

    1. In the Primary Cloud Gateway section, enter the local ASN and the Router ID.
    2. Scroll down to the Neighbors area and select +Add.
    3. Configure the following settings in the Neighbors area, as described in the following table, and select Save Changes.
      Table 18. BGP Gateway Option Descriptions
      Option Description
      Local ASN Enter the local Autonomous System Number (ASN)
      Router ID Enter the BGP Router ID
      Neighbor IP Enter the IP address of the BGP neighbor
      ASN Enter the ASN of the neighbor
      Inbound Filter Select an Inbound filer from the drop-down list
      Outbound Filter Select an Outbound filer from the drop-down list
      Additional Options – Select the view all link to configure the following additional settings:
      Local IP Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
      Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
      Note: When upgrading to the 5.1 release, any max-hop value of 1 will automatically be updated to a max-hop value of 2.
      Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different.
      Allow AS Select the check box to allow the BGP routes to be received and processed even if the Gateway detects its own ASN in the AS-Path.
      Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
      Enable BFD Enables subscription to the existing BFD session for the BGP neighbor.
      Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 1 to 65535 seconds. The default value is 60 seconds.
      Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 1 to 65535 seconds. The default value is 180 seconds.
      Connect Enter the time interval to try a new TCP connection with the peer if it detects that the TCP session is not passive. The default value is 120 seconds.
      MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and is used as a security guard for BGP peering.
      MD5 Password Enter a password for MD5 authentication.
      Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.

      The configured Neighbors are displayed in the Neighbors area.

      Figure 23. Configured BGP Neighbors List
      Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For additional information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  6. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 24. Add Route Summarization

     

    Table 19. Route Summarization Field Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Subnet Enter the IP subnet.
    AS Set Generate AS set path information from the summarized routes (while advertising the summary route to the peer). Under the AS Set column, select the Yes check box if applicable.
    Summary Only Select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  7. Select Save Changes.
    Note:
    • Only for Gateways running version 6.0 or later have an option to configure up to 4 tunnels based on VPN type. In addition, those tunnels destined to be a Non SD-WAN gateways can operate in either AA or A-HS mode to achieve load sharing/bearing preferences of the user.
    • For gateways running version less than 6.0, all active-active configurations are interpreted as active-hotstandby with tunnel 1 being active and tunnel 2 being hot-standby.

Monitor BGP Events

You can view the events related to the BGP sessions.

  1. In the SD-WAN service of the Enterprise Portal, click Monitor > Events .
  2. To view the events related to BGP, you can use the filter option. Select the Filter Icon next to the Search option and choose to filter the details by different categories.
    Figure 25. Monitor BGP Events
    The following are the events related to BGP:
    • BGP session established to Gateway neighbor
    • BGP session established to Edge neighbor
    • BGPv6 session established to Edge neighbor
    • Edge BGP neighbor unavailable
    • Edge BGPv6 neighbor unavailable
    • Gateway BGP neighbor unavailable

Troubleshooting BGP Settings

You can run Remote Diagnostics tests to view the logs of the BGP sessions and use the log information for troubleshooting purposes.

To run the tests for BGP:

  1. In the SD-WAN service of the Enterprise Portal, click Diagnostics > Remote Diagnostics .
  2. The Remote Diagnostics page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BGP sessions, scroll to the following sections and run the tests:
    • Troubleshoot BGP- List BGP Redistributed Routes – Run this test to view routes redistributed to BGP neighbors.
    • Troubleshoot BGP- List BGP Routes – Run this test to view the BGP routes from neighbors. You can enter IPv4 or IPv6 prefix to view specific BGP routes or leave the prefix empty to view all the BGP routes.
    • Troubleshoot BGP- List Routes per Prefix – Run this test to view all the Overlay and Underlay routes for a specific IPv4 or IPv6 prefix and the related details.
    • Troubleshoot BGP- Show BGP Neighbor Advertised Routes – Run this test to view the BGP routes advertised to a neighbor.
    • Troubleshoot BGP- Show BGP Neighbor Learned Routes – Run this test to view all the accepted BGP routes learned from a neighbor after filters.
    • Troubleshoot BGP- Show BGP Neighbor Received Routes – Run this test to view all the BGP routes learned from a neighbor before filters.
    • Troubleshoot BGP- Show BGP Neighbor details – Run this test to view the details of BGP neighbor.
    • Troubleshoot BGP- Show BGP Routes per Prefix – Run this test to view all the BGP routes and their attributes for the specified prefix.
    • Troubleshoot BGP- Show BGP Summary – Run this test to view the existing BGP neighbor and received routes.
    • Troubleshoot BGP- Show BGP Table – Run this test to view the BGP table.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Advertised Routes – Run this test to view the BGPv6 routes advertised to a neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Learned Routes – Run this test to view all the accepted BGPv6 routes learned from a neighbor after filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Received Routes – Run this test to view all the BGPv6 routes received from a neighbor before filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor details – Run this test to view the details of BGPv6 neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Routes per Prefix – Run this test to view all the BGPv6 routes for the prefix and their attributes.
    • Troubleshoot BGPv6- Show BGPv6 Summary – Run this test to view the existing BGPv6 neighbor and received routes.
    • Troubleshoot BGPv6- Show BGPv6 Table – Run this test to view the details of BGPv6 table.

    For additional information about all the supported BGP related Remote Diagnostics tests, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide.

OSPF/BGP Redistribution

Each of routing protocols OSPF and BGP may be enabled independently and the prior model of allowing only one routing protocol to be enabled on the system has been removed with this release. This release also allows the possibility of redistributing OSPF into BGP or BGP into OSPF (or both simultaneously), along with other possible route sources like prefixes learnt over the overlay, connected routes, static routes, etc.

In addition, with release 3.2, we are standardizing the redistribution behavior along more traditional lines (similar to that in other routing vendors). For example, if there is more than one route available for the same prefix, then only the best route for that prefix in the system RIB will be redistributed to the destination protocol if the configuration in the destination protocol allows redistribution for that route type.

Consider, as an example, redistribution of the prefix 192.168.1.0/24 into BGP. Let's say routes to the prefix 192.168.1.0/24 are locally available, learned from OSPF and separately learned as an Overlay prefix. Let's further assume that between the OFC flow ordering for the prefix, and route metrics, and route preference the OSPF route ranks above (is better than) the learned overlay route for that same prefix. Then, the OSPF route will be redistributed into BGP if OSPF redistribution has been turned on in BGP. Note that since the overlay learned prefix is not the best route for that prefix in the system RIB, it will not be redistributed into BGP even if the redistribution of overlay prefixes has been turned on in BGP.

In cases like the above, in order to facilitate the redistribution of the best route for a prefix into a given destination protocol, the user can enable redistribution for the specific route type that is the best route in the system.

Alternately, if the user prefers a different route source for that prefix to be redistributed into the destination protocol, the user can control the relative precedence of the route in the system RIB using the Overlay Flow Control facility provided by the management interface, or by varying the route metric.

OSPF/BGP Redistribution Metric Calculation

Starting with the 5.2 release, the route redistribution metric calculation has changed. When a route is redistributed from the Overlay to OSPF/BGP, the redistribution metric is calculated by taking the original route metric and adding the transit metric:
  • The transit metric is (0) if the route is learned from a directly connected Edge.
  • The transit metric is (90) if the route is learned via a Gateway.
  • The transit metric is (32 + hub's order value) if the route is learned via a Hub Edge.
For OSPF External Type-1 (OE1) routes, this is the final metric. For OSPF External Type-2 (OE2) routes, it will add up the non-preferred metric constant (8388607). This is why there is a very high metric value for an OE2 route type on Edge peers.

For BGP, this implies that the BGP MED value advertised by Hub Edges, which previously started from 9, 10, 11, and so forth, now starts from 33, 34, 35, and so forth.

See the following for additional information:

BFD Settings

Bidirectional Forwarding Detection (BFD) is a simple Hello protocol that is similar to detection components of well-known routing protocols. A pair of systems transmit BFD packets periodically over each path between the two systems, and if a system stops receiving BFD packets for long enough, the neighboring system is assumed to have failed.

A BFD session is established based on the needs of the application that would use BFD. The user has to explicitly configure the address and parameters for the BFD session and the subscribers/applications (BGP/OSPF) of the session, as there is no discovery mechanism in BFD.

Routing protocols like BGP or OSPF exchange the learned routes between Edges and Routers. These protocols exchange routes and detect route failures using their own mechanism. Generally, route failures are detected based on the keepalive mechanism where one entity echoes other entity on a frequent configured interval, that is the keepalive time. These routing protocols have higher keepalive timers which results in longer duration to detect the route failures. BFD detects route failures between two connected entities faster with low overhead on detection of failures.

The following are the advantages of implementing BFD with routing protocols.
  • Fast route failure detection with low re-convergence time.
  • Less overhead in route failure detection.
  • Uniform rate of route failure detection across routing protocols.

BFD can be defined as a simple service. The service primitives provided by BFD are to create, destroy, and modify a session, given the destination address and other parameters. BFD in return provides a signal to the clients indicating when the BFD session goes up or down.

There are two operating modes to BFD, asynchronous mode and demand mode. Arista VeloCloud supports asynchronous mode. In this mode, the systems periodically send BFD control packets to other systems and if several packets in a row are not received by a system, the session is declared to be down.
Note: BFD Echo mode is not supported.
Arista VeloCloud supports BFD for the following routing protocols:
  • BGP on Edges and Partner Gateways
  • OSPF on Edges

Configure BFD for Profiles

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities.

To configure a BFD session for Profiles:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device icon for a profile, or select a profile and select the Device tab
    Note: The Device tab is normally the default tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to the BDF area to open it.
  4. Select the BDF slider to ON position.
    Figure 26. Configure BFD for Profiles
  5. Configure the following settings, as described in the following table.
    Table 20. BFD for Profile Field Descriptions
    Peer Address Enter the IPv4 address of the remote peer to initiate a BFD session.
    Local Address Enter a locally configured IPv4 address for the peer listener. This address is used to send the packets.
    Note: You can select the IPv6 tab to configure IPv6 addresses for the remote peer and the peer listener.
    For IPv6, the local and peer addresses support only the following format:
    • IPv6 global unicast address (2001:CAFE:0:2::1)
    • IPv6 unique local address (FD00::1234:BEFF:ACE:E0A4)
    Multihop Select the check box to enable multi-hop for the BFD session. While BFD on Edge and Gateway supports directly connected BFD Sessions, you need to configure BFD peers in conjunction with multi-hop BGP neighbors. The multi-hop BFD option supports this requirement.

    Multihop must be enabled for the BFD sessions for NSD-BGP-Neighbors.
    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.

     

  6. Select the Plus (+) icon to add details of more peers.
  7. Select Save Changes.

    When you configure BFD rules for a profile, the rules are automatically applied to the Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for Edges for additional information.

VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configure BFD for Edges

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities. Once you have configured BFD rules for a Profile, the rules are automatically applied to the Edges that are associated with the profile. Optionally, you can override the inherited settings at the Edge level.

To override the configuration for a specific Edge:

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. In the Device tab, scroll down to the BFD Rules section.
  4. Select the Override check box to modify the BFD configuration settings for the selected Edge.
    Figure 27. Override BFD Rules at Edge Level
  5. Select Save Changes.

VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configure BFD for BGP for Profiles

You can configure BFD for BGP on SD-WAN Profiles.

By default, BFD is deactivated in BGP neighbor. You can enable BFD for a BGP session to subscribe to BFD session updates.

Enabling BFD for a BGP neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure describes how to enable BFD for an already configured BGP session on an Edge. To configure BGP settings, see Configure BGP from Edge to Underlay Neighbors for Profiles.

To enable BFD for BGP on partner Gateways, you must be an Operator super user. For additional information, see the Configure Partner Handoff section in the VeloCloud SD-WAN Operator Guide.

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device icon for a profile, or select a profile and select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to BGP to open the BGP section.
  4. Select the slider to ON position.
  5. In the BGP Editor window, select view all in the Additional Options column for a BGP neighbor and select the Enable BFD check box. You can enable a BFD subscription for multiple BGP neighbors, including NSD Neighbors.
    Multihop must be configured as Multihop BFD for NSD BGP Neighbors. For additional information about NSD Neighbors, see Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors.
    Figure 28. Configure BFD for BGP for Profiles
    Note: A single-hop BFD session is not supported for BGP over IPsec from the SD-WAN Edge.
  6. Configure the other settings as required and select OK.

    When you enable BFD for BGP settings in a profile, the setting is automatically applied to the Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for BGP for Edges for additional information.

    When a BGP neighbor receives an update that BFD session is down, the corresponding BGP session immediately goes down and the routes learnt through the BGP peer are flushed without waiting for the expiry of keepalive timer.

Configure BFD for BGP for Edges

You can override the inherited settings at the Edge level for BFD for BGP.

To override the configuration for a specific Edge:

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section, and then scroll down and select BGP arrow to open the BGP section.
  4. Select the Override check box and move the slider to the ON position to modify the BGP settings for the selected Edge.
    Figure 29. Override BFD for BGP for Edges

Configure BFD for OSPF for Profiles

You can configure BFD for OSPF for Profiles

By default, BFD is deactivated in OSPF. You can enable BFD for OSPF to subscribe to BFD session updates.

Enabling BFD for an OSPF neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure describes how to enable BFD for an already configured OSPF session on an Edge Interface. To configure OSPF settings, see Activate OSPF for Profiles.

To configure the Interface settings, see Configure Interface Settings for Profile.

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select a profile you want to configure BFD for OSPF settings and select the View link in the Device column of the Profile. The Device page for the selected Profile appears.
  3. In the Device tab, scroll down to the Connectivity section and select Interfaces. The Edge models available in the selected Profile are displayed.
  4. In the Interfaces section, select an Edge model to view the interfaces available in the Edge and select an interface to edit the settings.
  5. In the Interface edit window, you can configure OSPF settings under IPv4/IPv6 Settings. Select the OSPF check box and choose the OSPF Area from the drop-down list.
    Figure 30. Configure BFD for OSPF for Profiles
  6. Expand Advanced Settings and in the Custom Settings tab, select the Enable BFD check box.
  7. Configure the other settings as required and select Save.

    When you enable BFD for an OSPF area in a profile, the setting is automatically applied to the corresponding Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for OSPF for Edges for additional information.

    When an OSPF neighbor receives an update that BFD session is down, the corresponding OSPF session immediately goes down and the routes are flushed without waiting for the expiry of keepalive timer.

Configure BFD for OSPF for Edges

You can modify the inherited Profile settings at the Edge level for BFD for OSPF.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select an Edge you want to configure BFD for OSPF settings and select the View link in the Device column of the Edge. The Device page for the selected Edge appears.
  3. In the Device tab, scroll down to the Connectivity section and select Interfaces.
  4. In the Interfaces section, select an interface to edit the settings.
  5. In the Interface edit window, you can configure BFD for OSPF settings for the selected Edge under IPv4/IPv6 Settings as shown in the following screenshot.
    Figure 31. Edit Interface Window
  6. Expand Advanced Settings and in the Custom Settings tab, select the Enable BFD check box.
  7. Configure the required settings for the Edge as required and select Save.

Configure BFD for Gateways

You can configure BFD Settings for Gateways over IPsec tunnels.

Ensure that you have configured the following:

To configure BFD settings for a Gateway:

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Network Services .
  2. In the Non SD-WAN Destinations via Gateway area, select the Edit link in the BFD column that corresponds to the Non SD-WAN Destination.
    Figure 32. Configure BFD Settings for NSD via Gateways
  3. In the Edit BFD window, move the BFD Activated slider to the right to turn it on to configure the BFD settings for the Primary and Secondary Gateways.
    Figure 33. Edit BFD
  4. Configure the BFD settings, as described in the following table.
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Table 21. BFD for Gateways Field Descriptions
    Field Description
    Peer Address Enter the IP address of the remote peer to initiate a BFD session.
    Local Address Enter a locally configured IP address for the peer listener. This address is used to send the packets.
    Multihop This option is not supported for the Gateways.
    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Note: BFD is supported only on VTP Tunnels.

Monitor BFD Sessions

You can monitor the BFD sessions on Edges and Gateways.

To view the BFD sessions:

  1. In the SD-WAN Service of the Enterprise portal, go to Monitor > Routing .
  2. In the Routing screen, select the BFD tab.
    Note: You can select the Filter icon next to the Search option and choose to filter the details by different categories.

    The Monitor Edge BFD Sessions screen displays the BFD sessions on Edge and Gateway.

    Figure 34. Monitor BFD Sessions
    The BFD sessions include the following details for the Edges and Gateways:
    • Name of the Edge or Gateway
    • Segment name
    • Peer IPv4 or IPv6 address
    • Local IPv4 or IPv6 address
    • State of the BFD session
    • Remote and Local timers
    • Number of Events
    • Duration of the BFD session

    Select the link to an event number to view the break-up details of the events.

Monitor BFD Events

You can view the events related to the BFD sessions.

  1. In the SD-WAN service of the Enterprise portal, select Monitor > Events .
  2. To view the events related to BFD, you can use the filter option. Select the Filter icon next to the Search option and choose to filter the details by different categories.
    Figure 35. Monitor BFD Events
    The following are the events related to BFD sessions.
    • BFD session established to Gateway neighbor
    • BFD session established to edge neighbor
    • BFDv6 session established to edge neighbor
    • Edge BFD Configuration
    • Edge BFD IPv6 Configuration
    • Edge BFD neighbor unavailable
    • Edge BFDv6 neighbor unavailable
    • Gateway BFD neighbor unavailable

Troubleshooting BFD

You can run Remote Diagnostics tests to view the logs of the BFD sessions and use the log information for troubleshooting purposes.

To run the tests for BFD:

  1. In the SD-WAN Service of the Enterprise portal, select Diagnostics > Remote Diagnostics .
  2. The Edges page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BFD, scroll to the following sections and run the tests:
    • Troubleshoot BFD- Show BFD Peer Status – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. Select Run to view the details of the BFD peers.
    • Troubleshoot BFD- Show BFD Peer counters – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. Select Run to view the details of counters of the BFD peers.
    • Troubleshoot BFD- Show BFD Setting – Select Run to view the details of BFDv4 settings and status of neighbors.
    • Troubleshoot BFD6- Show BFD6 Setting – Select Run to view the details of BFDv6 settings and status of neighbors.

    For additional information about all the supported BFD related Remote Diagnostics tests, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide.

Overlay Flow Control

The Overlay Flow Control page displays a summarized view of all the routes in your network.

A new NSD bucket has been introduced for the classification of NSD Routes. The new NSD bucket preference logic will be applicable only when the Use NSD policy is enabled along with the Distributed Cost Calculation. The Use NSD policy can only be enabled after you enable the Distributed Cost Calculation.

You can view and edit the global routing preferences and the advertise actions for the Edges, Hubs, Partner Gateways, and Non SD-WAN Destinations via Edge and Gateway.

To configure the Overlay Flow Control settings, perform the following steps:

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
    Figure 36. Configure Overlay Flow Control

    The Overlay Flow Control page displays the following details:

    Table 22. Overlay Flow Control Option Descriptions
    Option Description
    Preferred VPN Exits Displays the priority of the destinations to where the traffic should be routed.
    Global Advertise Flags Displays the advertise actions of static, connected, internal, external, and uplink routes.
    Routes List Displays all routes. You can change the Preferred VPN Exits order for a particular subnet by selecting Edge Subnet in the Overlay Flow Control page.
  2. In the Overlay Flow Control page, you can configure the following settings:
    • Edit – Select to update the priorities and the advertise actions. See Configure Global Routing Preferences.
    • Refresh Routes – This option is available only when the Distributed Cost Calculation feature is enabled by the Operator. By default, the Orchestrator is actively involved in learning the dynamic routes. Edges and Gateways rely on the Orchestrator to calculate initial route preferences and return them to the Edge and Gateway. The Distributed Cost Calculation feature enables to distribute the route cost calculation to the Edges and Gateways. For IPv4, this option is available only when the Distributed Cost Calculation feature is enabled by Operator. For IPv6, Distributed Cost Calculation is enabled by default. The Operator cannot turn off this feature for IPv6.

      For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.

      Note: To enable the Distributed Cost Calculation feature, check with your supporting partner. If you are directly supported by Arista, contact the Support team.
    • Type YES and then select Refresh Routes to make the Edges and Gateways recalculate learned route costs and send them to the Orchestrator. In addition, the changes in the Overlay Flow Control are applied immediately on the new and existing learned routes.

      When you refresh the routes, the Customer Enterprise has the following impact on the network:

      • All the local dynamic routes are refreshed, and the preference and advertise action of these routes are updated. This updated information is advertised to the Gateway, Orchestrator, and eventually across the Enterprise. As this leads to an update in the routing table, there is a brief impact on the traffic for all the sites.
      • Any existing flow using these routes can potentially be affected due to the change in the routing entries.
      Note: It is recommended to use Refresh Routes in a maintenance window to minimize the impact on the Customer Enterprise.
    • VRF Global Routing Preferences – This option enables you to edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.
      Figure 37. Edit VRF Global Routing Preferences
      • Select Preferred VPN Exits to prioritize the VPN Exits.

        Select Edit and use the UP and DOWN arrows to modify the priorities.

        Figure 38. Edit Preferred VPN
      • In the Global Advertise Flags section, select the relevant check boxes to modify the advertise actions for the routes.
    • Routes List – This section displays the learned routes in the subnets. You can select the IPv4 or IPv6 tab to view the corresponding subnets. The following image shows IPv6 subnets. For additional information, see Configure Subnets.
      Figure 39. View Routes List

      The bottom panel of the Overlay Flow Controlwindow displays the subnets. You can prioritize the preferred destinations for the subnets and pin or unpin learned route preferences. For additional information, see For additional information on the subnets, see Configure Subnets.

Configure Global Routing Preferences

In the Overlay Flow Control window, you can edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.

The VRF Global Routing Preferences section displays the Preferred VPN Exits and the Global Advertise Flags areas. To edit these areas, refer to the following procedural steps:

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. In the Overlay Flow Control page, select Preferred VPN Exits and then select the Edit link to open the Edit Preferred VPN window.
    Figure 40. Edit Preferred VPN
  3. You can update the Preferred VPN Exits area and select the UP and DOWN arrows to modify the priorities.
  4. In the Overlay Flow Control page, select Global Advertise Flags to edit Global Advertise flags.
    Figure 41. Edit Global Advertise Flags
    1. In the Global Advertise Flags area, select the relevant check boxes to modify the advertise actions for the routes.
    2. Select Update to save the changes.

Configure Subnets

In the Overlay Flow Control window, you can update the priorities of the destinations for the learned routes in the subnets.

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. The Routes List section of the Overlay Flow Control window displays the subnets with the following details.
    Figure 42. Routes List

     

    Table 23. Configure Subnets Field Descriptions
    Option Description
    Segment Segment name.
    Subnet The network that the route corresponds to along with a list of Edges that learned the route.
    Preferred VPN Exits The route through which another branch can access the subnet.
    Route Type Displays the type of the route, which can be one of the following: Static, Connected, or Learned.
    Last Update The last updated date and time of the preferred VPN exit.
    Created On Date and time when the route was created.
    IPv4 Subnet  
    Eligible VPN Exits  
    Note: You can configure the subnets for both IPv4 and IPv6 addresses.

    Currently, you can view up to 256 subnet prefixes in the API call request. You can use the Filter option to search for the specific subnet prefix. The following information message appears if the results are more than the server could return. "There are more results that the server could return. Please narrow your search criteria."

    Select one or more subnets and select MORE to perform the following activities:
    • Pin Learned Route Preference – Pins the preferences of the selected learned route.
    • Unpin Learned Route Preference – Unpins the preference of the selected learned route to default settings.
    • Delete Learned Routes – Deletes the learned routes. This option does not delete the connected routes, static routes, routes from Overlay Flow Control, and routes from Edge Route table. The option is available only when Configure Distributed Cost Calculation is turned off.
  3. Select the Edit Subnet option for a subnet to modify the priorities of the preferred destination.
    1. In the Subnet window, you can move the destinations from the Eligible VPN Exits to Preferred VPN Exits and vice versa.
      Figure 43. Edit Subnet
    2. In the Preferred VPN Exits panel, select the UP and DOWN arrows to change the priorities and select Update.
    3. You can reset the cost calculation for the subnets when there are pinned routes available. Select Reset, which enables the Orchestrator to clear the pinned routes, recalculate the cost for the selected subnet based on the policy, and send the results to the Edges and Gateways.
      Note: For IPv4 Routes, the Reset option is available only when Distributed Cost Calculation is enabled.
      Note: The Reset option is available only when Distributed Cost Calculation is enabled.

      For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.