◄
►

Configure Dynamic Routing with OSPF or BGP

This section discusses how to configure dynamic routing with OSPF or BGP.

Edge learns routes from adjacent routers through OSPF and BGP. It sends the learned routes to the Gateway/Controller. The Gateway/Controller acts like a route reflector and sends the learned routes to other Edge. The Overlay Flow Control (OFC) enables enterprise-wide route visibility and control for ease of programming and for full and partial overlay.

Arista supports Inbound/Outbound filters to OSPF neighbors, OE1/OE2 route types, MD5 authentication. Routes learned through OSPF will be automatically redistributed to the controller hosted in the cloud or on-premise. Support for BGP Inbound/Outbound filters and the filter can be set to Deny, or optionally you can Add/Change the BGP attribute to influence the path selection, i.e. RFC 1998 community, MED, and local preference.

Note: For information about OSPF and BGP Redistribution, see OSPF/BGP Redistribution.

Activate OSPF for Profiles

Open Shortest Path First (OSPF) can be enabled only on a LAN interface as an active or passive interface. The Edge will only advertise the prefix associated with that LAN switch port. To get full OSPF functionality, you must use it in routed interfaces.
OSPF is an interior gateway protocol (IGP) that operates within a single autonomous system (AS).
Note: OSPF is configurable only on the Global Segment.
OSPFv3 is introduced in the 5.2 release and provides support for the following:
  • Support for OSPFv3 is introduced in the SD-WAN Edge for IPv6 underlay routing in addition to existing BGPv6 support. The following is supported:
    • Underlay IPv6 route learning.
    • Redistribution of OSPFv3 routes into overlay/BGP and vice-versa.
    • Support for Overlay Flow Control (OFC).
  • OSPFv3 is implemented with feature parity to OSPFv2 with the following exceptions:
    • Point to Point (P2P) is not supported.
    • BFDv6 with OSPFv3 is not supported.
    • md5 authentication is not available, as OSPFv3 header does not support it.
This section discusses how to configure dynamic routing with OSPFv2 and OSPFv3 along with Route Summarization.
Note: OSPFv2 supports only IPv4. OSPFv3 supports only IPv6 and is available starting with the 5.2 release.
Note: Route Summarization is available starting with the 5.2 release.

To activate OSPF, perform the steps in the procedure below:

  1. In the SD-WAN service of the Enterprise Portal, select Configure > Profiles .

    The Profile page displays.

    Note: Depending upon your login permissions, you might need to select a Customer or Partner first, then select the Configure tab as indicated in next step.
  2. Select a Profile from the list of available Profiles.
  3. Go to the Routing & NAT section in the UI and select the arrow next to OSPF.
  4. In the OSPF Areas section, configure the Redistribution Settings for OSPFv2/v3, BGP Settings, and if applicable, Route Summarization.
    Figure 1. Configure OSPF for Profiles
    Note: OSPFv2 supports only IPv4. OSPFv3 supports only IPv6 and is only available in the 5.2 release.
    Table 1. OSPF Area Option Descriptions
    Option Description
    Redistribution Settings
    Default Route Choose an OSPF route type (O1 or O2) to be used for default route. Default selection for this configuration is "None".
    Advertise Choose either Always or Conditional. (Choosing Always means to Advertise the default route always. Choosing Conditional means to redistribute default route only when Edge learns via overlay or underlay). The “Overlay Prefixes” option must be checked to use the Conditional default route.
    Overlay Prefixes If applicable, check the Overlay Prefixes check box.
    BGP Settings
    BGP To enable injection of BGP routes into OSPF, select the BGP check box. BGP routes can be redistributed into OSPF, so if this is applicable, enter or choose the configuration options as follows:
    Set Metric In the Set Metric text box, enter the metric. (This is the metric that OSPF would put in its external LSAs that it generates from the redistributed routes). The default metric is 20.
    Set Metric Type From the Set Metric Type drop-down menu, choose a metric type. (This is either type E1 or E2 (OSPF External-LSA type)); the default type is E2)

     

  5. In OSPF Areas, select +Add and configure the options, as described in the table below. Add additional areas, if necessary, by selecting +Add. The fields in the table below cannot be overridden at the Edge level.
    Table 2. OSPF Area Add Option Descriptions
    Option Description
    Area ID Select inside the Area ID text box, enter an OSPF area ID.
    Name Select inside the Name text box, enter a descriptive name for your area.
    Type By default, the Normal type is selected. Only Normal type is supported at this time.

     

  6. Next, configure the Interface Settings for OSPF. For configuration details, see either Configure Interface Settings for a Profile or Configure Interface Settings for Edges.
    Note: OSPF has to be activated at the Profile level first before you can configure it on Edge interfaces.
  7. If applicable, configure Route Summarization as follows:
    Note: The Route Summarization feature is available starting with the 5.2 release, for an overview and use case for this feature, see Route Summarization. For configuration details, follow the steps below in Step #10.
    1. Navigate to Route Summarization.
    2. Select +Add in the Route Summarization area. A new row is added to the Route Summarization area. Configure Route Summarization, as described in the table.
      Figure 2. Configuring Route Summarization

       

      Table 3. Route Summarization Option Descriptions
      Option Description
      Subnet Enter the IP subnet.
      No Advertise When No Advertise is set, all the external routes (Type-5) that are under this supernet are summarized and have chosen not to advertise it. This means it effectively blocks the whole supernet from advertising to its peer.
      Tag Enter the router Tag value (1-4294967295).
      Metric Type Enter the Metric Type (E1 or E2).
      Metric Enter the advertised metric for this route ((0-16777215).
    3. Add additional routes, if necessary, by selecting +Add. Route Summarization Clone or Delete to copy or delete a Route Summarization.
  8. Select Save Changes.

Route Filters

There are two different types of routing:
  • Inbound Routing includes preferences that can be learned or ignored from OSPF and installed into the Overlay Flow Control.
  • Outbound Routing indicates what prefixes can be redistributed into the OSPF.

Activate OSPF for Edges

Open Shortest Path First (OSPF) can be enabled only on a LAN interface as an active or passive interface. The Edge will only advertise the prefix associated with that LAN switch port. To get full OSPF functionality, you must use it in routed interfaces. After you configure the OSPF settings at the Profile level, all the Edges associated with the Profile will inherit the OSPF configuration from the Profile. However, you cannot override the OSPF configuration settings at the Edge level.

If needed, you can view the OSPF configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. Go to the Routing & NAT section and select the arrow next to OSPF.
  4. In the OSPF section, you can view all the inherited OSPF configuration such as OSPF areas, Redistribution settings for OSPFv2/v3, BGP settings, and Route Summarization.
    Figure 3. Activate OSPF for Edges

Configure BGP

You can configure the BGP per segment for a Profile or an Edge. Configuring BGP is available for Underlay Neighbors and Non SD-WAN Neighbors.

Arista VeloCloud SD-WAN supports 4-Byte ASN BGP as follows:
  • As the ASN of Edges.
  • Peer to a neighbor with 4-Byte ASN.
  • Accept 4-Byte ASNs in route advertisements.
See the following sections for configuring BGP for Underlay Neighbors and Non SD-WAN Neighbors:

Configure BGP from Edge to Underlay Neighbors for Profiles

You can configure the BGP per segment at the Profile level as well as at the Edge level. This section provides steps on how to configure BGP with Underlay Neighbors.

Arista VeloCloud SD-WAN supports 4-Byte ASN BGP. See Configure BGP for additional information.

Note: Route Summarization is new for the 5.2 release. For an overview, use case, and black hole routing details for Route Summarization, see section titled, Route Summarization. For configuration details, see the steps below.

To configure BGP, perform the following steps:

  1. In the SD-WAN service of the Enterprise portal, select the Configure tab.
  2. From the left menu, select Profiles. The Profile page displays.
  3. Select a Profile from the list of available Profiles (or Add a Profile if necessary).
  4. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  5. In the BGP area, toggle the radio button from Off to On.
    Figure 4. Configure BGP
  6. In the BGP area, enter the local Autonomous System Number (ASN) number in the appropriate text field.
  7. Configure the BGP Settings, as described in the following table.
    Table 4. Configure BGP Option Descriptions
    Option Description
    Router ID Enter the global BGP router ID. If you do not specify any value, the ID is automatically assigned. If you have configured a loopback Interface for the Edge, the IP address of the loopback Interface will be assigned as the router ID.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Uplink Community Enter the community string to be treated as uplink routes.

    Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes.

    Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
    Enable Graceful Restart check box Note when selecting this check box:

    The local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.

     

  8. Select +Add in the Filter List area to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 5. Add BGP Filter List
  9. In the appropriate text fields, set the rules for the filter, as described in the following table.
    Table 5. Add BGP Filter Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to be matched with the filter:
    • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
    • Community: Choose to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the Prosecutes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when Thebes routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None: The attributes of the matching routes remain the same.
    • Local Preference: The matching traffic is routed to the path with the specified local preference.
    • Community: The matching routes are filtered by the specified community string. You can also select the Community Additive check box to enable the additive option, which appends the community value to existing communities.
    • Metric: The matching traffic is routed to the path with the specified metric value.

     

  10. Select the Plus (+) icon to add more matching rules for the filter. Repeat the procedure to create more BGP filters.
    The configured filters are displayed in the Filter List area.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
  11. Scroll down to the Neighbors area and select +Add.
    Figure 6. Add BGP Neighbors

     

  12. Configure the following settings for the IPv4 addressing type, as described in the following table.
    Table 6. Add BGP Neighbor Option Descriptions
    Option Description
    Neighbor IP Enter the IPv4 address of the BGP neighbor
    ASN Enter the ASN of the neighbor
    Inbound Filter Select an Inbound filer from the drop-down list
    Outbound Filter Select an Outbound filer from the drop-down list
    Note: When overriding and configuring BGP neighbors at the Edge level, any Profile-level filters associated with the neighbors will be removed when you switch the Edge from one profile to another. So at the Edge level, you must make sure to re-associate the filters with the BGP neighbors after switching the Edge profile.
    Additional Options – Select the View All button to configure the following additional settings:
    Table 7. Additional BGP Settings Option Descriptions
    Option Description
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. The range is from 1 to 255 and the default value is 1.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is inherent by default and this field is not configurable.
    Local IP Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address forth outgoing packets. If you do not enter any value, the IP address of the physical Interface is used as the source IP address.
    Note: For eBGP, this field is available only when Max- hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Uplink Used to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes learned over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, also check "Overlay Prefix Over Uplink" in the Advanced Settings area.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.

     

  13. Select the Plus (+) icon to add more BGP neighbors.
    Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For additional information about Multi-hop BGP Routes, see the Remote Diagnostic Tests on Edges section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  14. Scroll down to Advanced Settings and select the down arrow to open the Advanced Settings section.
    Figure 7. Advanced Settings
  15. Configure the following advanced settings, as indicated in the following table, which are globally applied to all the BGP neighbors with IPv4 addresses.
    Table 8. Advanced BGP IPv4 Setting Option Descriptions
    Option Description
    Overlay Prefix Select the check box to redistribute the prefixes learned from the overlay.
    For example, when a Spoke is connected to primary and secondary Hub or Hub Cluster, the Spoke's subnets are redistributed by primary and secondary Hub or Hub Cluster to their neighbor with metric (MED) 33 and 34 respectively. You must configure "bgp always-compare-med" in the neighbor router for symmetric routing.
    Note: Prior to 5.1, the advertised MED values were starting from eight. From release 5.1 and later, the MED values advertised by HUB starts from 33.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to deactivate AS-PATH Carry Over. In certain topologies, deactivating AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Note: When the AS-PATH Carry Over is deactivated, tune your network to avoid routing loops.
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    When you select the Default Routeoption, the Advertise option is available as Conditional.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address in IPv4 format that BGP will be advertising to the peers. Select the plus + icon to add more network addresses.

    When you enable the Default Route option, the BGP routes are advertised based on the Default Route selection globally and per BGP neighbor, as shown in the following table:

    Table 9. Default Route Selection Details
    Default Route Selection Advertising Options
    Global Per BGP Neighbor
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.

     

  16. Select the IPv6 tab to configure the BGP settings for IPv6 addresses. Enter a valid IPv6 address of the BGP neighbor in the Neighbor IP field. The BGP peer for IPv6 supports the following address format:
    • Global unicast address (2001:CAFE:0:2::1)
    • Unique Local address (FD00::1234:BEFF:ACE:E0A4)
  17. Configure the other settings as required.
    Note: The Local IP address configuration is not available for IPv6 address type.
  18. Select Advanced to configure the following advanced settings, which are globally applied to all the BGP neighbors with IPv6 addresses.
    Table 10. Advanced BGP IPv6 Setting Option Descriptions
    Option Description
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    When you select the Default Route option, the Advertise option is available as Conditional.
    Networks Enter the network address in IPv6 format that BGP will be advertising to the peers. Select the Plus (+) Icon to add more network addresses.

     

  19. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 8. Add Route Summarization
    1. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
    2. Under the AS Set column, select the Yes check box if applicable.
    3. Under the Summary Only column, select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  20. Select Save Changes.
    Note: When you configure BGP settings for a profile, the configuration settings are automatically applied to the SD-WAN Edges that are associated with the profile.

    You can also configure BGP for Non SD-WAN Destination Neighbors in an Edge. For additional information, see Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors.

Configure BGP from Edge to Underlay Neighbors for Edges

You can override the inherited Profile settings at the Edge level when configuring BGP from the Edge to Underlay Neighbors.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges . The Edges page displays the existing Edges.
  2. Select the link to an Edge or select the View link in the Device column of the Edge.
  3. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  4. The BGP settings configured for the associated Profile are displayed. If required, you can select the Override check box and modify the BGP Settings.
    Note: When overriding and configuring BGP neighbors at the Edge level, any Profile-level filters associated with the neighbors will be removed when you switch the Edge from one profile to another. So at the Edge level, you must make sure to re-associate the filters with the BGP neighbors after switching the Edge profile.
  5. In addition to the BGP settings configured for a Profile, you can select an Edge Interface configured in the segment as the source Interface for BGP. For the IPv4 address type, you can select only the Loopback Interface as Source Interface and for the IPv6 address type, you can select any Edge Interface as the Source Interface.
    This field is available:
    • Only when you choose to override the BGP Settings at the Edge level.
    • For eBGP, only when Max-hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Important:
    • You cannot select an Edge Interface if you have already configured a local IP address in the Local IP field.
    • You cannot configure a local IP address if you have selected an Edge Interface in the Source Interface drop-down list.
  6. Select Save Changes on the Device screen to save the modified configuration.

Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors

The Non SD-WAN BGP Neighbors configuration is not applicable at Profile level. You can configure the NSD Neighbors only at the Edge level.

BGP is used to establish the BGP neighborship over the IPSec tunnels to the Non SD-WAN Sites. Direct IPSec tunnels are used for establishing a secure communication between the SD-WAN Edge and the Non SD-WAN Destination (NSD). In previous releases, Arista VeloCloud supported NSD tunnels from the SD-WAN Edge with the ability to add NVS static routes. In the 4.3 release, this functionality is extended to support BGP over IPSec to the NSD endpoint for a route-based VPN.

VeloCloud SD-WAN supports 4-Byte ASN BGP. See Configure BGP, for additional information.
Note: The Azure vWAN Automation from Edge feature is not compatible with BGP over IPSec. This is because only static routes are supported when automating connectivity from an Edge to an Azure vWAN.

 

Use Cases

Use Case 1: BGP Over IPSec from an Edge to an Azure VPN

Each Azure VPN gateway allocates one set of public Virtual Public IPs (VIP) for a branch Edge to form IPSec tunnels. Similarly, Azure also allocates one internal private subnet and assigns one internal IP per VIP. This internal tunnel-ip (peer tunnel-ip) will be used for creating BGP peering with the Azure Gateway.

Azure has a restriction that the BGP peer IP (Edge's local tunnel-ip) shouldn't be in the same connected subnet or 169.x.x.x subnet, and therefore we need to support multi-hop BGP on the Edge. In BGP terminology, the local tunnel-ip maps to BGP source address and peer tunnel-ip maps to neighbor/peer address. We need to form a mesh of BGP connections- one per NSD tunnel so that the return traffic from the NVS could be load-balanced (flow-based)- design on the Azure Gateway side. In the below diagram for the physical Edge, we have two public WAN links and so four tunnels to an Azure Gateway. Each tunnel is associated with one BGP connection uniquely identified by the local tunnel_ip and remote peer tunnel_ip. On the Virtual Edge, the only difference is that we have one public WAN link and a maximum of two tunnels and two BGP sessions to the Azure Gateway.

Figure 9. BGP Over IPSec from an Edge to an Azure VPN Use Case
Note: When an SD-WAN Edge is connected to the same Azure end-point using multiple WAN links, there is a maximum of two NSD-BGP neighbors that could be configured (since remote end has only two public_ips and two NSD-BGP peer_ips). Both NSD-BGP neighbors can be configured on the same link (primary/secondary tunnel), or tunnels on different links. If a customer attempts to configure more than two NSD-BGP neighbors and configure the same NSD-BGP peer_ip on more than one tunnel, the last configured BGP nbr_ip + local_ip would be on the SD-WAN Edge and Free Range Routing (FRR).

Use Case 2: BGP Over IPSec from Edge to AWS VPN/Transit Gateway

Unlike Azure, AWS VPN Gateway allocates one set of public VIPs per link to a branch Edge. The total sets of public IPs allocated to a branch Edge from an AWS Gateway will be equal to the number of Edge public WAN links that will connect to the AWS VPN Gateway. Similarly, a /30 internal/private subnet would be allocated per tunnel, which are used for BGP peering on that tunnel. These IPs could be manually overridden in AWS Gateway configuration to ensure they are unique across different availability zones.

Similar to the Azure use-case, the Edge will form a mesh of BGP connections- one per tunnel to the AWS gateway. This will allow load-balancing of the return traffic from the AWS VPN Gateway- design on the AWS side. In the diagram below, for the physical Edge, the AWS Gateway allocates one set of public IPs and one set of tunnel-ips (/30) for each Edge WAN link. There are a total of four tunnels, but terminate in different public IPs on the AWS Gateway and four BGP connections.

Figure 10. BGP Over IPSec from Edge to AWS VPN/Transit Gateway Use Case

Use Case 3: Edge Connecting to Both AWS and Azure VPN Gateways (Hybrid Cloud)

One branch Edge could be connected to both Azure Gateway and AWS Gateway for redundancy purposes or some workloads/apps hosted in one cloud provider while other workloads/apps hosted in a different cloud provider. Regardless of the use-case, the Edge always establishes one BGP session per tunnel and propagates the routes between SD-WAN and IaaS. The diagram below is an example of one branch Edge connected to both Azure and AWS clouds.

Figure 11. Edge Connecting to Both AWS and Azure VPN Gateways (Hybrid Cloud) Use Case

Use Case 4: Hub Cluster Connecting to Azure/AWS Transit Gateways

The Hub cluster members can form IPSec tunnels to the Azure/AWS transit Gateways and leverage the transit Gateways as Layer 3 for routing traffic between different VPCs. Without the native BGP over IPSec functionality on Hub, the Hub needs to connect to an L3 router (Cisco CSR widely used here) using native BGP and the L3 router forming a mesh of BGP over IPSec tunnels with different VPCs. L3 router serves as a transit end-point between different VPCs. Usecase-1 (left diagram below): Use Hub as a transit node between different VPCs in different Availability Zones (AZ) so that one VPC can talk to another VPC. Usecase-2 (right diagram below): Connect all Hubs in the cluster directly to a cloud transit gateway and can use the cloud gateway as a PE(L3) router for routes distribution between cluster members. In both use-cases, without the support for BGP over IPSec on Hub, hub connects to an L3 router like CSR using native BGP and CSR peers with transit/VPC gateway using BGP over IPSec.

Figure 12. Hub Cluster Connecting to Azure/AWS Transit Gateways Use Case

Use Case 5: Support Transit Functionality in Cloud Providers without Native Support

Some cloud providers like Google Cloud and AliCloud do not have native support for transit functionality (no transit Gateways), and with the support for BGP over IPSec, can rely on SD-WAN Edge/Hub deployed in the cloud to achieve the transit functionality between different VPCs/VNETs. Without the BGP over IPSec support, you must use an L3 router like CSR (solution (2)) to achieve the transit functionality.

Note: Prior to the 4.3 release, for customers who have reachability to the same NVS-Static destination via NVS-From-Gateway and NVS-From-Edge, the traffic from other branch SD-WAN Edges will prefer the path via NVS-Gateway. When customers upgrade their network to the 4.3 release or later, this traffic path from other branch- SD-WAN Edges will prefer the path via the NVS-Edge. Therefore, customers must update the NVS-Static-Destination’s metric of the NSD-Edge and the NSD-Gateway as per their traffic path preference.

Prerequisites:

To enable BGP with Non SD-WAN neighbors:

  1. In the SD-WAN service of the Enterprise portal, select Configure.
  2. From the left menu, select Edges. The Edges page displays.
  3. Select an Edge from the list of available Edges.
  4. Go to the Routing & NAT section in the UI and select the arrow next to BGP.
  5. In the BGP area, check the Override check box and toggle the radio button from Off to On.
    Figure 13. Override BGP Settings
  6. In the BGP Editor window, configure the following settings:
    1. Enter the local Autonomous System Number (ASN) and then configure the following in the BGP Settings section.
    2. Configure the BGP Settings, as described in the following table.
      Table 11. BGP Option Descriptions
      Option Description
      Router ID Enter the global BGP router ID. If you do not specify any value, the ID is automatically assigned. If you have configured a loopback Interface for the Edge, the IP address of the loopback Interface will be assigned as the router ID.
      Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
      Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
      Uplink Community Enter the community string to be treated as uplink routes.

      Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes.

      Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
      Enable Graceful Restart check box When selecting this check box the local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.

       

  7. Select +Add in the Filter List area to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 14. Add BGP Filter List
    1. In the appropriate text fields, set the rules for the filter, as described in the following table.
      Table 12. Add BGP Filter List Option Descriptions
      Option Description
      Filter Name Enter a descriptive name for the BGP filter.
      Match Type and Value Choose the type of the routes to be matched with the filter:
      • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
      • Community: Choose to match with a community and enter the community string in the Value field.
      Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
      Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
      Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
      • None: The attributes of the matching routes remain the same.
      • Local Preference: The matching traffic is routed to the path with the specified local preference.
      • Community: The matching routes are filtered by the specified community string. You can also select the Community Additive check box to enable the additive option, which appends the community value to existing communities.
      • Metric: The matching traffic is routed to the path with the specified metric value.
      • AS-Path-Prepend: Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.

       

    2. To add more matching rules to the filter, select the Plus ( +) icon.
  8. Select OK to create the filter.
    The configured filters are displayed in the BGP Editor window.
  9. Configure Underlay Neighbors for IPv4 and IPv6 addresses, as required. For additional information, see Configure BGP from Edge to Underlay Neighbors for Edges.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
    1. In the NSD Neighbors section, configure the following settings:
      Table 13. Add NSD Neighbors Option Descriptions
      Option Description
      NSD Name Select the NSD Name from the drop-down list. The NSDs already configured in the Branch to Non SD-WAN Destination via Edge area of the Orchestrator are displayed in the drop-down list.
      Link Name Choose the name of the WAN link associated with the NSD neighbor.
      Tunnel Type Choose the tunnel type of the Peer as Primary or Secondary.
      Neighbor IP Enter the IP address of the NSD neighbor.
      ASN Enter the ASN for the NSD neighbor.
      Inbound Filter Select an Inbound filer from the drop-down list.
      Outbound Filter Select an Outbound filer from the drop-down list.
      Additional Options – Select the view all link to configure the following additional settings:
      Uplink Used to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes learned over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, select the Overlay Prefix Over Uplink check box in the Advanced settings.
      Local IP Local IP is mandatory for configuring Non SD-WAN Neighbors. Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
      Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
      Note: When upgrading to the 5.1 release, any max-hop value of 1 will automatically be updated to a max-hop value of 2.
      Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is deactivated by default and this field is not configurable.
      Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
      Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
      Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
      Note: Single-hop BFD session is not supported for BGP over IPSec with NSD Neighbors. However, multi-hop BFD is supported. Local IP is mandatory for NSD-BGP sessions on the SD-WAN Edge. The SD-WAN Edge handles only the connected Interface IPs as a single-hop BFD.
      Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
      Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
      Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
      MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
      MD5 Password Enter a password for MD5 authentication.
      Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.
      Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and interface. Until the recursive resolution happens, the recursive routes point to an intermediate interface. For additional information about Multi-hop BGP Routes, see the Remote Diagnostic Tests on Edges section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  10. Select Advanced to configure the following settings, as described in the following table.
    Note: Advanced Settings are shared across both the underlay BGP neighbors and NSD BGP neighbors.
    Table 14. Advanced Neighbor Settings Option Descriptions
    Option Description
    Overlay Prefix Select the check box to redistribute the prefixes learned from the overlay.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to turn off AS-PATH Carry Over. In certain topologies, turning off AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Warning: When the AS-PATH Carry Over is turned off, tune your network to avoid routing loops.
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay.

    When you select the Default Route option, the Advertise option is available as Conditional.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address that BGP will be advertising to the peers. Click the Plus ( +) icon to add more network addresses

    When you enable the Default Route option, the BGP routes are advertised based on the Default Route selection globally and per BGP neighbor, as shown in the following table.

    Table 15. Default Route Selection Details
    Default Route Selection Advertising Options
    Global Per BGP Neighbor
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.

     

  11. Select OK to save the configured filters and NSD Neighbors.
  12. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 15. Add Route Summarization
    1. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
    2. Under the AS Set column, select the Yes check box if applicable.
    3. Under the Summary Only column, select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  13. Select Save Changes

    You can also configure BGP from Edge to underlay neighbors. For additional information, see Configure BGP from Edge to Underlay Neighbors for Profiles.

Configure BGP Over IPsec from Gateways

Ensure that you have configured the following:
Note: It is recommended to turn on Distributed Cost Calculation for best performance and scaling when using BGP over IPsec via Gateway. The Distributed Cost Calculation is supported starting from Release 3.4.0.

For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.

You can configure BGP Settings for Gateways over IPsec tunnels.
Only eBGP is supported with BGP over IPsec.
Note: It is recommended to use eBGP between SDWAN Gateway and NSD sites. If iBGP is used, applying local preference does not work with outbound filter. In that case, customer must choose metric or AS path prepend options to achieve desirable routing.

Arista VeloCloud SD-WAN allows Enterprise users to define and configure a Non SD-WAN Destination instance in order to establish a secure IPsec tunnel to a Non SD-WAN Destination through an SD-WAN Gateway.

Note: For the 5.2 release, when multiple NSDs are configured for the same segment, the same set of summary route configurations must be present across all NSDs.
Note: The Azure vWAN Automation from Gateway feature is not compatible with BGP over IPsec. This is because only static routes are supported when automating connectivity from a Gateway to an Azure vWAN.
Important: DCC is mandatory for ECMP to work properly.
  1. Go to Configure > Network Services , and then under Non SD-WAN Destinations, expand Non SD-WAN Destinations via Gateway.
    Note: The New NSD via Gateway option appears only when there are no items in the table. Follow Step 2 and Step 3 to create a new Non SD-WAN Destination.
    Figure 16. Configure Non SD-WAN Destinations
  2. Select +New to create a new Non SD-WAN Destination. The Non SD-WAN Destinations via Gateway dialog appears.
    Figure 17. Create New Non SD-WAN Destinations via Gateway
  3. In the Non SD-WAN Destinations via Gateway area, configure the required fields as described in the following table.
    Table 16. Non SD-WAN Destinations via Gateway Field Descriptions
    Option Description
    Name Enter a name for the Non SD-WAN Destination in the text box.
    Type Select an IPsec tunnel type from the drop-down menu.
    Tunnel Mode Active/ Hot-Standby mode supports to set up a maximum of 2 tunnel endpoints or Gateways.
    Active/Active mode supports to set up a maximum of 4 tunnel endpoints or Gateways. All Active tunnels can send and receive traffic through ECMP.
    VPN Gateway 1 Enter a valid IP address
    VPN Gateway 2 Enter a valid IP address. This field is optional
    The NSD is created and it appears in the Non SD-WAN Destinations via Gateway table, as shown in the following screenshot.
    Figure 18. List of Configured NSDs via Gateway
  4. In the Non SD-WAN Destination via Gateway area, slide the grey bar to the far right to the BGP column.
    1. Select the Edit link under the BGP column, the Edit BGP dialog displays.

      If the Edit link does not display under the BGP column, see Configure a Tunnel Between a Branch and a Non SD-WAN Destinations via Edge to enable an Edge to Non SD-WAN via Gateway.

    2. Toggle the BGP Activated radio button to the right to turn it green.
    3. Select +Add to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
      Table 17. Create Filter Field Descriptions
      Option Description
      Filter Name Enter a descriptive name for the BGP filter.
      Match Type and Value Choose the type of the routes to be matched with the filter:
      • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
      • Community: Choose to match with a community and enter the community string in the Value field.
      Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
      Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
      Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
      • None: The attributes of the matching routes remain the same.
      • Local Preference: The matching traffic is routed to the path with the specified local preference.
      • Community: The matching routes are filtered by the specified community string. You can also select the Community Additive check box to enable the additive option, which appends the community value to existing communities.
      • Metric: The matching traffic is routed to the path with the specified metric value.
      • AS-Path-Prepend: Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.

       

    4. Select the Plus (+) icon to add more matching rules for the filter. Repeat the procedure to create more filters.

      The configured filters are displayed in the Filter List area.

      Figure 19. BGP Filter List
  5. In the Edit BGP window, configure the BGP settings for the Primary and Secondary Gateways.
    Figure 20. Edit BGP Neighbors Gateway Settings
    Note: These BGP neighbors are assigned to their respective tunnels exclusively for neighborship establishment and subsequent control exchanges, ensuring these communication occurs solely over the designated tunnels.
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Note: For a customer deployment where a Non SD-WAN Destination (NSD) via Gateway is configured to use redundant tunnels, if the Primary and Secondary Gateways advertise a prefix with an equal AS path to the Primary and Secondary NSD tunnels, the Primary NSD tunnel will prefer a redundant Gateway path over the Primary Gateway. The impact of the Primary NSD over Gateway tunnel preferring the redundant Gateway path over the Primary Gateway is experienced only for return traffic to the Gateway from the NSD.

    If you do not want your BGP router to prefer the redundant Gateway, the workaround is to configure AS-PATH prepend and set the metric filter to a higher (3 or more) metric for the advertised prefix in the redundant Gateway. Doing this ensures the NSD's primary tunnel chooses the Primary Gateway for return traffic.

    1. In the Primary Cloud Gateway section, enter the local ASN and the Router ID.
    2. Scroll down to the Neighbors area and select +Add.
    3. Configure the following settings in the Neighbors area, as described in the following table, and select Save Changes.
      Table 18. BGP Gateway Option Descriptions
      Option Description
      Local ASN Enter the local Autonomous System Number (ASN)
      Router ID Enter the BGP Router ID
      Neighbor IP Enter the IP address of the BGP neighbor
      ASN Enter the ASN of the neighbor
      Inbound Filter Select an Inbound filer from the drop-down list
      Outbound Filter Select an Outbound filer from the drop-down list
      Additional Options – Select the view all link to configure the following additional settings:
      Local IP Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
      Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
      Note: When upgrading to the 5.1 release, any max-hop value of 1 will automatically be updated to a max-hop value of 2.
      Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different.
      Allow AS Select the check box to allow the BGP routes to be received and processed even if the Gateway detects its own ASN in the AS-Path.
      Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
      Enable BFD Enables subscription to the existing BFD session for the BGP neighbor.
      Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 1 to 65535 seconds. The default value is 60 seconds.
      Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 1 to 65535 seconds. The default value is 180 seconds.
      Connect Enter the time interval to try a new TCP connection with the peer if it detects that the TCP session is not passive. The default value is 120 seconds.
      MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and is used as a security guard for BGP peering.
      MD5 Password Enter a password for MD5 authentication.
      Note: Starting from the 4.5 release, the use of the special character "<" in the password is no longer supported. In cases where users have already used "<" in their passwords in previous releases, they must remove it to save any changes on the page.

      The configured Neighbors are displayed in the Neighbors area.

      Figure 21. Configured BGP Neighbors List
      Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For additional information about Multi-hop BGP Routes, see the Remote Diagnostic Tests on Edges section in the Arista VeloCloud SD-WAN Troubleshooting Guide.
  6. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 22. Add Route Summarization

     

    Table 19. Route Summarization Field Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Subnet Enter the IP subnet.
    AS Set Generate AS set path information from the summarized routes (while advertising the summary route to the peer). Under the AS Set column, select the Yes check box if applicable.
    Summary Only Select the Yes check box to allow only the summarized route to be sent.
    Add additional routes, if necessary, by selecting +Add. To clone or delete a route summarization, use the appropriate buttons, located next to +Add.
  7. Select Save Changes.
    Note:
    • Only for Gateways running version 6.0 or later have an option to configure up to 4 tunnels based on VPN type. In addition, those tunnels destined to be a Non SD-WAN gateways can operate in either AA or A-HS mode to achieve load sharing/bearing preferences of the user.
    • For gateways running version less than 6.0, all active-active configurations are interpreted as active-hotstandby with tunnel 1 being active and tunnel 2 being hot-standby.

Monitor BGP Events

You can view the events related to the BGP sessions.
  1. In the SD-WAN service of the Enterprise portal, select Monitor > Events .
  2. To view the events related to BGP, you can use the filter option. Click the Filter icon next to the Search option and choose to filter the details by different categories.

    The following image shows some of the BGP events.

    Figure 23. Monitor BGP Events
    The following are the events related to BGP.
    • BGP session established to Gateway neighbor
    • BGP session established to Edge neighbor
    • BGPv6 session established to Edge neighbor
    • Edge BGP neighbor unavailable
    • Edge BGPv6 neighbor unavailable
    • Gateway BGP neighbor unavailable

Troubleshooting BGP Settings

You can run Remote Diagnostics tests to view the logs of the BGP sessions and use the log information for troubleshooting purposes.

To run the tests for BGP:

  1. In the SD-WAN service of the Enterprise Portal, select Diagnostics > Remote Diagnostics .
  2. The Remote Diagnostics page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BGP sessions, scroll to the following sections and run the tests:
    • Troubleshoot BGP- List BGP Redistributed Routes – Run this test to view routes redistributed to BGP neighbors.
    • Troubleshoot BGP- List BGP Routes – Run this test to view the BGP routes from neighbors. You can enter IPv4 or IPv6 prefix to view specific BGP routes or leave the prefix empty to view all the BGP routes.
    • Troubleshoot BGP- List Routes per Prefix – Run this test to view all the Overlay and Underlay routes for a specific IPv4 or IPv6 prefix and the related details.
    • Troubleshoot BGP- Show BGP Neighbor Advertised Routes – Run this test to view the BGP routes advertised to a neighbor.
    • Troubleshoot BGP- Show BGP Neighbor Learned Routes – Run this test to view all the accepted BGP routes learned from a neighbor after filters.
    • Troubleshoot BGP- Show BGP Neighbor Received Routes – Run this test to view all the BGP routes learned from a neighbor before filters.
    • Troubleshoot BGP- Show BGP Neighbor details – Run this test to view the details of BGP neighbor.
    • Troubleshoot BGP- Show BGP Routes per Prefix – Run this test to view all the BGP routes and their attributes for the specified prefix.
    • Troubleshoot BGP- Show BGP Summary – Run this test to view the existing BGP neighbor and received routes.
    • Troubleshoot BGP- Show BGP Table – Run this test to view the BGP table.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Advertised Routes – Run this test to view the BGPv6 routes advertised to a neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Learned Routes – Run this test to view all the accepted BGPv6 routes learned from a neighbor after filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Received Routes – Run this test to view all the BGPv6 routes received from a neighbor before filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor details – Run this test to view the details of BGPv6 neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Routes per Prefix – Run this test to view all the BGPv6 routes for the prefix and their attributes.
    • Troubleshoot BGPv6- Show BGPv6 Summary – Run this test to view the existing BGPv6 neighbor and received routes.
    • Troubleshoot BGPv6- Show BGPv6 Table – Run this test to view the details of BGPv6 table.

    For additional information about all the supported BGP related Remote Diagnostics tests, see the Remote Diagnostic Tests on Edges section in the Arista VeloCloud SD-WAN Troubleshooting Guide.

OSPF/BGP Redistribution

Each of routing protocols OSPF and BGP may be enabled independently and the prior model of allowing only one routing protocol to be enabled on the system has been removed with this release. This release also allows the possibility of redistributing OSPF into BGP or BGP into OSPF (or both simultaneously), along with other possible route sources like prefixes learnt over the overlay, connected routes, static routes, etc.

In addition, with release 3.2, we are standardizing the redistribution behavior along more traditional lines (similar to that in other routing vendors). For example, if there is more than one route available for the same prefix, then only the best route for that prefix in the system RIB will be redistributed to the destination protocol if the configuration in the destination protocol allows redistribution for that route type.

Consider, as an example, redistribution of the prefix 192.168.1.0/24 into BGP. Let's say routes to the prefix 192.168.1.0/24 are locally available, learned from OSPF and separately learned as an Overlay prefix. Let's further assume that between the OFC flow ordering for the prefix, and route metrics, and route preference the OSPF route ranks above (is better than) the learned overlay route for that same prefix. Then, the OSPF route will be redistributed into BGP if OSPF redistribution has been turned on in BGP. Note that since the overlay learned prefix is not the best route for that prefix in the system RIB, it will not be redistributed into BGP even if the redistribution of overlay prefixes has been turned on in BGP.

In cases like the above, in order to facilitate the redistribution of the best route for a prefix into a given destination protocol, the user can enable redistribution for the specific route type that is the best route in the system.

Alternately, if the user prefers a different route source for that prefix to be redistributed into the destination protocol, the user can control the relative precedence of the route in the system RIB using the Overlay Flow Control facility provided by the management interface, or by varying the route metric.

 

OSPF/BGP Redistribution Metric Calculation

Starting with the 5.2 release, the route redistribution metric calculation has changed. When a route is redistributed from the Overlay to OSPF/BGP, the redistribution metric is calculated by taking the original route metric and adding the transit metric:
  • The transit metric is (0) if the route is learned from a directly connected Edge.
  • The transit metric is (90) if the route is learned via a Gateway.
  • The transit metric is (32 + hub's order value) if the route is learned via a Hub Edge.
For OSPF External Type-1 (OE1) routes, this is the final metric. For OSPF External Type-2 (OE2) routes, it will add up the non-preferred metric constant (8388607). This is why there is a very high metric value for an OE2 route type on Edge peers.

For BGP, this implies that the BGP MED value advertised by Hub Edges, which previously started from 9, 10, 11, and so forth, now starts from 33, 34, 35, and so forth.

See the following for additional information:

BFD Settings

Bidirectional Forwarding Detection (BFD) is a simple Hello protocol that is similar to detection components of well-known routing protocols. A pair of systems transmit BFD packets periodically over each path between the two systems, and if a system stops receiving BFD packets for long enough, the neighboring system is assumed to have failed.

A BFD session is established based on the needs of the application that would use BFD. The user has to explicitly configure the address and parameters for the BFD session and the subscribers/applications (BGP/OSPF) of the session, as there is no discovery mechanism in BFD.

Routing protocols like BGP or OSPF exchange the learned routes between Edges and Routers. These protocols exchange routes and detect route failures using their own mechanism. Generally, route failures are detected based on the keepalive mechanism where one entity echoes other entity on a frequent configured interval, that is the keepalive time. These routing protocols have higher keepalive timers which results in longer duration to detect the route failures. BFD detects route failures between two connected entities faster with low overhead on detection of failures.

The following are the advantages of implementing BFD with routing protocols.
  • Fast route failure detection with low re-convergence time.
  • Less overhead in route failure detection.
  • Uniform rate of route failure detection across routing protocols.

BFD can be defined as a simple service. The service primitives provided by BFD are to create, destroy, and modify a session, given the destination address and other parameters. BFD in return provides a signal to the clients indicating when the BFD session goes up or down.

There are two operating modes to BFD, asynchronous mode and demand mode. Arista VeloCloud supports asynchronous mode. In this mode, the systems periodically send BFD control packets to other systems and if several packets in a row are not received by a system, the session is declared to be down.
Note: BFD Echo mode is not supported.
Arista VeloCloud supports BFD for the following routing protocols:
  • BGP on Edges and Partner Gateways
  • OSPF on Edges

Configure BFD for Profiles

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities.

To configure a BFD session for Profiles:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device icon for a profile, or select a profile and select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to the BDF area to open it.
  4. Select the BDF slider to ON position.
    Figure 24. Configure BFD for Profiles
  5. Configure the required settings, as described in the following table.
    Table 20. BFD for Profile Field Descriptions
    Field Description
    Peer Address Enter the IPv4 address of the remote peer to initiate a BFD session.
    Local Address Enter a locally configured IPv4 address for the peer listener. This address is used to send the packets.
    Note: You can select the IPv6 tab to configure IPv6 addresses for the remote peer and the peer listener.
    For IPv6, the local and peer addresses support only the following format:
    • IPv6 global unicast address (2001:CAFE:0:2::1)
    • IPv6 unique local address (FD00::1234:BEFF:ACE:E0A4)
    Multihop Select the check box to enable multi-hop for the BFD session. While BFD on Edge and Gateway supports directly connected BFD Sessions, you need to configure BFD peers in conjunction with multi-hop BGP neighbors. The multi-hop BFD option supports this requirement.

    Multihop must be enabled for the BFD sessions for NSD-BGP-Neighbors.
    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.

     

  6. Select the (Plus+) icon to add details of more peers.
  7. Select Save Changes.
    When you configure BFD rules for a profile, the rules are automatically applied to the Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for Edges for additional information.
VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configure BFD for Edges

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities. Once you have configured BFD rules for a Profile, the rules are automatically applied to the Edges that are associated with the profile. Optionally, you can override the inherited settings at the Edge level.

To override the configuration for a specific Edge:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. In the Device tab, scroll down to the BFD Rules section.
  4. Select the Override check box to modify the BFD configuration settings for the selected Edge.
    Figure 25. Configure BFD for Edges
  5. Select Save Changes.
VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configure BFD for BGP for Profiles

You can configure BFD for BGP on SD-WAN Profiles.

By default, BFD is deactivated in BGP neighbor. You can enable BFD for a BGP session to subscribe to BFD session updates. Enabling BFD for a BGP neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure discusses how to enable BFD for an already configured BGP session on an Edge. To configure BGP settings, see Configure BGP from Edge to Underlay Neighbors for Profiles.

To enable BFD for BGP on partner Gateways, you must be an Operator super user. For additional information, see the Configure Partner Handoff section in the VeloCloud SD-WAN Operator Guide.

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device icon for a profile, or select a profile and select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to BGP to open the BGP section.
  4. Select the slider and move it to the ON position.
  5. In the BGP Editor window, select view all in the Additional Options column for a BGP neighbor and select the Enable BFD check box. You can enable a BFD subscription for multiple BGP neighbors, including NSD Neighbors in the 4.3 release. NOTE: Multihop must be configured as Multihop BFD for NSD BGP Neighbors in the 4.3 release. For additional information about NSD Neighbors, see section titled, Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors.
    Figure 26. Configure BFD for BGP for Profiles
    Note: A single-hop BFD session is not supported for BGP over IPsec from the SD-WAN Edge.
  6. Configure the other settings as required and select OK.

    When you enable BFD for BGP settings in a profile, the setting is automatically applied to the Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for BGP for Edges for additional information.

    When a BGP neighbor receives an update that BFD session is down, the corresponding BGP session immediately goes down and the routes learned through the BGP peer are flushed without waiting for the expiry of keepalive timer.

Configure BFD for BGP for Edges

You can override the inherited settings at the Edge level for BFD for BGP.

To override the configuration for a specific Edge:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select the Device icon next to an Edge, or select the link to an Edge and then select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section, and then scroll down and select BGP arrow to open the BGP section.
  4. Select the Override check box and move the slider to the ON position to modify the BGP settings for the selected Edge.
    Figure 27. Configure BFD for BGP for Edges

Configure BFD for OSPF for Profiles

You can configure BFD for OSPF for Profiles.

By default, BFD is deactivated in OSPF. You can enable BFD for OSPF to subscribe to BFD session updates.

Enabling BFD for an OSPF neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure discusses how to enable BFD for an already configured OSPF session on an Edge Interface. To configure OSPF settings, see Activate OSPF for Profiles.

To configure the Interface settings, see Configure Interface Settings for a Profile.

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select a profile you want to configure BFD for OSPF settings and select the View link in the Device column of the Profile. The Device page for the selected Profile appears.
  3. On the Device tab, scroll down to the Connectivity section and select Interfaces. The Edge models available in the selected Profile are displayed.
  4. In the Interfaces section, select an Edge model to view the interfaces available in the Edge and select an interface to edit the settings.
  5. In the Interface edit window, you can configure OSPF settings under IPv4/IPv6 Settings. Select the OSPF check box and choose the OSPF Area from the drop-down list.
  6. In the Interface edit window, you can configure OSPF settings under IPv4/IPv6 Settings. Select the OSPF check box and choose the OSPF Area from the drop-down list.
    Figure 28. Configure BFD for OSPF for Profiles
  7. Expand Advanced Settings and on the Custom Settings tab, select the Enable BFD check box.
  8. Configure the other settings as required and select Save.

    When you enable BFD for an OSPF area in a profile, the setting is automatically applied to the corresponding Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for OSPF for Edges for additional information.

    When an OSPF neighbor receives an update that BFD session is down, the corresponding OSPF session immediately goes down and the routes are flushed without waiting for the expiry of keepalive timer.

Configure BFD for OSPF for Edges

You can modify the inherited Profile settings at the Edge level for BFD for OSPF.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select an Edge you want to configure BFD for OSPF settings and select the View link in the Device column of the Edge. The Device page for the selected Edge appears.
  3. On the Device tab, scroll down to the Connectivity section and select Interfaces.
  4. In the Interfaces section, select an interface to edit the settings.
  5. In the Interface edit window, you can configure BFD for OSPF settings for the selected Edge under IPv4/IPv6 Settings as shown in the following screen shot.
    Figure 29. Configure BFD for OSPF for Edges
  6. Expand Advanced Settings and on the Custom Settings tab, select the Enable BFD check box.
  7. Configure the required settings for the Edge as required and select Save.

Configure BFD for Gateways

You can configure BFD Settings for Gateways over IPsec tunnels.
Ensure that you have configured the following:

To configure BFD settings for a Gateway:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Network Services .
  2. In the Non SD-WAN Destinations via Gateway area, select the Edit link in the BFD column that corresponds to the Non SD-WAN Destination.
    Figure 30. Configure Non SD-WAN Destinations via Gateway
  3. In the BFD Editor window, move the BFD Activated slider to the right to turn it on to configure the BFD settings for the Primary and Secondary Gateways.
  4. Configure the BFD settings, as described in the following table.
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Figure 31. Edit BFD for Gateways

     

    Table 21. Edit BFD Field Descriptions
    Field Description
    Peer Address Enter the IP address of the remote peer to initiate a BFD session.
    Local Address Enter a locally configured IP address for the peer listener. This address is used to send the packets.
    Multihop This option is not supported for the Gateways.
    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Note: BFD is supported only on VTP Tunnels.

Monitor BFD Sessions

You can monitor the BFD sessions on Edges and Gateways.

To view the BFD sessions:

  1. In the SD-WAN service of the Enterprise portal, go to Monitor > Routing .
  2. On the Routing screen, select the BFD tab.
    Note: You can select the Filter icon next to the Search option and choose to filter the details by different categories.

    The Edge BFD Sessions screen displays the BFD sessions on Edge and Gateway.

    Figure 32. Monitor BFD Sessions
    The BFD sessions include the following details for the Edges and Gateways:
    • Name of the Edge or Gateway
    • Segment name
    • Peer IPv4 or IPv6 address
    • Local IPv4 or IPv6 address
    • State of the BFD session
    • Remote and Local timers
    • Number of Events
    • Duration of the BFD session

    Select the link to an event number to view the break-up details of the events.

Monitor BFD Events

You can view the events related to the BFD sessions.

In the SD-WAN service of the Enterprise portal, select Monitor > Events .

To view the events related to BFD, you can use the filter option. Select the Filter icon next to the Search option and choose to filter the details by different categories.

The following image shows some of the BFD events.

Figure 33. Monitor BFD Events
The following are the events related to BFD sessions.
  • BFD session established to Gateway neighbor
  • BFD session established to edge neighbor
  • BFDv6 session established to edge neighbor
  • Edge BFD Configuration
  • Edge BFD IPv6 Configuration
  • Edge BFD neighbor unavailable
  • Edge BFDv6 neighbor unavailable
  • Gateway BFD neighbor unavailable

Troubleshooting BFD

You can run Remote Diagnostics tests to view the logs of the BFD sessions and use the log information for troubleshooting purposes.

To run the tests for BFD:

  1. In the SD-WAN service of the Enterprise portal, select Diagnostics > Remote Diagnostics .
  2. The Edges page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BFD, scroll to the following sections and run the tests:
    • Troubleshoot BFD- Show BFD Peer Status – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. Select Run to view the details of the BFD peers.
    • Troubleshoot BFD- Show BFD Peer counters – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. Select Run to view the details of counters of the BFD peers.
    • Troubleshoot BFD- Show BFD Setting – Select Run to view the details of BFDv4 settings and status of neighbors.
    • Troubleshoot BFD6- Show BFD6 Setting – Select Run to view the details of BFDv6 settings and status of neighbors.

    For additional information about all the supported BFD related Remote Diagnostics tests, see the Remote Diagnostic Tests on Edges section in the Arista VeloCloud SD-WAN Troubleshooting Guide.

Overlay Flow Control

The Overlay Flow Control page displays a summarized view of all the routes in your network.

VeloCloud SD-WAN supports NSD bucket for the classification of NSD Routes. The new NSD bucket preference logic will be applicable only when the Use NSD policy is enabled along with the Distributed Cost Calculation. The Use NSD policy can only be enabled after you enable the Distributed Cost Calculation.

You can view and edit the global routing preferences and the advertise actions for the Edges, Hubs, Partner Gateways, and Non SD-WAN Destinations via Edge and Gateway.

To configure the Overlay Flow Control settings, perform the following steps:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Overlay Flow Control .
    Figure 34. Configure Overlay Flow Control

    The Overlay Flow Control page displays the following details:

    Table 22. Overlay Flow Control Option Descriptions
    Option Description
    Preferred VPN Exits Displays the priority of the destinations to where the traffic should be routed.
    Global Advertise Flags Displays the advertise actions of static, connected, internal, external, and uplink routes.
    Routes List Displays all routes. You can change the Preferred VPN Exits order for a particular subnet by selecting Edge Subnet on the Overlay Flow Control page.

     

  2. On the Overlay Flow Control page, you can configure the following settings:
    • Edit – Select to update the priorities and the advertise actions. See Configure Global Routing Preferences.
    • Refresh Routes – This option is available only when the Distributed Cost Calculation feature is enabled by the Operator. By default, the Orchestrator is actively involved in learning the dynamic routes. Edges and Gateways rely on the Orchestrator to calculate initial route preferences and return them to the Edge and Gateway. The Distributed Cost Calculation feature enables to distribute the route cost calculation to the Edges and Gateways. For IPv4, this option is available only when the Distributed Cost Calculation feature is enabled by Operator. For IPv6, Distributed Cost Calculation is enabled by default. The Operator cannot turn off this feature for IPv6.

      For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.

      Note: To enable the Distributed Cost Calculation feature, check with your supporting partner. If you are directly supported by Arista, contact the VeloCloud SD-WAN team.
    • Type YES and then select Refresh Routes to make the Edges and Gateways recalculate learned route costs and send them to the Orchestrator. In addition, the changes in the Overlay Flow Control are applied immediately on the new and existing learned routes.
      When you refresh the routes, the Customer Enterprise has the following impact on the network:
      • All the local dynamic routes are refreshed, and the preference and advertise action of these routes are updated. This updated information is advertised to the Gateway, Orchestrator, and eventually across the Enterprise. As this leads to an update in the routing table, there is a brief impact on the traffic for all the sites.
      • Any existing flow using these routes can potentially be affected due to the change in the routing entries.
      Note: It is recommended to use Refresh Routes in a maintenance window to minimize the impact on the Customer Enterprise.
    • VRF Global Routing Preferences – This option enables you to edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.
      Figure 35. VRF Global Routing Preferences
      • Select Preferred VPN Exits to prioritize the VPN Exits.

        Select Edit and use the UP and DOWN arrows to modify the priorities.

        Figure 36. Edit Preferred VPN
      • In the Global Advertise Flags section, select the relevant check boxes to modify the advertise actions for the routes.
    • Routes List – This section displays the learned routes in the subnets. You can select the IPv4 or IPv6 tab to view the corresponding subnets. The following image shows IPv6 subnets. For additional information, see Configure Subnets.
      Figure 37. Routes List

      The bottom panel of the Overlay Flow Control window displays the subnets. You can prioritize the preferred destinations for the subnets and pin or unpin learned route preferences. For additional information on the subnets, see Configure Subnets.

Configure Global Routing Preferences

In the Overlay Flow Control window, you can edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.The VRF Global Routing Preferences section displays the Preferred VPN Exits and the Global Advertise Flags areas. To edit these areas, perform the following steps:
  1. In the SD-WAN service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. On the Overlay Flow Control page, select Preferred VPN Exits and then select the Edit link to open the Edit Preferred VPN screen.
    Figure 38. Edit Preferred VPN
  3. You can update the Preferred VPN Exits area and select the UP and DOWN arrows to modify the priorities.
  4. On the Overlay Flow Control page, select Global Advertise Flags to edit Global Advertise flags.
    Figure 39. Edit Global Advertise Flags
    1. In the Global Advertise Flags area, select the relevant check boxes to modify the advertise actions for the routes.
    2. Select Update to save the changes.

    The updated settings are displayed on the Overlay Flow Control page.

Configure Subnets

In the Overlay Flow Control window, you can update the priorities of the destinations for the learned routes in the subnets.
  1. In the SD-WAN service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. The Routes List section of the Overlay Flow Control window displays the subnets with the following details, as show in the image and table below.
    Figure 40. Routes List

     

    Table 23. Routes List Option Descriptions
    Option Description
    Segment Segment name.
    Subnet The network that the route corresponds to along with a list of Edges that learned the route.
    Preferred VPN Exits The route through which another branch can access the subnet.
    Route Type Displays the type of the route, which can be one of the following: Static, Connected, or Learned.
    Last Update The last updated date and time of the preferred VPN exit.
    Created On Date and time when the route was created.
    IPv4 Subnet  
    Eligible VPN Exits  
    Note: You can configure the subnets for both IPv4 and IPv6 addresses.
    Currently, you can view up to 256 subnet prefixes in the API call request. You can use the Filter option to search for the specific subnet prefix. The following information message appears if the results are more than the server could return. 
    There are more results that the server could return. Please narrow your search criteria
    Select one or more subnets and select MORE to perform the following activities:
    • Pin Learned Route Preference – Pins the preferences of the selected learned route.
    • Unpin Learned Route Preference – Unpins the preference of the selected learned route to default settings.
    • Delete Learned Routes – Deletes the learned routes. This option does not delete the connected routes, static routes, routes from Overlay Flow Control, and routes from Edge Route table. The option is available only when Configure Distributed Cost Calculation is turned off.
  3. Select the Edit Subnet option for a subnet to modify the priorities of the preferred destination.
    1. In the Subnet window, you can move the destinations from the Eligible VPN Exits to Preferred VPN Exits and vice versa.
      Figure 41. Edit Subnet
    2. In the Preferred VPN Exits panel, select the UP and DOWN arrows to change the priorities and select Update.
    3. You can reset the cost calculation for the subnets when there are pinned routes available. Select Reset, which enables the Orchestrator to clear the pinned routes, recalculate the cost for the selected subnet based on the policy, and send the results to the Edges and Gateways.
      Note: For IPv4 Routes, the Reset option is available only when Distributed Cost Calculation is enabled.

      For additional information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the Arista VeloCloud SD-WAN Operator Guide.