印刷

Configuring Dynamic Routing with OSPF or BGP

This section discusses configuring dynamic routing with OSPF or BGP.

The Edge learns routes from adjacent routers through OSPF and BGP, and then sends the learned routes to the Gateway/Controller. The Gateway/Controller acts as a route reflector and sends the learned routes to other Edge. The Overlay Flow Control (OFC) enables enterprise-wide route visibility and control for ease of programming and for full and partial overlay.

VeloCloud supports Inbound/Outbound filters to OSPF neighbors, OE1/OE2 route types, MD5 authentication. Routes learned through OSPF automatically redistribute to the controller hosted in the cloud or on-premise. Support for BGP Inbound/Outbound filters and the filter can be set to Deny, or optionally you can add or change the BGP attribute to influence the path selection, such as RFC 1998 community (https://datatracker.ietf.org/doc/html/rfc1998), MED, and local preference.

Note: See OSPF and BGP Redistribution for more information.

Activating OSPF for Profiles

Users can enable Open Shortest Path First (OSPF) only on a LAN interface as an active or passive interface. The Edge advertises only the prefix associated with that specific LAN switch port. To access full OSPF functionality, the user must use OSPF on routed interfaces.

OSPF (Open Shortest Path First) is an interior gateway protocol (IGP) that operates within a single autonomous system (AS).
Note: Configure OSPF only on the Global Segment.
OSPFv3 provides support for the following:
  • VeloCloud introduced support for OSPFv3 in the SD-WAN Edge for IPv6 underlay routing feature in addition to the existing BGPv6 support and supports the following:
    • Underlay IPv6 route learning
    • Redistribution of OSPFv3 routes into overlay/BGP and vice-versa
    • Support for Overlay Flow Control (OFC).
  • OSPFv3 has feature parity with OSPFv2 and does not support the following:
    • Point to Point (P2P)
    • BFDv6 with OSPFv3
    • OSPFv3 header therefore no MD5 authentication available
Note: OSPFv2 supports only IPv4. OSPFv3 supports only IPv6 and is available starting with the 5.2 release.
Note: Route Summarization available starting with the 5.2 release.

To activate OSPF, perform the following steps:

  1. In the SD-WAN service of the Enterprise Portal, select Configure.
    Note: Depending upon your login permissions, you might need to select a Customer or Partner profile first, then select Configure.
  2. Select Profiles to display the list of available Profiles.
  3. Select a Profile from the list.
  4. In the Routing and NAT section, select OSPF.
  5. Under OSPF Areas, configure the Redistribution Settings for OSPFv2/3, BGP Settings, and if applicable, configure Route Summarization.
    Figure 1. Configuring OSPF
    Table 1. OSPF Option Descriptions
    Option Description
    Redistribution Settings  
    Default Route Select an OSPF route type (O1 or O2) to use for the default route. The configuration uses None by default.
    Advertise Select either Always or Conditional. Selecting Always advertises the default route always. Selecting Conditional means to redistribute default route only when the Edge learns it through overlay or underlay. Check the Overlay Prefixes option to use the Conditional default route.
    Overlay Prefixes If applicable, check Overlay Prefixes.
    BGP Settings  
    BGP To enable injection of BGP routes into OSPF, select the BGP. BGP routes can be redistributed into OSPF, so if this is applicable, enter or choose the configuration options as follows:
    Set Metric Enter the metric in Set Metric. This is the metric that OSPF puts in external LSAs that it generates from the redistributed routes). The default metric is 20.
    Set Metric Type From the Set Metric Type menu, select a metric type. This is either type E1 or E2 OSPF External-LSA type. The default type is E2).
  6. In OSPF Areas, select +Add and configure the options. Add additional areas, if necessary, by selecting +Add. The fields in the table below cannot be overridden at the Edge level.
    Table 2. OSPF Areas Option Descriptions
    Option Description
    Area ID Enter an OSPF Area ID.
    Name Enter a descriptive name.
    Type Normal Type selected by default as the only supported option.
  7. Next, configure the Interface Settings for OSPF. For configuration details, see either Configure Interface Settings for Profiles with New Orchestrator UI or Configure Interface Settings for Edges with New Orchestrator UI.
    Note: Activate OSPF at the Profile level first before configuring it on Edge interfaces.
  8. If applicable, configure Route Summarization.
    Note: The Route Summarization feature became available with the 5.2 release, for an overview and use case for this feature, see Route Summarization.
  9. Locate the Route Summarization area.
  10. Select +Add in the Route Summarization area to add a new row. Configure Route Summarization as described in the table below.
    Figure 2. Configuring Route Summarization
    Table 3. Route Summarization Option Descriptions
    Option Description
    Subnet Enter the IP subnet.
    No Advertise When setting No Advertise, all the external routes (Type-5) under this supernet summarize and do not advertise it. This means it effectively blocks the whole supernet from advertising to its peer.
    Tag Enter the router Tag value from 1 to 4294967295.
    Metric Type Enter the Metric Type as E1 or E2.
    Metric Enter the advertised metric for this route from 0 to 16777215.
  11. Add additional routes, if necessary, by selecting +Add. Use Clone or Delete to clone or delete a route summarization.
  12. Select Save Changes.

Route Filters

There are two different types of routing - Inbound and Outbound.
  • Inbound routing includes preferences that can be learned or ignored from OSPF and installed into the Overlay Flow Control.
  • Outbound Routing indicates what prefixes can be redistributed into the OSPF.

Activate OSPF for Edges

Users can enable Open Shortest Path First (OSPF) on a LAN (routed or switched) or a WAN interface. Currently, the system does not provide an option to override the passive interface setting for OSPF on a VLAN interface, as the Edge only supports OSPF on a switched interface as a passive interface. The Edge only advertises the prefix associated with that LAN switch port. To get full OSPF functionality, users must use it on routed interfaces. After users configure the OSPF settings at the Profile level, all the Edges associated with the Profile inherit the OSPF configuration from the Profile. However, users cannot override the OSPF configuration settings at the Edge level.

Note: Edges running lower versions, 6.0 or lower, do not process OSPF configurations in non-global segments even though Orchestrator allows OSPF configuration at the Profile level.

Use the following steps to view OSPF configuration for a specific Edge device:

  1. In the SD-WAN service of the Enterprise Portal, select Configure > Edges .
  2. Select Device next to an Edge, or select the link to an Edge and then select the Device tab.
  3. Go to the Routing & NAT section and select OSPF.
  4. In the OSPF section, you can view all the inherited OSPF configuration such as OSPF areas, Redistribution settings for OSPFv2/v3, BGP settings, and Route Summarization.
    Figure 3. Activating OSPF for Edges

Configuring BGP

You can configure the BGP per segment for a Profile or an Edge, and configure BGP for Underlay Neighbors and Non SD-WAN Neighbors. VeloCloud supports 4-Byte ASN BGP as follows:
  • As the ASN of Edges
  • Peer to a neighbor with 4-Byte ASN
  • Accept 4-Byte ASNs in route advertisements

Configure BGP from Edge to Underlay Neighbors for Profiles

You can configure the BGP per segment at the Profile level as well as at the Edge level. This section provides steps on how to configure BGP with Underlay Neighbors.

VeloCloud supports 4-Byte ASN BGP. See Configure BGP for more information.

Note: Route Summarization is new for the 5.2 release. For an overview, use case, and black hole routing details for Route Summarization, see section titled, Route Summarization.
  1. In the SD-WAN service of the Enterprise Portal, select Configure.
  2. Select Profiles to display the list of available profiles.
  3. Select a Profile from the list of available Profiles or Add a Profile if necessary.
  4. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  5. In the BGP area, enabled BGP by selecting On.
    Figure 4. Quick Start Profile for BGP
  6. In the BGP area, enter the local Autonomous System Number (ASN) number in the appropriate text field.
  7. Configure BGP Settings:
    Table 4. BGP Settings Option Descriptions
    Option Description
    Router ID Enter the global BGP router ID. If you do not specify any value, the ID assigns automatically. If you configured a loopback Interface for the Edge, the IP address of the loopback Interface assigns as the router ID.
    Keep Alive Enter the keep alive timer in seconds, the duration between the keep alive messages sent to the peer. Use a range from 0 to 65535 seconds with a default value of 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Uplink Community Enter the community string to be treated as uplink routes. Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes. Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
    Enable Graceful Restart Please note when selecting this check box:The local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.
       
  8. Select +Add in the Filter List area to create one or more filters. Apply the filters to a neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 5. Adding BGP Filters
  9. Add the Filter rules.
    Table 5. Filter Rules Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to be matched with the filter:
    • Prefix for IPv4 or IPv6: Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
    • Community: Choose to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the Prosecutes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when Thebes routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None- The attributes of the matching routes remain the same.
    • Local Preference- The matching traffic routs to the path with the specified local preference.
    • Community- The matching routes filter by the specified community string. You can also select Community Additive to enable the additive option, which appends the community value to existing communities.
    • Metric- The matching traffic routs to the path with the specified metric value.
  10. Select plus (+) to add more matching rules for the filter. Repeat the procedure to create more BGP filters.
    The configured Filters display in the Filter List.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
  11. Navigate to Neighbors and select +Add.
    Figure 6. Adding Neighbors
  12. Configure the following settings for the IPv4 addressing type:
    Table 6. IPv4 Addressing Type Option Descriptions
    Option Description
    Neighbor IP Enter IP address of the BGP neighbor.
    ASN Enter the ASN of the neighbor.
    Inbound Filter Select an Inbound Filter from the list.
    Outbound Filter Select an Outbound Filter from the list.
       
    Note: When overriding and configuring BGP neighbors at the Edge level, any Profile-level filters associated with the neighbors will be removed when you switch the Edge from one profile to another. So at the Edge level, you must make sure to reassociate the filters with the BGP neighbors after switching the Edge profile.
    Table 7. Additional Option Descriptions
    Option Description
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. The range is from 1 to 255 and the default value is 1.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is inherent by default and this field is not configurable.
    Local IP Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address forth outgoing packets. If you do not enter any value, the IP address of the physical Interface is used as the source IP address.
    Note: For eBGP, this field is available only when Max- hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Uplink Used to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes leant over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, also check "Overlay Prefix Over Uplink" in the Advanced Settings area.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character < in the password is no longer supported. In cases where users have already used < in their passwords in previous releases, they must remove it to save any changes on the page.
       
  13. Select (+) to add more BGP neighbors.
    Note:

    Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for destination that needs these routes to be looked up, routes requiring recursive lookup resolve to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For more information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the VeloCloud SD-WAN Troubleshooting Guide.

  14. Navigate to Advanced Settings and select the down arrow to open the Advanced Settings section.
    Figure 7. Configuring Advanced Settings
  15. Configure the following advanced settings and apply them globally to all the BGP neighbors with IPv4 addresses.
    Table 8. Advanced Settings Option Descriptions
    Option Description
    Overlay Select the check box to redistribute the prefixes learned from the overlay. For example, when a Spoke is connected to primary and secondary Hub or Hub Cluster, the Spoke's subnets are redistributed by primary and secondary Hub or Hub Cluster to their neighbor with metric (MED) 33 and 34 respectively. You must configure "bgp always-compare-med" in the neighbor router for symmetric routing.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to deactivate AS-PATH Carry Over. In certain topologies, deactivating AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Warning: When the AS-PATH Carry Over is deactivated, tune your network to avoid routing loops.
    Connected Routes Select the check box to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select the check box to redistribute the default route only when Edge learns the BGP routes through overlay or underlay. When you select the Default Route option, the Advertise option is available as Conditional.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address in IPv4 format that BGP will be advertising to the peers. Select the plus + icon to add more network addresses.
       

    When you enable the Default Route option, the BGP routes advertise based on the Default Route selection globally and per BGP neighbor.

    Table 9. Default Route Selections
    Default Route Selection    
    Global Per BGP Neighbor Advertising Options
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.
         
  16. Select the IPv6 tab to configure the BGP settings for IPv6 addresses. Enter a valid IPv6 address of the BGP neighbor in the Neighbor IP field. The BGP peer for IPv6 supports the following address format:
    • Global unicast address (2001:CAFE:0:2:1)
    • Unique Local address (FD00:1234:BEFF:ACE:E0A4)
  17. Configure the other settings as required.
    Note: The Local IP address configuration is not available for IPv6 address type.
  18. Select Advanced to configure the following advanced settings, which globally apply to all the BGP neighbors with IPv6 addresses.
    Table 10. Advanced Option Descriptions
    Option Description
    Connected Routes Select to redistribute all the connected Interface subnets.
    Default Route Select to redistribute the default route only when Edge learns the BGP routes through overlay or underlay. When you select the Default Route option, the Advertise option is available as Conditional.
    Networks Enter the network address in IPv6 format that BGP advertises to the peers. Select (+) to add more network addresses.
       

    Route Summarization

    The Route Summarization feature is available in the 5.2 release, for an overview and use case of this functionality, see Route Summarization.

  19. Select +Add in the Route Summarization area and add a new to the Route Summarization area.
    Figure 8. Adding Route Summarization
  20. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
  21. Under the AS Set column, select Yes if applicable.
  22. Under the Summary Only column, select Yes to send only the summarized route.
  23. Add additional routes, if necessary, by selecting +Add. To Clone or Delete a route summarization, use the appropriate buttons, located next to +Add.
    The BGP Settings section displays the BGP configuration settings.
  24. Select Save Changes when complete to save the configuration.
    Note: When you configure BGP settings for a profile, the configuration settings are automatically applied to the SD-WAN Edges that are associated with the profile.

    You can also configure BGP for Non SD-WAN Destination Neighbors in an Edge. For more information, see Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors.

Configuring BGP from Edge to Underlay Neighbors for Edges

You can override the inherited Profile settings at the Edge level when configuring BGP from the Edge to Underlay Neighbors.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges . The Edges page displays the configured Edges.
  2. Select on an Edge or select View in the Device column of the Edge.
  3. Go to the Routing & NAT section and select the arrow next to BGP to expand.
  4. The BGP Settings configured for the associated Profile are displayed. If required, you can select Override and modify the BGP Settings.
    Note: Overriding and configuring BGP neighbors at the Edge level removes any Profile-level filters associated with the neighbors when you switch the Edge from one profile to another. At the Edge level, you must make sure to reassociate the filters with the BGP neighbors after switching the Edge profile.
  5. In addition to the BGP Settings configured for a Profile, you can select an Edge Interface configured in the segment as the source Interface for BGP. For the IPv4 address type, you can select only the Loopback Interface as Source Interface and for the IPv6 address type, you can select any Edge Interface as the Source Interface. This field becomes available:
    • Only when you choose to override the BGP Settings at the Edge level.
    • For eBGP, only when Max-hop count is more than 1. For iBGP, it is always available as iBGP is inherently multi-hop.
    Important:
    • You cannot select an Edge Interface if you have already configured a local IP address in the Local IP field.
    • You cannot configure a local IP address if you have selected an Edge Interface in the Source Interface list.
  6. Select Save Changes to save the modified configuration.

BGP over IPsec from Edge to Non SD-WAN Neighbors Overview

The Non SD-WAN BGP Neighbors configuration does not apply to the Profile level. You can configure the NSD Neighbors only at the Edge level.

BGP establishes the BGP neighborship over the IPSec tunnels to the Non SD-WAN Sites. Direct IPSec tunnels establish a secure communication between the SD-WAN Edge and the Non SD-WAN Destination (NSD). In previous releases, VeloCloud supported NSD tunnels from the SD-WAN Edge with the ability to add NVS static routes. In the 4.3 release, this functionality extends support to BGP over IPSec to the NSD endpoint for a route-based VPN.

VeloCloud SD-WAN supports 4-Byte ASN BGP. See Configure BGP for more information.
Note: The Azure vWAN Automation from Edge feature does not provide compatibility with BGP over IPSec because it supports only static routes when automating connectivity from an Edge to an Azure vWAN.

Each Azure VPN gateway allocates one set of public Virtual Public IPs (VIP) for a branch Edge to form IPSec tunnels. Similarly, Azure also allocates one internal private subnet and assigns one internal IP per VIP. This internal tunnel-ip (peer tunnel-ip) will be used for creating BGP peering with the Azure Gateway.

Azure has a restriction that the BGP peer IP (Edge's local tunnel-ip) cannot exist in the same connected subnet or 169.x.x.x subnet, and therefore VeloCloud supports multi-hop BGP on the Edge. In BGP terminology, the local tunnel-ip maps to BGP source address and the peer tunnel-ip maps to neighbor or peer address. The configuration must form a mesh of BGP connections- one per NSD tunnel so that the return traffic from the NVS can be load-balanced (flow-based) on the Azure Gateway side. The example network diagram for the physical Edge displays two public WAN links and therefore consists of four tunnels to an Azure Gateway. Each tunnel associates with one BGP connection uniquely identified by the local tunnel_ip and remote peer tunnel_ip. On the Virtual Edge, the configuration has one public WAN link and a maximum of two tunnels and two BGP sessions to the Azure Gateway.

Figure 9. BGP Over IPSec from an Edge to an Azure VPN
Note: When an SD-WAN Edge connects to the same Azure end-point using multiple WAN links, a maximum of two NSD-BGP neighbors can be configured since remote end has only two public-ips and two NSD-BGP peer-ips. Both NSD-BGP neighbors can be configured on the same link (primary/secondary tunnel), or tunnels on different links. If you attempt to configure more than two NSD-BGP neighbors and configure the same NSD-BGP peer-ip on more than one tunnel with the last configured BGP nbr-ip + local-ip on the SD-WAN Edge and Free Range Routing (FRR).

Use Case 1- BGP Over IPSec from an Edge to an Azure VPN

Unlike Azure, AWS VPN Gateway allocates one set of public VIPs per link to a branch Edge. The total sets of public IPs allocated to a branch Edge from an AWS Gateway equals to the number of Edge public WAN links that connects to the AWS VPN Gateway. Similarly, a /30 internal/private subnet allocates per tunnel used for BGP peering on that tunnel. These IPs can be manually overridden in AWS Gateway configuration to ensure uniqueness across different availability zones.

Similar to the Azure use-case, the Edge forms a mesh of BGP connections- one per tunnel to the AWS gateway. This allows load-balancing of the return traffic from the AWS VPN Gateway- design on the AWS side. In the example diagram, for the physical Edge, the AWS Gateway allocates one set of public IPs and one set of tunnel-ips (/30) for each Edge WAN link for a total of four tunnels, and terminate in different public IPs on the AWS Gateway and four BGP connections.

Figure 10. BGP Over IPSec from Edge to AWS VPN and Transit Gateway

Use Case 3- An Edge Connecting to Both AWS and Azure VPN Gateways (Hybrid Cloud)

One branch Edge could be connected to both Azure Gateway and AWS Gateway for redundancy purposes or some workloads and apps hosted in one cloud provider while other workloads and apps host in a different cloud provider. Regardless of the use-case, the Edge always establishes one BGP session per tunnel and propagates the routes between SD-WAN and IaaS. The diagram below is an example of one branch Edge connected to both Azure and AWS clouds.

Figure 11. An Edge Connecting to Both AWS and Azure VPN Gateways (Hybrid Cloud)

Use Case 4- A Hub Cluster Connecting to Azure and AWS Transit Gateways

The Hub cluster members can form IPSec tunnels to the Azure and AWS transit Gateways and leverage the transit Gateways as Layer 3 for routing traffic between different VPCs. Without the native BGP over IPSec functionality on Hub, the Hub needs to connect to an L3 routerusing native BGP and the L3 router forming a mesh of BGP over IPSec tunnels with different VPCs. The L3 router serves as a transit end-point between different VPCs.

Use Case-1 uses a Hub as a transit node between different VPCs in different Availability Zones (AZ) so that one VPC can communicate to another VPC. Use Case-2 connects all Hubs in the cluster directly to a cloud transit gateway and can use the cloud gateway as a PE(L3) router for routes distribution between cluster members. In both use-cases, without the support for BGP over IPSec on the Hub, the Hub connects to an L3 router like CSR using native BGP and CSR peers with a transit and VPC gateway using BGP over IPSec.

Figure 12. A Hub Cluster Connecting to Azure/AWS Transit Gateways

Use Case 5- Supporting Transit Functionality in Cloud Providers without Native Support

Some cloud providers such as Google Cloud and AliCloud do not have native support for transit functionality (no transit Gateways), and with the support for BGP over IPSec, can rely on SD-WAN Edge/Hub deployed in the cloud to achieve the transit functionality between different VPCs/VNETs. Without BGP over IPSec support, you must use an L3 router to achieve the transit functionality.

Note:

Prior to the 4.3 release, for customers with reachability to the same NVS-Static destination through an NVS-From-Gateway and NVS-From-Edge, the traffic from other branch SD-WAN Edges prefers the path through a NVS-Gateway. When you upgrade your network to the 4.3 release or later, this traffic path from other branch- SD-WAN Edges prefers the path through the NVS-Edge. Therefore, you must update the NVS-Static-Destination metric of the NSD-Edge and the NSD-Gateway as per the traffic path preference.

Next Steps- See Configuring BGP over IPSec from Edge to Non SD-WAN Neighbors.

Configuring BGP over IPSec from Edge to Non SD-WAN Neighbors

 

Prerequisites

Use the following steps to enable BGP with Non SD-WAN neighbors:

  1. In the SD-WAN service of the Enterprise Portal, select Configure.
  2. From the left menu, select Edges. The Edges page displays.
  3. Select an Edge from the list of available Edges.
  4. Go to the Routing & NAT section in the UI and select the arrow next to BGP.
  5. In the BGP area, select Overrideand toggle the radio button from Off to On.
    Figure 13. Configuring BGP Settings

    In BGP Editor, configure the following settings:

  6. Enter the local Autonomous System Number (ASN) and then configure the following in the BGP Settings section.
  7. Configure the BGP Settings:
    Table 11. BGP Settings Option Descriptions
    Option Description
    Router ID Enter the global BGP router ID. If you do not specify any value, the ID is automatically assigned. If you have configured a loopback Interface for the Edge, the IP address of the loopback Interface will be assigned as the router ID.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Uplink Community Enter the community string to be treated as uplink routes. Uplink refers to link connected to the Provider Edge(PE). Inbound routes towards the Edge matching the specified community value will be treated as Uplink routes. The Hub/Edge is not considered as the owner for these routes. Enter the value in number format ranging from 1 to 4294967295 or in AA:NN format.
    Enable Graceful Restart Please note when enabling graceful restart- The local router does not support forwarding during the routing plane restart. This feature supports preserving forwarding and routing in case of peer restart.
       

     

  8. Select +Add in the Filter List area to create one or more filters. Apply these filters to the neighbor and deny or change the attributes of the route. The same filter can be used for multiple neighbors.
    Figure 14. Adding Filters to the BGP Configuration
  9. In the appropriate fields, set the rules for the filter:
    Table 12. Filter Rules Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to match with the filter:
    • Prefix for IPv4 or IPv6- Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
    • Community- Select to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None- The attributes of the matching routes remain the same.
    • Local Preference- The matching traffic is routed to the path with the specified local preference.
    • Community- The matching routes are filtered by the specified community string. You can also select Community Additive to enable the additive option, which appends the community value to existing communities.
    • Metric- The matching traffic is routed to the path with the specified metric value.
    • AS-Path-Prepend- Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.
       
  10. To add more matching rules to the filter, select the Plus (+) icon.
  11. Select OK to create the filter. The configured filters display in BGP Editor.
  12. Configure Underlay Neighbors for IPv4 and IPv6 addresses, as required. For more information, see Configure BGP from Edge to Underlay Neighbors for Edges.
    Note: The maximum number of supported BGPv4 Match/Set rules is 512 (256 inbound, 256 outbound). Exceeding 512 total Match/Set rules is not supported and may cause performance issues, resulting in disruptions to the enterprise network.
  13. In the NSD Neighbors section, configure the following settings:
    Table 13. NSD Neighbors Option Descriptions
    Option Description
    NSD Name Select the NSD Name from the list. The NSDs already configured in the Branch to Non SD-WAN Destination via Edge area of the Orchestrator display in the list.
    Link Name Select the name of the WAN link associated with the NSD neighbor.
    Tunnel Type Choose the tunnel type of the Peer as Primary or Secondary.
    Neighbor IP Enter the IP address of the NSD neighbor.
    ASN Enter the ASN for the NSD neighbor.
    Inbound Filter Select an Inbound filter from the list.
    Outbound Filter Select an Outbound filter from the list.
    Additional Options – Select View All to configure the following additional settings:
    Uplink Use to flag the neighbor type to Uplink. Select this flag option if it is used as the WAN overlay towards MPLS. It will be used as the flag to determine whether the site will become a transit site (e.g. SD-WAN Hub), by propagating routes leant over a SD-WAN overlay to a WAN link toward MPLS. If you need to make it a transit site, select the Overlay Prefix Over Uplink check box in the Advanced Settings.
    Local IP Local IP is mandatory for configuring Non SD-WAN Neighbors. Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
    Note: When upgrading to the 5.1 release, any max-hop value of 1 automatically updates to a max-hop value of 2.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is deactivated by default and this field is not configurable.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Note: Single-hop BFD session is not supported for BGP over IPSec with NSD Neighbors. However, multi-hop BFD is supported. Local IP is mandatory for NSD-BGP sessions on the SD-WAN Edge. The SD-WAN Edge handles only the connected Interface IPs as a single-hop BFD.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character < in the password is no longer supported. In cases where users have already used < in their passwords in previous releases, they must remove it to save any changes on the page.
       
    Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and interface. Until the recursive resolution happens, the recursive routes point to an intermediate interface. For more information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the Arista VeloCloud SD-WAN Troubleshooting Guide
  14. Select Advanced to configure the following settings:
    Note: Advanced Settings are shared across both the underlay BGP neighbors and NSD BGP neighbors.
    Table 14. Advanced BGP Settings Option Descriptions
    Option Description
    Overlay Prefix Select the check box to redistribute the prefixes learned from the overlay.
    Turn off AS-Path carry over By default, this should be left unchecked. Select the check box to turn off AS-PATH Carry Over. In certain topologies, turning off AS-PATH Carry Over will influence the outbound AS-PATH to make the L3 routers prefer a path towards an Edge or a Hub.
    Warning: When the AS-PATH Carry Over is turned off, tune your network to avoid routing loops.
    Connected Routes Select to redistribute all the connected Interface subnets.
    OSPF Select the check box to enable OSPF redistribute into BGP.
    Set Metric When you enable OSPF, enter the BGP metric for the redistributed OSPF routes. The default value is 20.
    Default Route Select to redistribute the default route only when Edge learns the BGP routes through overlay or underlay. When you select the Default Route option, the Advertise option is available as Conditional.
    Overlay Prefixes over Uplink Select the check box to propagate routes learned from overlay to the neighbor with uplink flag.
    Networks Enter the network address in IPv6 format that BGP advertises to the peers. Select (+) to add more network addresses.
       

    When you enable the Default Route option, the BGP routes are advertised based on the Default Route selection globally and per BGP neighbor:

    Table 15. Default Route Selection
    Default Route Selection
    Global Per BGP Neighbor Advertising Options
    Yes Yes The per BGP neighbor configuration overrides the global configuration and hence default route is always advertised to the BGP peer.
    Yes No BGP redistributes the default route to its neighbor only when the Edge learns an explicit default route through the overlay or underlay network.
    No Yes Default route is always advertised to the BGP peer.
    No No The default route is not advertised to the BGP peer.
  15. Select OK to save the configured filters and NSD Neighbors.

    The BGP Settings section displays the configured settings.

  16. To configure Route Summarization, select +Add in the Route Summarization area and configure the required settings. For additional details, see Route Summarization.
    Figure 15. Configuring Route Summarization
    1. Under the Subnet column, enter the network range that you want to summarize in the A.B.C.D/M format and the IP subnet.
    2. Under the AS Set column, click the Yes check box if applicable.
    3. Under the Summary Only column, click the Yes check box to allow only the summarized route to be sent.
    4. Add additional routes, if necessary, by clicking +Add. To Clone or Delete a route summarization, use the appropriate buttons, located next to +Add.
  17. Select Save Changes when complete to save the configuration.
You can also configure BGP from Edge to underlay neighbors. For more information, see Configure BGP from Edge to Underlay Neighbors for Edges.

Configuring BGP Over IPsec from Gateways

Only eBGP is supported with BGP over IPsec.

You can configure BGP Settings for Gateways over IPsec tunnels.

Note: It is recommended to use eBGP between SDWAN Gateway and NSD sites. If using iBGP, applying local preference does not work with an outbound filter. In that case, you must choose metric or AS path prepend options to achieve desirable routing.

VeloCloud allows Enterprise users to define and configure a Non SD-WAN Destination instance in order to establish a secure IPsec tunnel to a Non SD-WAN Destination through an SD-WAN Gateway.

Note: For the 5.2 release, when multiple NSDs are configured for the same segment, the same set of summary route configurations must be present across all NSDs.
Note: The Azure vWAN Automation from Gateway feature is not compatible with BGP over IPsec. This is because only static routes are supported when automating connectivity from a Gateway to an Azure vWAN.
Important: Distributed Cost Calculation (DCC) is mandatory for ECMP to work properly.
Note: It is recommended to enable Distributed Cost Calculation for best performance and scaling when using BGP over IPsec via Gateway. The Distributed Cost Calculation is supported starting from Release 3.4.0. For more information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the VeloCloud SD-WAN Operator Guide.
  1. Go to Configure > Network Services and then under Non SD-WAN Destinations, expand Non SD-WAN Destinations via Gateway.
    Note: The New NSD via Gateway option appears only when there are no items in the table. Follow Steps 2 and 3 to create a new Non SD-WAN Destination.
    Figure 16. Configuring a Non SD-WAN Destination
  2. Select +New to create a new Non SD-WAN Destination.
    Figure 17. Adding a Non SD-WAN Destinations via Gateway
  3. In Non SD-WAN Destinations via Gateway, configure the following fields:
    Table 16. Non SD-WAN Destinations via Gateway Option Descriptions
    Option Description
    Name Enter a name for the Non SD-WAN Destination in the text box.
    Type Select an IPsec tunnel type from the menu.
    Tunnel Mode Active/ Hot-Standby mode supports to set up a maximum of 2 tunnel endpoints or Gateways.
    Active/Active mode supports to set up a maximum of 4 tunnel endpoints or Gateways. All Active tunnels can send and receive traffic through ECMP.
    VPN Gateway 1 Enter a valid IP address.
    VPN Gateway 2 Enter an optional valid IP address.
       

     

  4. Select Create to add the new destination.

  5. In the Non SD-WAN Destination via Gateway area, slide the grey bar to the far right to the BGP column.

    Select the Edit link under the BGP column.

    If the Edit link does not display under the BGP column, see the section Configure a Tunnel Between a Branch and a Non SD-WAN Destinations via Edge to enable an Edge to Non SD-WAN via Gateway.

    After selecting the Edit link under the BGP column, the Edit BGP dialog displays.

  6. Toggle the BGP Activated radio button to the right to turn it green.
  7. Select +Add to create one or more filters. These filters are applied to the neighbor to deny or change the attributes of the route. The same filter can be used for multiple neighbors.
  8. Configure the options in the Filter List area:
    Table 17. Filter List Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Match Type and Value Choose the type of the routes to match with the filter:
    • Prefix for IPv4 or IPv6- Choose to match with a prefix for IPv4 or IPv6 address and enter the corresponding prefix IP address in the Value field.
    • Community- Select to match with a community and enter the community string in the Value field.
    Exact Match The filter action is performed only when the BGP routes match exactly with the specified prefix or community string. By default, this option is enabled.
    Action Type Choose the action to be performed when the BGP routes match with the specified prefix or the community string. You can either permit or deny the traffic.
    Action Set When the BGP routes match the specified criteria, you can set to route the traffic to a network based on the attributes of the path. Select one of the following options from the drop-down list:
    • None- The attributes of the matching routes remain the same.
    • Local Preference- The matching traffic is routed to the path with the specified local preference.
    • Community- The matching routes are filtered by the specified community string. You can also select Community Additive to enable the additive option, which appends the community value to existing communities.
    • Metric- The matching traffic is routed to the path with the specified metric value.
    • AS-Path-Prepend- Allows pre-pending multiple entries of Autonomous System (AS) to a BGP route.
       
  9. Select (+) to add more matching rules for the filter. Repeat the procedure to create more filters.
    Figure 18. Displaying the Filter List
    Figure 19. Editing BGP Neighbors
    Note: These BGP neighbors are assigned to their respective tunnels exclusively for neighborship establishment and subsequent control exchanges, ensuring these communication occurs solely over the designated tunnels.
  10. In the BGP Editor window, configure the BGP settings for the Primary and Secondary Gateways.
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Note: For a customer deployment where a Non VeloCloud SD-WAN Destination (NSD) via Gateway is configured to use redundant tunnels, if the Primary and Secondary Gateways advertise a prefix with an equal AS path to the Primary and Secondary NSD tunnels, the Primary NSD tunnel will prefer a redundant Gateway path over the Primary Gateway. The impact of the Primary NSD over Gateway tunnel preferring the redundant Gateway path over the Primary Gateway is experienced only for return traffic to the Gateway from the NSD. If you do not want your BGP router to prefer the redundant Gateway, the workaround is to configure AS-PATH prepend and set the metric filter to a higher (3 or more) metric for the advertised prefix in the redundant Gateway. Doing this ensures the NSDs primary tunnel chooses the Primary Gateway for return traffic.
  11. In the Primary Cloud Gateway section, enter the local ASN and the Router ID.
  12. Scroll down to the Neighbors area and select +Add.
  13. Configure the following settings in the Neighbors area:
    Figure 20. Configuring BGP Neighbors
    Table 18. BGP Neighbors Option Descriptions
    Option Description
    Local ASN Enter the local Autonomous System Number (ASN)
    Router ID Enter the BGP Router ID.
    Neighbor IP Enter the IP address of the NSD neighbor.
    ASN Enter the ASN for the NSD neighbor.
    Inbound Filter Select an Inbound filter from the list.
    Outbound Filter Select an Outbound filter from the list.
    Additional Options – Select View All to configure the following additional settings:
    Local IP Local IP is mandatory for configuring Non SD-WAN Neighbors. Local IP address is the equivalent of a loopback IP address. Enter an IP address that the BGP neighborships can use as the source IP address for the outgoing packets.
    Max-hop Enter the number of maximum hops to enable multi-hop for the BGP peers. For the 5.1 release and later, the range is from 2 to 255 and the default value is 2.
    Note: When upgrading to the 5.1 release, any max-hop value of 1 automatically updates to a max-hop value of 2.
    Note: This field is available only for eBGP neighbors, when the local ASN and the neighboring ASN are different. With iBGP, when both ASNs are the same, multi-hop is deactivated by default and this field is not configurable.
    Allow AS Select the check box to allow the BGP routes to be received and processed even if the Edge detects its own ASN in the AS-Path.
    Default Route The Default Route adds a network statement in the BGP configuration to advertise the default route to the neighbor.
    Enable BFD Enables subscription to existing BFD session for the BGP neighbor.
    Note: Single-hop BFD session is not supported for BGP over IPSec with NSD Neighbors. However, multi-hop BFD is supported. Local IP is mandatory for NSD-BGP sessions on the SD-WAN Edge. The SD-WAN Edge handles only the connected Interface IPs as a single-hop BFD.
    Keep Alive Enter the keep alive timer in seconds, which is the duration between the keep alive messages that are sent to the peer. The range is from 0 to 65535 seconds. The default value is 60 seconds.
    Hold Timer Enter the hold timer in seconds. When the keep alive message is not received for the specified time, the peer is considered as down. The range is from 0 to 65535 seconds. The default value is 180 seconds.
    Connect Enter the time interval to try a new TCP connection with the peer if it detects the TCP session is not passive. The default value is 120 seconds.
    MD5 Auth Select the check box to enable BGP MD5 authentication. This option is used in a legacy network or federal network, and it is common that BGP MD5 is used as a security guard for BGP peering.
    MD5 Password Enter a password for MD5 authentication.
    Note: Starting from the 4.5 release, the use of the special character < in the password is no longer supported. In cases where users have already used < in their passwords in previous releases, they must remove it to save any changes on the page.

    The configured Neighbors are displayed in the Neighbors area.

    Figure 21. Displaying Configured Neighbors
    Note: Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit Interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit Interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and Interface. Until the recursive resolution happens, the recursive routes point to an intermediate Interface. For more information about Multi-hop BGP Routes, see the "Remote Diagnostic Tests on Edges" section in the VeloCloud SD-WAN Troubleshooting Guide.
  14. Select Save Changes.

    Route Summarization

    The Route Summarization feature is available in the 5.2 release, for an overview and use case of this functionality, see Route Summarization. For configuration details, follow the steps below.

  15. Scroll down to the Route Summarization area.
  16. Select +Add in the Route Summarization to add a new row to the Route Summarization table. Configure route summarization, as described in the table below.
    Table 19. Route Summarization Option Descriptions
    Option Description
    Filter Name Enter a descriptive name for the BGP filter.
    Subnet Enter the IP subnet.
    AS Set Generate AS set path information from the summarized routes (while advertising the summary route to the peer). Under the AS Set column, select Yes if applicable.
    Summary Only Select Yes to allow only the summarized route to be sent.

     

  17. Add additional routes, if necessary, by selecting +Add. To Clone or Delete a route summarization, use the appropriate buttons, located next to +Add.

    The BGP Settings section displays the BGP configuration settings.

    Figure 22. Displaying Route Summarization
  18. Select Save Changes when complete to save the configuration.
    Note:
    • Only for Gateways running version 6.0 or later have an option to configure up to 4 tunnels based on VPN type. In addition, those tunnels destined to be a Non SD-WAN gateway can operate in either AA or A-HS mode to achieve load sharing/bearing preferences of the user.
    • For gateways running version less than 6.0, all active-active configurations are interpreted as active-hotstandby with tunnel 1 being active and tunnel 2 being hot-standby.

Monitoring BGP Events

You can view the events related to the BGP sessions.

In the SD-WAN service of the Enterprise Portal, select Monitor > Events .

To view the events related to BGP, you can use the filter option. Select Filter next to Search and opt to filter the details by different categories.

Figure 23. Monitoring BGP Events
Review the following BGP events:
  • BGP session established to Gateway neighbor
  • BGP session established to Edge neighbor
  • BGPv6 session established to Edge neighbor
  • Unavailable BGP neighbor
  • Unavailable BGPv6 neighbor
  • Unavailable Gateway BGP neighbor

Troubleshooting BGP Sessions

You can run Remote Diagnostics tests to view the logs of the BGP sessions and use the log information for troubleshooting purposes.

Use the following steps to run tests for troubleshooting BGP:

  1. In the SD-WAN service of the Enterprise Portal, select Diagnostics > Remote Diagnostics .
  2. The Remote Diagnostics page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BGP sessions, scroll to the following sections and run the tests:
    • Troubleshoot BGP- List BGP Redistributed Routes – Run this test to view routes redistributed to BGP neighbors.
    • Troubleshoot BGP- List BGP Routes – Run this test to view the BGP routes from neighbors. You can enter IPv4 or IPv6 prefix to view specific BGP routes or leave the prefix empty to view all the BGP routes.
    • Troubleshoot BGP- List Routes per Prefix – Run this test to view all the Overlay and Underlay routes for a specific IPv4 or IPv6 prefix and the related details
    • Troubleshoot BGP- Show BGP Neighbor Advertised Routes – Run this test to view the BGP routes advertised to a neighbor.
    • Troubleshoot BGP- Show BGP Neighbor Learned Routes – Run this test to view all the accepted BGP routes learned from a neighbor after filters.
    • Troubleshoot BGP- Show BGP Neighbor Received Routes – Run this test to view all the BGP routes learned from a neighbor before filters.
    • Troubleshoot BGP- Show BGP Neighbor details – Run this test to view the details of BGP neighbor.
    • Troubleshoot BGP- Show BGP Routes per Prefix – Run this test to view all the BGP routes and their attributes for the specified prefix.
    • Troubleshoot BGP- Show BGP Summary – Run this test to view the existing BGP neighbor and received routes.
    • Troubleshoot BGP- Show BGP Table – Run this test to view the BGP table.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Advertised Routes – Run this test to view the BGPv6 routes advertised to a neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Learned Routes – Run this test to view all the accepted BGPv6 routes learned from a neighbor after filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor Received Routes – Run this test to view all the BGPv6 routes received from a neighbor before filters.
    • Troubleshoot BGPv6- Show BGPv6 Neighbor details – Run this test to view the details of BGPv6 neighbor.
    • Troubleshoot BGPv6- Show BGPv6 Routes per Prefix – Run this test to view all the BGPv6 routes for the prefix and their attributes.
    • Troubleshoot BGPv6- Show BGPv6 Summary – Run this test to view the existing BGPv6 neighbor and received routes.
    • Troubleshoot BGPv6- Show BGPv6 Table – Run this test to view the details of BGPv6 table.

    For more information about all the supported BGP related Remote Diagnostics tests, see the Remote Diagnostic Tests on Edges section in the VeloCloud SD-WAN Troubleshooting Guide.

OSPF and BGP Redistribution

Each of routing protocols OSPF and BGP may be enabled independently and the prior model of allowing only one routing protocol to be enabled on the system has been removed with this release. This release also allows the possibility of redistributing OSPF into BGP or BGP into OSPF (or both simultaneously), along with other possible route sources like prefixes learnt over the overlay, connected routes, static routes, etc.

In addition, with release 3.2, VeloCloud standardizes the redistribution behavior along more traditional lines (similar to that in other routing vendors). For example, if there is more than one route available for the same prefix, then only the best route for that prefix in the system RIB will be redistributed to the destination protocol if the configuration in the destination protocol allows redistribution for that route type.

Consider, as an example, redistribution of the prefix 192.168.1.0/24 into BGP, and routes to the prefix 192.168.1.0/24 are locally available, learned from OSPF and separately learned as an Overlay prefix. Let's further assume that between the OFC flow ordering for the prefix, and route metrics, and route preference the OSPF route ranks above (is better than) the learned overlay route for that same prefix. Then, the OSPF route will be redistributed into BGP if OSPF redistribution has been turned on in BGP. Note that since the overlay learned prefix is not the best route for that prefix in the system RIB, it will not be redistributed into BGP even if the redistribution of overlay prefixes has been turned on in BGP.

In cases like the above, in order to facilitate the redistribution of the best route for a prefix into a given destination protocol, the user can enable redistribution for the specific route type that is the best route in the system.

Alternately, if the user prefers a different route source for that prefix to be redistributed into the destination protocol, the user can control the relative precedence of the route in the system RIB using the Overlay Flow Control facility provided by the management interface, or by varying the route metric.

Route-Overlap Suppression in Overlay Flow Control

In Overlay Flow Control (OFC), the system employs a route-overlap suppression mechanism to improve network stability and efficiency. This behavior ensures that certain routing prefixes are not advertised into the OFC if they overlap with locally connected interface subnets.

When a received prefix overlaps with a subnet of a locally connected interface, it will not be advertised into the OFC. The system suppresses and intentionally discards both summarized and supernet routes when the user enables this setting. This behavior prevents routing loops, packet misforwarding, and overlay ownership conflicts.

Consider an example configuration scenario in which an Edge LAN interface is configured with the IP address 10.10.10.9/29 and operates within OSPF Area 1. The Edge establishes OSPF neighborship with a LAN core switch, assigned the IP address 10.10.10.10/29. In this scenario, if the Edge receives an OSPF advertisement for the IP subnet 10.10.10.0/24, this subnet will not be added to the OFC table due to route-overlap suppression.

OSPF/BGP Redistribution Metric Calculation

Starting with the 5.2 release, the route redistribution metric calculation has changed. When a route is redistributed from the Overlay to OSPF/BGP, the redistribution metric is calculated by taking the original route metric and adding the transit metric:
  • The transit metric is (0) if the route is learned from a directly connected Edge.
  • The transit metric is (90) if the route is learned via a Gateway.
  • The transit metric is (32 + hub's order value) if the route is learned via a Hub Edge.

For OSPF External Type-1 (OE1) routes, this is the final metric. For OSPF External Type-2 (OE2) routes, it adds up the non-preferred metric constant (8388607). This is why there is a very high metric value for an OE2 route type on Edge peers.

For BGP, this implies that the BGP MED value advertised by Hub Edges, which previously started from 9, 10, 11, and so forth, now starts from 33, 34, 35, and so forth.

BFD Settings Overview

Bidirectional Forwarding Detection (BFD) provides a simple Hello protocol similar to detection components of well-known routing protocols. A pair of systems transmit BFD packets periodically over each path between the two systems, and if a system stops receiving BFD packets for long enough, the neighboring system is assumed to have failed.

A BFD session is established based on the needs of the application that would use BFD. The user has to explicitly configure the address and parameters for the BFD session and the subscribers/applications (BGP/OSPF) of the session, as there is no discovery mechanism in BFD.

Routing protocols like BGP or OSPF exchange the learned routes between Edges and Routers. These protocols exchange routes and detect route failures using their own mechanism. Generally, route failures are detected based on the keepalive mechanism where one entity echoes other entity on a frequent configured interval, that is the keepalive time. These routing protocols have higher keepalive timers which results in longer duration to detect the route failures. BFD detects route failures between two connected entities faster with low overhead on detection of failures.

The following are the advantages of implementing BFD with routing protocols:
  • Fast route failure detection with low re-convergence time.
  • Less overhead in route failure detection.
  • Uniform rate of route failure detection across routing protocols.

BFD can be defined as a simple service. The service primitives provided by BFD are to create, destroy, and modify a session, given the destination address and other parameters. BFD in return provides a signal to the clients indicating when the BFD session goes up or down.

There are two operating modes to BFD, asynchronous mode and demand mode. Arista VeloCloud supports asynchronous mode. In this mode, the systems periodically send BFD control packets to other systems and if several packets in a row are not received by a system, the session is declared to be down.
Note: BFD Echo mode is not supported.
VeloCloud supports BFD for the following routing protocols:
  • BGP on Edges and Partner Gateways
  • OSPF on Edges

Configuring BFD for Profiles

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities.

To configure a BFD session for Profiles, use the following steps:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device Icon for a profile, or select a profile and select the Device tab.
  3. In the Device tab, scroll down to the Routing and NAT section and select the arrow next to the BFD area to open it.
  4. Select the BFD slider to ON position.
  5. Configure the following settings:
    Table 20. Routing and NAT Field Descriptions
    Field Description
    Peer Address Enter the IPv4 address of the remote peer to initiate a BFD session. Local Address Enter a locally configured IPv4 address for the peer listener. This address sends the packets.
    Note: Select the IPv6 tab to configure IPv6 addresses for the remote peer and the peer listener.
    Local Address For IPv6, the local and peer addresses support only the following format:
    • IPv6 global unicast address (2001:CAFE:0:2::1)
    • IPv6 unique local address (FD00::1234:BEFF:ACE:E0A4)
    Multihop Select the check box to enable multi-hop for the BFD session. While BFD on Edge and Gateway supports directly connected BFD Sessions, you need to configure BFD peers in conjunction with multi-hop BGP neighbors. The multi-hop BFD option supports this requirement.

    Multihop must be enabled for the BFD sessions for NSD-BGP-Neighbors.

    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
  6. Select the (+) to add details of more peers.
  7. Select Save Changes.
    Figure 24. Configuring BFD for Profiles

    When you configure BFD rules for a profile, the rules are automatically applied to the Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configuring BFD for Edges for more information.

    VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configuring BFD for Edges

VeloCloud SD-WAN allows to configure BFD sessions to detect route failures between two connected entities. Once you have configured BFD rules for a Profile, the rules are automatically applied to the Edges that are associated with the profile. Optionally, you can override the inherited settings at the Edge level.

Use the following steps to override the configuration for a specific Edge:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select the Device Icon next to an Edge, or select an Edge and select the Device tab.
  3. In the Device tab, scroll down to the BFD Rules.
  4. Select Override to modify the BFD configuration settings for the selected Edge.
    Figure 25. Configuring BFD for Edges
  5. Select Save Changes.
    VeloCloud SD-WAN supports configuring BFD for BGP and OSPF.

Configuring BFD with BGP for Profiles

Configure BFD for BGP on SD-WAN Profiles.

By default, BFD is deactivated in BGP neighbor. You can enable BFD for a BGP session to subscribe to BFD session updates.

Enabling BFD for a BGP neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure describes how to enable BFD for an already configured BGP session on an Edge. To configure BGP settings, see Configure BGP from Edge to Underlay Neighbors for Profiles.

To enable BFD for BGP on partner Gateways, you must be an Operator super user. For more information, see the Configure Partner Handoff section in the VeloCloud SD-WAN Operator Guide.

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select the Device Icon for a profile, or select a profile and select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to the BGP area to open it.
  4. Select the BDF slider to ON position.
  5. In the BGP Editor window, select view all in the Additional Options column for a BGP neighbor and select Enable BFD. You can enable a BFD subscription for multiple BGP neighbors, including NSD Neighbors in the 4.3 release.
    Note:
    Multihop must be configured as Multihop BFD for NSD BGP Neighbors in the 4.3 release. For more information about NSD Neighbors, see Configure BGP Over IPsec from Edge to Non SD-WAN Neighbors
    Figure 26. Configuring BFD for BGP on a Profile
    Note: BGP over IPsec from the SD-WAN Edge does not support a single-hop BFD session.
  6. Configure the other settings as required and select OK.

    When you enable BFD for BGP settings in a profile, the setting automatically applies to the Edges associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for BGP for Edges for more information.

    When a BGP neighbor receives an update that BFD session is down, the corresponding BGP session immediately goes down and the routes learnt through the BGP peer are flushed without waiting for the expiration of a keepalive timer.

Configuring BFD with BGP for Edges

You can override the inherited settings at the Edge level for BFD for BGP.

Use the following steps to override the BFD configuration for an Edge:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select the Device Icon next to an Edge, or select an Edge and select the Device tab.
  3. In the Device tab, scroll down to the Routing & NAT section and select the arrow next to the BGP area to open it.
  4. Select Override and move the slider to the ON position to modify the BGP settings for the selected Edge.
    Figure 27. Configuring BFD with BGP for Edges

Configuring BFD with OSPF for Profiles

You can configure BFD for OSPF for Profiles.

By default, BFD is deactivated in OSPF. You can enable BFD for OSPF to subscribe to BFD session updates.

Enabling BFD for an OSPF neighbor does not create a BFD session. You must explicitly configure a BFD session. See Configure BFD for Profiles.

The following procedure describes how to enable BFD for an already configured OSPF session on an Edge Interface. To configure OSPF settings, see Activate OSPF for Profiles.

To configure the Interface settings, see Configure Interface Settings for Profiles.

  1. In the SD-WAN service of the Enterprise portal, select Configure > Profiles .
  2. Select a profile and select View from the Device column of the profile. The Device page for the selected profile appears.
  3. Navigate to Connectivity and select Interfaces to display the Edge models available for the selected Profile.
  4. In the Interfaces section, click an Edge model to view the interfaces available in the Edge and select an interface to edit the settings.
  5. On the Interface edit window, you can configure OSPF settings under IPv4/IPv6 Settings. Select OSPF and then select the OSPF Area from the list.
    Figure 28. Configuring BFD with OSPF for a Profile
  6. Expand Advanced Settings and in the Custom Settings, select Enable BFD.
  7. Configure the other settings as required and select Save.

    When you enable BFD for an OSPF area in a profile, the setting is automatically applied to the corresponding Edges that are associated with the profile. If required, you can override the configuration for a specific Edge. See Configure BFD for OSPF for Edges for more information.

    When an OSPF neighbor receives an update that BFD session is down, the corresponding OSPF session immediately goes down and the routes are flushed without waiting for the expiration of keepalive timer.

Configuring BFD with OSPF for Edges

You can modify the inherited Profile settings at the Edge level for BFD for OSPF.

If required, you can override the configuration for a specific Edge as follows:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Edges .
  2. Select an Edge you want to configure BFD for OSPF settings and click the View link in the Device column of the Edge. The Device page for the selected Edge appears.
  3. Navigate to Connectivity and select Interfaces.
  4. In the Interfaces section, click an interface to edit the settings.
  5. On the Interface edit window, you can configure OSPF settings under IPv4/IPv6 Settings. Select OSPF and then select the OSPF Area from the list.
    Figure 29. Configuring BFD with OSPF on an Edge
  6. Expand Advanced Settings and in the Custom Settings tab, select Enable BFD.
  7. Configure the required settings for the Edge as required and select Save.

Configuring BFD for Gateways

You can configure BFD Settings for Gateways over IPsec tunnels.

Use the following steps to configure BFD for a Gateway:

  1. In the SD-WAN service of the Enterprise portal, select Configure > Network Services .
  2. In the Non SD-WAN Destinations via Gateway area, select Edit in the BFD column that corresponds to the Non SD-WAN Destination.
    Figure 30. Configuring BFD for Gateways
  3. In the BFD Editor window, move the BFD Activated slider to the right to turn it on to configure the BFD settings for the Primary and Secondary Gateways.
  4. Configure the BFD settings:
    Note: The Secondary Gateway option is available only if you have configured a secondary Gateway for the corresponding Non SD-WAN Destination.
    Figure 31. Editing BFD Settings
    Table 21. BFD Settings Option Descriptions
    Field Description
    Peer Address Enter the IP address of the remote peer to initiate a BFD session.
    Local Address Enter a locally configured IP address for the peer listener. This address is used to send the packets.
    Multihop This option is not supported for the Gateways.
    Detect Multiplier Enter the detection time multiplier. The remote transmission interval is multiplied by this value to determine the detection timer for connection loss. The range is from 3 to 50 and the default value is 3.
    Receive Interval Enter the minimum time interval, in milliseconds, at which the system can receive the control packets from the BFD peer. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Transmit Interval Enter the minimum time interval, in milliseconds, at which the local system can send the BFD control packets. The range is from 300 to 60000 milliseconds and the default value is 300 milliseconds.
    Note: BFD is supported only on VTP Tunnels.

Monitoring BFD Sessions

Monitor the BFD Sessions on Edges and Gateways. To view the BFD Sessions, use the following steps:

  1. In the SD-WAN service of the Enterprise portal, select Monitor > Routing .
  2. In the Routing screen, select the BFD tab.
  3. Click on Filter next to Search and opt to filter the details by different categories.

    The Edge BFD Sessions table displays the BFD Sessions of the Edge and the Gateway.

    Figure 32. Monitoring Edge and Gateway BFD Sessions
    The BFD Sessions include the following details for Edges and Gateways:
    • Name of the Edge or Gateway
    • Segment name
    • Peer IPv4 or IPv6 address
    • Local IPv4 or IPv6 address
    • State of the BFD session
    • Remote and Local timers
    • Number of Events
    • Duration of the BFD session
    Click an Event number to view the details of the events.

Monitoring BFD Events

View the events related to the BFD sessions.

In the SD-WAN service of the Enterprise portal, click Monitor > Events .

To view the events related to BFD, use the Filter option. Click Filter next to the Search option and choose to filter the details by different categories.

Figure 33. Monitoring BFD Events

The following events relate to BFD sessions:

  • BFD session established to Gateway neighbor
  • BFD session established to Edge neighbor
  • BFDv6 session established to Edge neighbor
  • Edge BFD Configuration
  • Edge BFD IPv6 Configuration
  • Edge BFD neighbor unavailable
  • Edge BFDv6 neighbor unavailable
  • Gateway BFD neighbor unavailable

Troubleshooting BFD

Run Remote Diagnostics tests to view the logs of the BFD sessions and use the log information for troubleshooting purposes.

Use the following steps to run tests for troubleshooting BFD:

  1. In the SD-WAN service of the Enterprise Portal, select Diagnostics > Remote Diagnostics .
  2. The Remote Diagnostics page displays all the active Edges.
  3. Select the Edge that you want to troubleshoot. The Edge enters live mode and displays all the possible Remote Diagnostics tests than you can run on the Edge.
  4. For troubleshooting BFD sessions, scroll to the following sections and run the tests:
    • Troubleshoot BFD- Show BFD Peer Status – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. select Run to view the details of the BFD peers.
    • Troubleshoot BFD- Show BFD Peer counters – Choose the Segment from the drop-down list. Enter the Peer and Local IP addresses of an already configured BFD session. Select Run to view the details of counters of the BFD peers.
    • Troubleshoot BFD- Show BFD Setting – Select Run to view the details of BFDv4 settings and status of neighbors.
    • Troubleshoot BFD6- Show BFD6 Setting – Select Run to view the details of BFDv6 settings and status of neighbors.

    For more information about all the supported BFD-related Remote Diagnostics tests, see the Remote Diagnostic Tests on Edges section in the VeloCloud SD-WAN Troubleshooting Guide.

Overlay Flow Control

The Overlay Flow Control page displays a summarized view of all the routes in your network.

For the 4.3 release, a new NSD bucket has been introduced for the classification of NSD Routes. The new NSD bucket preference logic applies only when the Use NSD policy is enabled along with the Distributed Cost Calculation. The Use NSD policy can only be enabled after you enable the Distributed Cost Calculation.

You can view and edit the global routing preferences and the advertise actions for the Edges, Hubs, Partner Gateways, and Non SD-WAN Destinations via Edge and Gateway.

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
    Figure 34. Configuring Overlay Flow Control
    The Overlay Flow Control table displays the following details:
    Table 22. Overlay Flow Control Option Descriptions
    Option Description
    Preferred VPN Exits Displays the priority of the destinations to where the traffic should be routed.
    Global Advertise Flags Displays the advertise actions of static, connected, internal, external, and uplink routes.
    Routes List Displays all routes. You can change the Preferred VPN Exits order for a particular subnet by selecting Edge Subnet in the Overlay Flow Control page.

     

  2. In the Overlay Flow Control page, you can configure the following settings:
    • Edit – Select to update the priorities and the advertise actions. See Configure Global Routing Preferences.
    • Refresh Routes – This option is available only when the Distributed Cost Calculation feature is enabled by the Operator. By default, the Orchestrator is actively involved in learning the dynamic routes. Edges and Gateways rely on the Orchestrator to calculate initial route preferences and return them to the Edge and Gateway. The Distributed Cost Calculation feature enables to distribute the route cost calculation to the Edges and Gateways. For IPv4, this option is available only when the Distributed Cost Calculation feature is enabled by Operator. For IPv6, Distributed Cost Calculation is enabled by default. The Operator cannot turn off this feature for IPv6. For more information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the VeloCloud SD-WAN Operator Guide.
      Note: To enable the Distributed Cost Calculation feature, check with your supporting partner. If you are directly supported by Arista Networks, contact the support team.
    • Type YES and then select Refresh Routes to make the Edges and Gateways recalculate learned route costs and send them to the Orchestrator. In addition, the changes in the Overlay Flow Control are applied immediately on the new and existing learned routes.
      When you refresh the routes, the Customer Enterprise has the following impact on the network:
      • All the local dynamic routes are refreshed, and the preference and advertise action of these routes are updated. This updated information is advertised to the Gateway, Orchestrator, and eventually across the Enterprise. As this leads to an update in the routing table, there is a brief impact on the traffic for all the sites.
      • Any existing flow using these routes can potentially be affected due to the change in the routing entries.
        Note: It is recommended to use Refresh Routes in a maintenance window to minimize the impact on the Customer Enterprise.
    • VRF Global Routing Preferences – This option enables you to edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.
    Figure 35. Configuring VRF Global Routing Preferences
  3. Select Preferred VPN Exits to prioritize the VPN Exits, and then select Edit to use the UP and DOWN arrows to modify the priorities.
    Figure 36. Editing the Preferred VPN
  4. In the Global Advertise Flags section, select the relevant check boxes to modify the advertise actions for the routes.
    • Routes List – This section displays the learned routes in the subnets. Select the IPv4 or IPv6 tab to view the corresponding subnets. The following image shows IPv6 subnets. For more information, see Configure Subnets.
    Figure 37. Configuring Route Lists

    The bottom panel of the Overlay Flow Control window displays the subnets. You can prioritize the preferred destinations for the subnets and pin or unpin learned route preferences. For more information, see Configure Subnets.

Configuring Global Routing Preferences

In the Overlay Flow Control window, you can edit the global routing preferences, advertise actions, and modify the priorities of the destinations to where the traffic should be routed.

The VRF Global Routing Preferences section displays the Preferred VPN Exits and the Global Advertise Flags areas.

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. In the Overlay Flow Control page, select Preferred VPN Exits and then select the Edit link to open the Edit Preferred VPN screen.
    Figure 38. Editing the Preferred VPN
  3. You can update the Preferred VPN Exits area and select the UP and DOWN arrows to modify the priorities.
  4. In the Overlay Flow Control page, select Global Advertise Flags to open the Edit Preferred VPN screen.
    Figure 39. Configuring Global Advertise Flags
    1. In the Global Advertise Flags area, select the relevant check boxes to modify the advertise actions for the routes.
    2. Select Update to save the changes.

    The updated settings are displayed in the Overlay Flow Control page.

Configuring Subnets

In the Overlay Flow Control window, you can update the priorities of the destinations for the learned routes in the subnets.

  1. In the SD-WAN Service of the Enterprise portal, select Configure > Overlay Flow Control .
  2. The Routes List section of the Overlay Flow Control window displays the subnets with the following details:
    Figure 40. Displaying the Route List
    Table 23. Router List Option Descriptions
    Option Description
    Segment Segment name
    Subnet The network that the route corresponds to along with a list of Edges that learned the route.
    Preferred VPN Exits The route through which another branch can access the subnet.
    Route Type Displays the type of the route, which can be one of the following: Static, Connected, or Learned.
    Last Update The last updated date and time of the preferred VPN exit.
    Created On Date and time when the route was created.
    IPv4 Subnet  
    Eligible VPN Exits  
    Note: You can configure the subnets for both IPv4 and IPv6 addresses.
    Currently, you can view up to 256 subnet prefixes in the API call request. You can use the Filter option to search for the specific subnet prefix. The following information message appears if the results are more than the server could return.
    There are more results that the server could return. Please narrow your search criteria.
    Select one or more subnets and select MORE to perform the following activities:
    • Pin Learned Route Preference – Pins the preferences of the selected learned route.
    • Unpin Learned Route Preference – Unpins the preference of the selected learned route to default settings.
    • Delete Learned Routes – Deletes the learned routes. This option does not delete the connected routes, static routes, routes from Overlay Flow Control, and routes from Edge Route table. The option is available only when Configure Distributed Cost Calculation is turned off.
  3. Select the Edit Subnet option for a subnet to modify the priorities of the preferred destination.
    1. In the Subnet window, you can move the destinations from the Eligible VPN Exits to Preferred VPN Exits and vice versa.
      Figure 41. Configuring Preferred VPN Exits
    2. In the Preferred VPN Exits panel, select the UP and DOWN arrows to change the priorities and select Update.
    3. You can reset the cost calculation for the subnets when there are pinned routes available. Select Reset, which enables the Orchestrator to clear the pinned routes, recalculate the cost for the selected subnet based on the policy, and send the results to the Edges and Gateways.
      Note: For IPv4 Routes, the Reset option is available only when Distributed Cost Calculation is enabled.
      Note: The Reset option is available only when Distributed Cost Calculation is enabled.

      For more information on Distributed Cost Calculation, refer to the Configure Distributed Cost Calculation section in the VeloCloud SD-WAN Operator Guide.

..