Print

Azure Virtual WAN IPsec Tunnel Automation

Arista Edge Cloud Orchestrator supports integration and automation of Azure Virtual WAN from VeloCloud Gateway and VeloCloud Edge to enable Branch-to-Azure VPN Connectivity.

Azure Virtual WAN IPsec Tunnel Automation Overview

Azure Virtual WAN is a network service that facilitates optimized and automated Virtual Private Network (VPN) connectivity from enterprise branch locations to or through Microsoft Azure. Azure subscribers provision Virtual Hubs corresponding to Azure regions and connect branches (which may or may not be SD-WAN enabled) through IP Security (IPsec) VPN connections.

To establish branch-to-Azure VPN connectivity, Orchestrator supports Azure Virtual WAN and VeloCloud SD-WAN integration and automation by leveraging the Azure backbone. Currently, the following Azure deployment options support the VeloCloud SD-WAN:
  • IPsec from Gateway to Azure virtual WAN hub with automation.
  • Direct IPsec from Edge to Azure virtual WAN hub with automation.

Azure Virtual WAN Gateway automation

The following diagram illustrates the IPsec tunnel from Gateway to Azure virtual WAN hub.

Figure 1. IPsec tunnel from Gateway

Azure Virtual WAN Edge automation

The following diagram illustrates the IPsec tunnel directly from Edge to Azure virtual WAN hub.

Figure 2. IPsec tunnel Directly from Edge

Prerequisite Azure Configuration

Enterprise network administrators must complete the following prerequisite configuration tasks at the Azure portal to ensure that the Orchestrator application can function as the Service Principal (identity for the application) for Azure Virtual WAN and Gateway integration.

Register Orchestrator Application

It describes registering a new Azure Active Directory (AD) application.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.

To register a new application in Azure AD:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Click All Services and search for Azure Active Directory.
  3. Select Azure Active Directory and go to App registrations > New registration . The Register an application screen appears.
    Figure 3. Register an application
  4. Enter the name for your Orchestrator application in the Name field.
  5. Select a supported account type, which determines who can use the application.
  6. Click Register.

    Your Orchestrator application is registered, and it displays in the All applications and Owned applications tabs.

    Note that the Directory (tenant) ID and Application (client) ID are used during the Orchestrator configuration for Cloud Subscription.

Assign the Orchestrator Application to the Contributor Role

You must assign the application to a role to access resources in your Azure subscription. You can set the scope at the subscription, resource group, or resource level. Lower levels of scope inherit permissions.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.

To assign a Contributor role at the subscription scope:

  1. Select All Services and search for Subscriptions.

    From the list of subscriptions, select the subscription to which you want to assign your application. Select the global subscriptions filter if you do not see the subscription you are looking for. Make sure the subscription you want is available for the portal.

  2. Select Access control (IAM).
  3. Select +Add > Add Role Assignment . The box appears.
    Figure 4. Access Control
  4. Select the Contributor role from the Role drop-down menu to assign to the application. To allow the application to execute actions like reboot, start, and stop instances, the system recommends that users assign the Contributor role to the App Registration.
  5. Select Azure AD user, group, or service principal from the Assign access to drop-down menu. By default, Azure AD applications are not in the available display options. To find your application, search for the name and select it.
  6. Select Save.

    Now, the application is assigned to the Contributor role and it appears in the list of users assigned to a role for that scope.

Register a Resource Provider

To download Virtual WAN Virtual Private Network (VPN) configurations, the Orchestrator requires a Blob Storage Account that acts as an intermediary data store from where you can download the configurations. The Orchestrator aims to create a seamless user experience by providing a transient storage account for each of the download task. To download VPN site configurations, you must manually register the Microsoft.Storage resource provider on your Azure Subscription. By default, the Microsoft.Storage resource provider is not registered on Azure Subscriptions.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.
  • You have the Contributor or Owner roles permission.

To register a resource provider for your subscription:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Services and search for Subscriptions.
    From the list of subscriptions, select your subscription.
  3. Go to Settings > Resource Providers .
    Figure 5. Resource Providers
  4. Select Microsoft.Storage from the available resource providers and select Register.

    The resource provider is registered and configures your subscription to work with the resource provider.

Create a Client Secret

It creates a new client secret in Azure AD for authentication.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.

To create a new client secret in Azure AD:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Click Azure Active Directory > App Registration .
  3. Clickon your registered Orchestrator application on the Owned Application tab.
  4. Select Certificates & Secrets > New Client Secret . The Add a Client Secret screen appears.
    Figure 6. Certificates & Secrets
  5. Provide details such as a description and expiration value for the secret,and click Add. It creates a client secret from the registered application.
    Note: Copy and save the new client secret value used during the Cloud Subscription in Orchestrator.

Configure Azure Virtual WAN for Branch-to-Azure VPN Connectivity

This section describes the procedures to configure Azure for integrating Azure Virtual WAN and Gateway to enable the branch-to-Azure VPN connectivity. Before you begin to configure the Azure Virtual WAN and the other Azure resources:
  • Verify that none of the subnets of your on-premises network overlap with the existing virtual networks you want to connect to. Your virtual network does not require a gateway subnet and cannot have any virtual network gateways. For steps to create a virtual network, see Create a Virtual Network.
  • Obtain an IP address range for your Hub region and ensure that the address range that you specify for the Hub region does not overlap with any of your existing virtual networks that you connect to.
  • Ensure you have an Azure subscription. If not, create a free account.
For step-by-step instructions about the various procedures in the Azure portal side for integrating Azure Virtual WAN and Gateway, see:

Create a Resource Group

It creates a resource group in Azure.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.

To create a resource group in Azure:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Services and search for Resource Groups.
  3. Select Resource Groups from the available resource groups and select +Add. The Create a Resource Group screen appears.
    Figure 7. Resource Group
  4. From the Subscription drop-down menu, select your Microsoft Azure Subscription.
  5. In the Resource group box, enter a unique name for your new Resource group.
    A resource group name can include alphanumeric characters, periods (.), underscores (_), hyphens (-), and parentheses (), but the name cannot end with a period.
  6. From the Region drop-down menu, select the location for your resource group, where most of your resources will reside.
  7. Select Review+create and then select Create.

    It creates resource group and appears on the Azure portal dashboard.

Create a Virtual WAN

It creates a Virtual WAN in Azure.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.
  • You have a resource group to add the Virtual WAN.

To create a Virtual WAN in Azure:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Services and search for Virtual WAN.
  3. Select Virtual WAN from the available resource groups and select +Add. The Create WAN screen appears.
    Figure 8. Create WAN
  4. From the Subscription drop-down menu, select your Microsoft Azure Subscription.
  5. In the Resource group tab, select your resource group to add the Virtual WAN.
  6. From the Resource group location drop-down menu, select the location where the metadata associated with the Virtual WAN resides.
  7. In the Name tab, enter a unique name for your Virtual WAN.
  8. From the Type drop-down menu, select Standard as the Virtual WAN.
  9. Select Create.

    It creates a Virtual WAN and appears on the Azure portal dashboard.

Create a Virtual Hub

It creates a Virtual Hub in Azure.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.
  • You have a resource group to add the Virtual WAN.

To create a Virtual WAN in Azure:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Resources and select the Virtual WAN you created earlier from the list of available resources.
  3. Under the Virtual WAN Architecture from the available resource groups and select Hubs.
  4. Select +New Hubs. The Create Virtual Hubs screen appears.
    Figure 9. Create Virtual Hubs > Basics
  5. In the Basics tab, enter the following Virtual Hub details.
    1. From the Region drop-down menu, select the location where the Virtual Hub resides.
    2. In the Name tab, enter the unique name for your Hub.
    3. In the Hub private address space tab, enter the address range for the Hub in Classless Inter-Domain Routing (CIDR) notation.
  6. Select Site to site and enable Site to site (VPN gateway) before connecting to VPN sites by selecting Yes.
    Note: A VPN Gateway is required for tunnel automation to work, otherwise it is not possible to create VPN connections.
    Figure 10. Virtual Hub > Site to site
    • From the Gateway scale units drop-down menu, select a scaling value.
  7. Select Review + Create.

    It creates a Virtual Hub and appears on the Azure portal dashboard.

    Next steps:

Create a Virtual Network

It creates a Virtual Network in Azure.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.
  • You have a resource group to add the Virtual WAN.

To create a Virtual Network in Azure:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Services and search for Virtual Network.
  3. Select Virtual Network from the available resource groups and select +Add. The Create Virtual Network screen appears.
    Figure 11. Create Virtual Network
  4. In the Name tab, enter a unique name for your Virtual Network.
  5. In the Address space text box, enter the address range for the Virtual Network in Classless Inter-Domain Routing (CIDR) notation.
  6. From the Subscription drop-down menu, select your Microsoft Azure Subscription.
  7. In the Resource group tab, select your resource group to add the Virtual Network.
  8. From the Resource group location drop-down menu, select the location where the metadata associated with the Virtual Network resides.
  9. Under the Subnet area, enter the name and address range for the subnet.
  10. Select Create.

    It creates a Virtual Network and appears on the Azure portal dashboard.

    Next steps:

Create a Virtual Connection Between VNet and Hub

It creates a virtual connection between Virtual Networks (VNets) and the Virtual Hub in a particular Azure region.

Before you begin:
  • Make sure you have an Azure subscription. If not, create a free account.
  • You have Virtual Hubs and Virtual Networks created.

To create a virtual network connection between a VNet and a Virtual Hub in a particular Azure region:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Go to All Resources and select the Virtual WAN you created earlier from the list of available resources.
  3. Under the Virtual WAN architecture area, select Virtual network connections.
  4. Select +Add connection. The Add connection screen appears.
    Figure 12. Virtual Network Connection
  5. In the Connection name tab, enter the unique name for the virtual connection.
  6. Select the Hub you want to associate with this connection from the Hub drop-down menu.
  7. Select your Microsoft Azure subscription from the Subscription drop-down menu.
  8. From the Virtual Network Virtual network drop-down menu, select the Virtual Network you want to connect to this Hub.
  9. Select OK.

    It establishes a peering connection between the selected VNet and the Hub.

Configure Orchestrator for Azure Virtual WAN IPsec Automation from the Gateway

You can configure Orchestrator to integrate Azure Virtual WAN and Gateway, enabling the branch-to-Azure VPN connectivity.

Note: By default, the Azure Virtual WAN feature is deactivated. To enable the feature, an Operator Super user must set the session.options.enableAzureVirtualWAN system property to true.
Note: The Non SD-WAN Destination (NSD) tunnel only supports static routes when using the Azure Virtual WAN Automation from Gateway feature. As a result, it is not currently compatible with BGP over IPsec.
Before you begin the Orchestrator configuration for Azure Virtual WAN - Gateway automation, make sure you complete all the steps explained in the Prerequisite Azure Configuration and Configure Azure Virtual WAN for Branch-to-Azure VPN Connectivity sections. For step-by-step instructions about the various procedures that you need to complete in the Orchestrator for integrating Azure Virtual WAN and Gateway, see: To view the details of Non SD-WAN Destinations network services configured for an enterprise, see Monitor Non-SD-WAN Destinations.

Associate a Microsoft Azure with an SD-WAN Profile

After configuring a Non SD-WAN Destination of Microsoft Azure Virtual Hub in Orchestrator, you must associate the non-SD-WAN Destination with the desired Profile to establish the tunnels between Gateways and Microsoft Azure Virtual Hub.

To associate a Non-SD-WAN Destination to a Profile, perform the following steps:
  1. In the SD-WAN service of the Enterprise portal, go to Configure > Profiles . The Profiles page appears.
  2. Select a profile with which you want to associate your Microsoft Azure Non-SD-WAN Destination, and then select the View link in the Device column.
  3. In the Device settings page, under VPN services, activate Cloud VPN by turning on the toggle button.
    Figure 13. Branch-to-Branch VPN

     

  4. Under Edge to Non SD-WAN Sites, select the Enable Edge to Non SD-WAN via Gateway check box.
  5. Select your Non-SD-WAN Destination from the Microsoft Azure Virtual Hub drop-down menu to establish VPN connection between the branch and the Microsoft Azure Non-SD-WAN Destination.
  6. Select Save Changes.

It establishes a tunnel between the branch and the Microsoft Azure Non-SD-WAN Destination.

Edit a VPN Site

This task manually adds the SD-WAN routes into the Azure network.
Before you begin:
  • You must complete provisioning the Azure VPN sites on the Orchestrator side.

To add SD-WAN routes to the Azure network:

  1. Log in to your Microsoft Azure account. The Microsoft Azure home screen appears.
  2. Select All Resources and select the Virtual WAN you created earlier from the list of available resources.
  3. Under the Virtual WAN Architecture area, select VPN sites.
  4. From the available list of VPN sites, select your VPN site (for example, Non SD-WAN Destination name.primary). It is added due to the non-SD-WAN destination provisioning step being done using the Orchestrator.
  5. Select on the name of the selected VPN site, and from the top of the next screen, select Edit site.
    Figure 14. Edit Virtual Wan- VPN Sites
  6. In the Private address tab, enter the address range for the SD-WAN routes.
  7. Select Confirm. Similarly, following the earlier steps, you can edit your Redundant VPN site.
    Note: Currently, Azure vWAN supports only Active/Active tunnel mode, and it does not have the provision to specify priority or primary tunnel to the VPN site (Primary and Redundant sites), and therefore load balancing will be done by Azure on equal cost multi-path routing. It may cause asymmetric traffic flow and increase the latency for those flows. The workaround to avoid the asymmetric flow is to remove the Gateway redundancy on the Azure vWAN Hub NVS tunnel; however, removing redundant Gateway tunnel may not be acceptable for all deployments and needs to handle with caution.

Configure a Non SD-WAN Destination of Type Microsoft Azure Virtual Hub

 

Perform the following steps to configure a Non SD-WAN Destination of type Microsoft Azure Virtual Hub in the Orchestrator.

  1. Configure a Cloud subscription.
    For more information, see Configure API Credentials.
  2. Ensure to create Virtual WAN and Hubs in Azure. For steps, see Configure Azure Virtual WAN for Branch-to-Azure VPN Connectivity.
  3. In the SD-WAN service of the Enterprise portal, navigate to Configure > Network Services, and then under Non SD-WAN Destinations, expand Non SD-WAN Destinations via Gateway.
  4. Click New, and then enter the Name andchoose the Type of the Non SD-WAN Destination.
    Once you enter the Type as Microsoft Azure Virtual Hub,a new Virtual Hub Configuration section is displayed.
    Figure 15. Non SD-WAN Destinations via Gateway
  5. Configure the following settings:
 
Option Description
Name Edit the previously entered name for the Non SD-WAN Destination.
Type Displays the type as Microsoft Azure Virtual Hub. You cannot edit this option.
Tunnel ModeActive/Active mode supports to set up a maximum of 4 tunnel endpoints or Gateways. All Active tunnels can send and receive traffic through ECMP. Active/Hot-Standby mode supports to set up a maximum of 2 tunnel endpoints or Gateways.
Active/Active mode supports to set up a maximum of 4 tunnel endpoints or Gateways. All Active tunnels can send and receive traffic through ECMP.
ECMP Load Sharing Method Flow Load Based (Default) Flow load based algorithm maps the new flow to the path with least number of flows mapped among the available paths to the destination.
Hash Load Based algorithm takes input parameters from 5-tuple (SrcIP, DestIP, SrcPort, DestPort, Protocol). These inputs can be any or all or any subset of this tuple based on user configuration. Flow is mapped to the path based on hash value with selected inputs.
Subscription Select a subscription from the drop-down menu.
Virtual WAN The application fetches all the available Virtual WANs dynamically from Azure. Select a virtual WAN from the drop-down menu.
Resource Group The application auto-populates the resource group to which the selected Virtual WAN is associated.
Virtual Hub Select a virtual Hub from the drop-down menu.
Azure Region The application auto-populates the Azure region corresponding to the selected Virtual Hub.
Enable Tunnel(s) Select the Enable Tunnel(s) check box to allow VPN Gateways to initiate VPN connections to the target Virtual Hub as soon as the site is successfully provisioned.
Note:
  • VPN Gateways initiate the IKE negotiation only when the Non SD-WAN Destination is configured on at least one profile.
  • For Microsoft Azure Non SD-WAN Destination, the default local authentication ID value used is Gateway Interface Public IP.
  1. Click Create.
    The Orchestrator automatically initiates deployment, provisions Azure VPN Sites, and downloads the VPN Site Configuration for the newly configured sites. It stores the configuration in the Orchestrator’s Non SD-WAN Destination configuration database.
    Figure 16. New Non SD-WAN destination via Gateway
    Once the Azure VPN sites are provisioned at the Orchestrator side, you can view the VPN sites (Primary and Redundant) in the Azure portal by navigating to Virtual WAN > Virtual WAN architecture > VPN sites.
  2. Perform the following steps to complete the other configurations:
    • Associate the Microsoft Azure Non SD-WAN Destination to a Profile to establish a tunnel between a branch and Azure Virtual Hub. For more information, see Associate a Microsoft Azure with an SD-WAN Profile.
    • You must add SD-WAN routes into Azure network manually. For more information, see Edit a VPN Site.
    • After associating a Profile to the Microsoft Azure Non SD-WAN Destination, you can return to the Non SD-WAN Destinations via Gateway section by navigating to Configure Network Services, and then configure the BGP settings for the Non SD-WAN Destination. Scroll to the name of your Non SD-WAN Destination, and then click the Edit link in the BGP column. For more information, see Configuring BGP Over IPsec from Gateways.
    • In the Non SD-WAN Destinations via Gateway area, click the Edit link in the BFD column for a Non SD-WAN Destination, to configure the BFD settings. For more information, see Configuring BFD for Gateways.
    For information about Azure Virtual WAN Gateway Automation, see Configure Orchestrator for Azure Virtual WAN IPsec Automation from the Gateway.

Synchronize VPN Configuration

After successful Non-SD-WAN Destination provisioning, whenever there are changes in the endpoint IP address of the Azure Hub or static routes, you need to resynchronize the Azure Virtual Hub and Non-SD-WAN Destination configurations. Clicking the Resync configuration button in the Non-VeloCloud Sites will automatically fetch the VPN configuration details from the Azure portal and will update the Orchestrator local configuration.

Configure Orchestrator for Azure Virtual WAN IPsec Automation from Edge

You can configure Orchestrator to integrate Azure Virtual WAN and Edge to enable the branch-to-Azure VPN connectivity directly from Edge.

Note: When using the Azure Virtual WAN Automation from Edge feature, the Non-SD-WAN Destination (NSD) tunnel only supports static routes. As a result, this feature is not currently compatible with BGP over IPsec.
Before you begin the Orchestrator configuration for Azure Virtual WAN - Edge automation, make sure you complete all the steps explained in the Prerequisite Azure Configuration and Configure Azure Virtual WAN for Branch-to-Azure VPN Connectivity sections. For step-by-step instructions about the various procedures you need to complete in the Orchestrator side for integrating Azure Virtual WAN and Edge, see:

Associate a Microsoft Azure with an SD-WAN Edge and Add Tunnels

After configuring a Non-SD-WAN Destination Microsoft Azure Virtual Hub from Edge, you must associate the Non-SD-WAN Destination with an Edge and configure tunnels to establish IPsec tunnels between the Edge and Microsoft Azure Virtual Hub. At the Edge level, to associate a Non-SD-WAN Destination to an Edge, perform the following steps:
  1. In the SD-WAN service of the Enterprise portal, go to Non-SD-WAN Destinations via Edge.
  2. Select the Edge with which you want to associate your Microsoft Azure Non-SD-WAN Destination, and then click the View link in the Device column.
  3. In the Device settings page, expand Non-SD-WAN Destinations via Edge under VPN services, and then select the Override tab.
  4. Select the Non-SD-WAN Destinations via Edge tab.
    Figure 17. Non-SD-WAN Destination via Edge
  5. Select your Microsoft Azure Virtual Hub service from the Name drop-down menu to establish VPN connection between the branch and the Microsoft Azure Non-SD-WAN Destination.
  6. To configure Edge tunnels, click the + link under Action. The Add Tunnel dialog appears.
    Figure 18. Adding Tunnel
    1. From the Public WAN Link drop-down menu, select a WAN link to establish an IPsec tunnel and click Save. For the WAN links to appear in the drop-down menu, you need first to configure the WAN links for the Edges from the Configure > Edges > Device > WAN Settings page, and wait for the Edge’s WAN links to come up with the valid public IPs. The link’s public IP is the Local Identification value of the tunnel. You can select only the WAN link with a Public IP address.
    2. Azure APIs automatically establish a tunnel between the Edge and the Microsoft Azure Non SD-WAN Destination. After that, the Orchestrator sends the tunnel configuration to the Edge to establish tunnel to the Azure service. Note that the automation for each tunnel takes about 1 to 5 minutes to complete. After the tunnel automation is complete, you can view the details of the configured tunnel and the Public WAN link.
    3. After creating tunnels, you can perform the following actions at the Edge level:
      • Update a tunnel: When the Edge Public WAN link IP address of the tunnel changes, the Orchestrator automatically enqueues an automation job to update the Azure VPN site link and the VPN tunnel configurations. Under Action, click the + link to view the tunnel settings such as PSK.
      • Delete a network service: Select a network service and click Delete.
      • Deactivate a network service: Under the Enable Service column, deselect the check box to deactivate a specific network service.
  7. Click Save Changes.

You can monitor the automated deployment status of the Microsoft Azure Non SD-WAN Destinations configured for an Enterprise from the Monitor > Network Services > Non-SD-WAN Destinations via Edge page in the SD-WAN service of the Enterprise portal. See Monitor Non-SD-WAN Destinations.

Monitor Non-SD-WAN Destinations

You can view the details of Non-SD-WAN Destinations configured for the Enterprise from the Monitor Network Services page in the SD-WAN service of the Enterprise portal. In the Network Services page, you can view:
  • Non-SD-WAN Destinations via Gateway - Displays the configured Non-SD-WAN Destinations along with the other configuration details such as Name of the Non-SD-WAN Destination, Public IP Address, Status of the Non-SD-WAN Destination, Status of the tunnel, Number of profiles and Edges that use the Non-SD-WAN Destination, Last contacted date and time, and Number of related state change Events.
  • Non-SD-WAN Destinations via Edge - Displays the configured Non-SD-WAN Destinations along with the other configuration details such as Name of the Non-SD-WAN Destination, Public IP Address, Status of the tunnel, Number of profiles and Edges that use the Non-SD-WAN Destination, Last contacted date and time, and Deployment status.
Note: Tunnel deployment status monitoring is only supported for Non-SD-WAN Destinations via Edge network service.
To monitor the automation deployment status of Microsoft Azure Non-SD-WAN Destinations via Edge:
  1. In the SD-WAN service of the Enterprise portal, click Monitor > Network Services . The Network Services page appears.
  2. Under Non-SD-WAN Destinations via Edge, click the link in the Deployment Status column to view the deployment status of the Non-SD-WAN Destinations.
    Figure 19. Monitor Network Services
    The following are the seven different states for an Edge action:
    • Enqueued: The Edge action is enqueued.
    • Pending: When the Edge action waits for a backend worker process to pick it up and start working on it.
    • Notified: The Edge action is in this state after a backend worker process picks up the Edge action and starts working on it.
    • Completed: The Edge action is in this state if the Edge action task is successfully completed.
    • Errored: The Edge action is in this state if an error has occurred.
    • Timed Out: The Edge action is in this state if it takes more than the expected amount of time to complete the Edge action task.
    • Pending Delete: The Edge action is in this state if it is pending deletion.
..