VeloCloud SD-WAN in Azure Virtual WAN Hub Deployment

The VeloCloud SD-WAN in Azure Virtual WAN (vWAN) Hub deployment describes the configurations that are required to manually deploy a Virtual Edge as a Network Virtual Appliance (NVA) in Azure vWAN Hub network.

About VeloCloud SD-WAN in Azure Virtual WAN Hub Deployment

Overview

During cloud migration, there were lot of challenges on how to connect remote locations to Azure VNets in a simple, optimized, and secure way across myriad connectivity options. VeloCloud SD-WAN addresses these problems by leveraging Dynamic Multipath Optimization ™ (DMPO) technologies and distributed cloud gateway coverage across the globe. VeloCloud SD-WAN transforms the unpredictable broadband transport to Enterprise-class quality connections, ensuring the application performance from remote locations to Azure Cloud.

To meet different deployment scenarios for customers who deploy Azure Virtual WAN, VeloCloud SD-WAN have been progressively adding more capabilities to the solution. With this new integration, customers can now deploy VeloCloud Edges directly inside Azure Virtual WAN hubs manually, resulting in an offering that natively integrates Azure Virtual WAN’s customizable routing intelligence with VeloCloud SD-WAN’s optimized last-mile connectivity.

The following diagram illustrates the VeloCloud SD-WAN and Azure vWAN NVA Manual Deployment scenario.

Figure 1. VeloCloud SD-WAN and Azure vWAN NVA Manual Deployment Scenario

Deploy VeloCloud SD-WAN Edge in Azure Virtual WAN Hub

To deploy VeloCloud Edges in a Virtual Hub manually, you must have already created a Resource Group, virtual WAN (vWAN), and virtual Hub (vHUB) on the Azure side.

Once the vWAN Hub is up and running and routing status is complete, you must meet the following prerequisites before proceeding with the Manual deployment of an Azure vWAN Network Virtual Appliance (NVA) via VeloCloud Orchestrator:

Obtain Enterprise account access to VeloCloud Orchestrator.
Obtain access to the Microsoft Azure portal with the appropriate IAM roles.
Software image requirements for this deployment are as follows:
- VeloCloud Orchestrator: 4.5.0 and above.
- VeloCloud Gateway: 4.5.0 and above.
- VeloCloud Edges: 4.2.1 and above.
Create an Azure Managed Identity. For steps, see Create Managed Identity.

To deploy VeloCloud SD-WAN Edge in Azure Virtual WAN Hub, perform the following steps:

In the Orchestrator, create a Virtual Edge by navigating to Configure > Edges > New Edge .
In the Orchestrator, once the Edges are created, change the interface settings for all Edges as follows:
- Change GE1 interface to Route with Autodetect WAN overlay.
- Change GE2 to Route with WAN overlay deactivated.
- The GE3 to GE8 interfaces are not used in this deployment.
Note: You can configure Profiles with Virtual Edge interface settings as required by this integration so that you do not have to change interface settings after creating Virtual Edges on the Orchestrator.

Note: If you attempt to downgrade an Edge from Release 4.2.1 to an earlier release, the Edge will become stuck in an activating loop.
SSH access to VeloCloud SD-WAN Azure NVAs is managed by the Azure support team. The Azure side enforces security policies that only allow the source IP address 168.63.129.16 to SSH to Azure Virtual Edges. To allow a Virtual Edge to accept SSH from this source IP, navigate to Configure > Edges > Firewall > Edge Access > Support Access , and add the IP address 168.63.129.16 under the Allow the following IPs field.

Figure 2. Configure Edge Support Access

Note: You can perform the Step 3 configuration on a Profile used by many or all of the Virtual Edges so you do not need to do it for each individual Virtual Edge.

For additional details regarding this IP configuration, see Azure IP address 168.63.129.16 overview.
Copy the Orchestrator URL and the Activation Key of each Virtual Edge.
For example:
- vcoxx-usvi1.velocloud.net
- Activation Key1: XXXX:ZE8F:YYYY:67YT
- Activation Key2: XXXX:ZE8F:ZZZZ:67YT
Login to the Azure portal and search for the "VeloCloud SD-WAN in vWAN" application in the Azure Market place. The VeloCloud SD-WAN in vWAN managed application page appears. You can use this application to automate the deployment of Virtual Edges in Virtual WAN Hub.

Figure 3. VeloCloud SD-WAN in vWAN Managed Application
Select Create on the managed application and enter the following basic details:
Figure 4. Configure VeloCloud SD-WAN in vWAN Managed Application Basic Details
- Subscription: The subscription which has the created Virtual WAN hub.
- Resource Group: Create a new resource group or select the existing one.
- Region: Select the region in which the Virtual WAN Hub is created. Virtual Edges will be deployed in that Virtual WAN Hub.
- Application Name: Enter a name for your managed application.
- Managed Resource Group- Provide the application's managed resource group. The managed resource group holds all the resources that are required by the managed application which the consumer has limited access to.
In the VeloCloud SD-WAN in Virtual WAN tab, select Virtual WAN Hub in the selected region. The Virtual Edges will be deployed in this Hub.
Figure 5. Configure VeloCloud SD-WAN in vWAN Managed Application Details

Once the customer selects a Virtual WAN Hub, the BGP neighbor IP Addresses and the ASN of the Virtual WAN Hub appears. Make a note of this information as it is needed to configure BGP neighborships on the Orchestrator.

To deploy the NVA via the Managed Application, enter the following required details, and add the already created user assigned managed identity (For steps on how to create an Azure Managed Identity, see Create Managed Identity.) to grant the Managed Application access to other existing resources:
- Scale unit: Select the scale as required.
  Table 1. Scale Unit Options
  
  Scale Unit Instance Type
  
  2 D2v2
  
  4 D3v2
  
  10 D4v2
- VeloCloud SD-WAN Orchestrator: Paste the Orchestrator URL from Step 3.
- IgnoreCertErrors: Set this flag as False. Change this flag to True only if the Orchestrator URL cannot be used and the Orchestrator IP address must be provided.
- ActivationKey for Edge1: Paste the activation key from Step 3.
- ActivationKey for Edge2: Paste the activation key from Step 3.
- BGP ASN: The ASN that will be configured on the Virtual Edges in the VeloCloud Orchestrator. The following ASNs are reserved by Azure or IANA:
  - ASNs reserved by Azure:
    - Public ASNs: 8074, 8075, and 12076.
    - Private ASNs: 65515, 65517, 65518, 65519, and 65520.
  - ASNs reserved by IANA:
    - 23456, 64496-64511, 65535-65551, and 429496729.
- ClusterName: Enter a unique name for the deployment which does not include special characters such as #, @, _,-, and so on.
- User assigned managed identity: Select the identity to deploy the NVA by selecting the +Add button. In the Add user assigned managed identity section that appears on the right-side of the page, select the user assigned managed identity that you have previously created and select Add.
  
  Figure 6. Add User Assigned Managed Identity
- Once added, the user assigned managed identity appears in the User assigned managed identity table as shown in the following screenshot.
  
  Figure 7. View User Assigned Managed Identity
After entering all the required fields, select Review + create.
The deployment process will start and takes approximately 10 to 15 minutes to complete. Once the deployment is complete, the Virtual Edges will connect and activate against the Orchestrator.
Once all of the Virtual Edges are connected to the Orchestrator, you need to configure static routes and BGP neighbors so that the Virtual Edges can connect to the Azure Virtual WAN Hub:
1. Configure Static Routes: Add /32 static routes sufficient that there is a unique route pointing to the respective GE2 Interface on each Virtual Edge. To add a static route, the Orchestrator requires a next hop IP address. Acquire the next hop IP address by running the Remote Diagnostic “Interface Status” test in the Remote Diagnostics UI page of the Orchestrator. Select the first IP address of the subnet assigned to GE2 and configure it as the next hop. The following image shows an IP address assigned to GE2 as 10.101.112.6/25 and the first IP address of this subnet is 10.101.112.1, which is used to configure the static route on the Orchestrator.
  The following is the output from Test & Troubleshoot > Remote Diagnostics > Interface Status diagnostic test.
  
  Figure 8. View Interface Status
  
  Two static routes are configured on the Edge to reach BGP neighbors as shown in the following screenshot.
  
  Figure 9. View Static Route Settings
2. BGP Neighbor Configuration: Configure BGP neighbors for each Virtual Edge as shown in the following diagram. Use BGP neighbor IPs and the ASN number as displayed in the information message in Step 7.
  
  Figure 10. View BGP Neighbor Configuration
  
  Once static routes and BGP neighborships are configured, the Virtual Edges should begin learning routes from the Azure Virtual WAN Hub. BGP neighborship status can be verified under Monitor > Network Services .
(Optional) Add the Virtual Edges into a cluster. Go to Configure > Network Services > Edge Cluster , create a new cluster Hub and add the Virtual Edges into the cluster.
(Optional) To add a Virtual Network Connection with the Virtual Networks (vNETs) to the vHub, go to Azure vWAN > Connectivity > Virtual network connections .

Figure 11. Add a Virtual Network Connection
Select Add Connection and provide a Connection Name, choose the Hub, Subscription, and Resource Group. Select the vNET and the associated Route table that needs to be connected to the Hub. For example, it is the ‘default’ route table in a vNET.

Figure 12. Add Connection

For the vWAN NVA Edge, the image is a 2 NIC Deployment, in other words the GE1 interface is not used as the ‘Management’ interface. This is unique to the vWAN NVA image.

On all other cloud Edges, the GE1 interface is allocated as a ‘Management’ interface and cannot be used for data traffic.

Note: For Customers whose Azure vWAN Hub Routers are created with 'Cloud Services infrastructure', see Hub Upgrade Instructions for VeloCloud Edge Deployed as Azure vWAN NVA.

Accessing the Command Line of Virtual Edges Deployed into an Azure vWAN vHub

Azure vWAN is operated as a managed service. Unlike other virtual machines deployed into Azure, vWAN does not offer the ability to associate a public key to the virtual machine (VM) when it is configured. Since Azure also does not allow password-based SSH authentication, this effectively renders the CLI of the vEdge unreachable.

To overcome these restrictions and access the vEdge's CLI for troubleshooting and operational purposes, the VeloCloud SD-WAN's Secure Edge Access feature should be used. This will use the Orchestrator to create key-based, per-user SSH access to the vEdge's CLI.

Refer to the following documentation to enable Secure Edge Access: Access SD-WAN Edges Using Key-based Authentication.

Note: During Secure Edge Access key creation process, specifying a password is listed as "optional." However, including a password is required to be configured to access Azure NVAs. The user will be prompted to provide the password during the SSH login process after first using key-based authentication.

Table 1. Scale Unit Options
Scale Unit	Instance Type
2	D2v2
4	D3v2
10	D4v2

Create Managed Identity

This section discusses the steps to create an Azure Managed Identity.

Under Subscription, create a Custom Role say vWANNVACustomRole with the following permissions.

"permissions": [ { "actions": [ "Microsoft.Network/publicIPAddresses/join/action", "Microsoft.Network/publicIPAddresses/read", "Microsoft.Network/networkVirtualAppliances/delete", "Microsoft.Network/networkVirtualAppliances/read", "Microsoft.Network/networkVirtualAppliances/write", "Microsoft.Network/networkVirtualAppliances/restart/action", "Microsoft.Network/networkVirtualAppliances/getDelegatedSubnets/action", "Microsoft.Network/virtualHubs/read" ], "notActions": [], "dataActions": [], "notDataActions": [] } ]

Create a new user-assigned managed identity say NVAmgdIdentity in the desired Managed Group and Region.

Figure 13. Create User Assigned Managed Identity
Under the resource group where the vWAN Hub is deployed, assign the Managed Identity by navigating to Resource Group > IAM > Add Role Assignment .

In the Add role assignment screen, under the Role tab, search for the custom role that you created i.e., 'vWANNVACustomRole'.

Figure 14. Add Role Assignment
In the Members tab, select Managed Identity. In the Select managed identities section that appears on the right-side of the page, select the user assigned managed identity 'NVAmgdIdentity' that you have previously created and select Select. The selected managed identity appears under the Selected Members area.

Figure 15. Add Role Assignment
Select Review+Assign to assign the selected Managed Identity the custom role with scope as the resource group the vWAN hub is deployed in.

Hub Upgrade Instructions for VeloCloud Edge Deployed as Azure vWAN NVA

This document is intended for customers who use VeloCloud Edges in Azure and deploy them as Network Virtual Appliances (NVAs) in the Azure Virtual WAN (vWAN) Hub.

For additional information, see Virtual WAN FAQ.

Upgrade Instructions

Azure is deprecating its Cloud Services-based infrastructure, so the Virtual WAN team is upgrading their virtual routers from their current Cloud Services infrastructure to Virtual Machine Scale Sets based deployments. If you navigate to your Virtual WAN hub resource and see a message to upgrade your router to the latest version as shown in the following screenshot, select Update router to latest software version button to initiate router upgrade.

Note: All newly created Virtual Hubs will be automatically deployed on the latest Virtual Machine Scale Sets-based infrastructure and do not require this upgrade.

After selecting Upgrade Router to the latest software version, a message will indicate that this operation must be performed during a maintenance window.

Figure 17. Upgrade Router to the Latest Software Version

The Hub Status would display Updating and the Routing State as Provisioning. This process will take approximately 30 to 60 minutes to complete.

After successful completion of the router update, the Hub Status should display Succeeded and the Routing State should display Provisioned as shown in the following screenshot.

Figure 19. Successful Completion of the Router Update

IP addresses are represented in the Virtual Hub's resource JSON as the virtualRouterIps field. Alternatively, you can find it in the Virtual Hub > BGP Peers menu.

Copy the IP Addresses. For example, in this case the IP addresses are 172.16.32.8 and 172.16.32.9. These are the IP addresses on the Virtual Hub that the BGP Peers ( VeloCloud SD-WAN NVA) will need to be configured.

On the Orchestrator, the Virtual Edge BGP connections to the Virtual Hub will be displayed as Down, either in Connect or Active state.

Before configuring BGP neighbors on the Virtual Edge, static routes must be configured to allow the Virtual Edges to connect to the Azure Virtual WAN Hub.

Static Routes Configuration

To configure static routes, add sufficient /32 static routes to ensure that there is a unique route pointing to the respective GE2 interface on each Virtual Edge. To add a static route, the Orchestrator requires a next-hop IP address. The next hop IP address can be obtained by running the Remote Diagnostic “Interface Status” test in the Remote Diagnostics UI page of the Orchestrator. Select the first IP address of the subnet assigned to GE2 and configure it as the next hop.

The following image shows an IP address assigned to GE2 as 172.16.112.5/25, with the first IP address of this subnet being 172.16.112.1. This IP address is used to configure the static route on the Orchestrator.

The following is the output from Test & Troubleshoot > Remote Diagnostics > Interface Status diagnostic test.

Two static routes are configured on the Edge to reach BGP neighbors, as illustrated in the following screenshot.

Figure 22. Configured Static Routes on the Edge

BGP Neighbor Configuration

Configure BGP neighbors for each Virtual Edge as shown in the following screenshot. Use the BGP neighbor IPs and the ASN number as displayed in the virtual Hub BGP Peers output. Also, make sure to configure the BGP Max-Hop to 2.

Figure 23. Configure BGP Neighbors for Each Virtual Edge

Once static routes and BGP neighbors have been configured, the Virtual Edges should begin learning routes from the Azure Virtual WAN Hub. You can verify the status of the BGP neighbors under Monitor > Network Services .

VeloCloud SD-WAN 6.1 - Administration Guide - VeloCloud SD-WAN in Azure Virtual WAN Hub Deployment 打印