vCD Extender – Warm Migration Video

This will be my last post on vCD Extender – an exciting addition to vCloud Director and wanted to provide a comprehensive look at Extender’s functionality.

Link to vCD Extender Warm Migration Setup – Part 1 of 2

Link to vCD Extender Warm Migration Setup – Part 1 of 2

Link to vCD Extender Warm Migration Demo Walkthrough

With this video, I created a new VM called WebApp3 and plan to migrate it over to the vCD Cloud environment. I’ve updated my logical demo diagram to the following:

Here’s the video I recorded – enjoy!

vCD Extender – Warm Migration Demonstration

Continuation of vCD Extender Setup – Part 1 and Part 2

Now you’re ready for your first migration. Going back to my high-level diagram, I will be migrating WebApp2 to the Provider vCD environment and ensuring connectivity is available over the VPN tunnel.

Before we start, we can see WebApp1 can ping WebApp2:

Let’s go to Migrations – Warm Migration:

Let’s select WebApp2 – remember, it must be running for a warm migration:

Selecting the defaults – I only have one network here, so it defaulted to our L2VPN network of “company-network”

Changing the Target RPO to 5 minutes (this is the fastest RPO currently) and naming the migration job “WebApp2-Migration” – this will start immediately.

Okay, now we see the job running! Depending on your network connectivity, this will take some time to replicate over the WAN.

So let’s see what happens behind the scenes.

On my provider side, I see my cloud replication instance accepting the incoming replication:

We can also see on the provider vCenter, a shell VM is substantiated and then removed from inventory. I also see some kind of SSL update to one of the hosts (I presume this is to allow direct replication):

On the tenant side, I can also see a task in here to enable and start replication of WebApp2:

While the On-Prem replication instance shows an outbound replication ID:

OK, initial replication is complete! We are now ready to cut WebApp2 over.

Back to Workloads subtab -> hit the Start Cutover button

It will prompt you for the target and you can also choose to power it on after cutover. This is handy if you had multiple VM’s that were ready for cutover, but perhaps not ready to cut them over at the same time.

Hitting the start button, you are asked one last time if you’re ready to cut over…

Alright, we are cutting over…

We can see the pings stop…

We can see WebApp2 was shut down –

I start seeing some new tasks on the provider vCenter side…

Now we also see WebApp2 pop up in vCD –

Looks like we have power on!

And now waiting a few moments for the VM to fully boot…and we have connectivity! Note that I am double nested here, so this duplicate packet behavior is something I’ve seen with VPN tunnels. 

WebApp1 can see WebApp2, let’s see if it’s the reverse! Yes, we do.

By my count, this took about 4 minutes from me pushing the button, final synchronization and having network connectivity between my two WebApp servers.

I hope to have a demo video next – enjoy setting up vCD Extender and migrating your customers to your vCD Cloud! #longlivevCD


vCD Extender – Warm Migration Setup – 2 of 2

Continuation of vCD Extender – Warm Migration Setup – 1 of 2

3. Configure L2 Appliance configuration inside of tenant Extender Appliance

Alright, now we are ready to set up the DC Extension on the tenant side. First, we need to tell the tenant Extender appliance where to position the L2VPN-Client edge in the tenant environment.

First, let’s log into the Extender tenant appliance:

Click on DC Extensions and Add Appliance Configuration (top button) – remember, our tenant does not have NSX (but could as you see below!) so we will have Extender deploy a standalone L2VPN Client edge.

Now let’s put in the configuration parameters for the L2 Appliance. As you can see, this is for my “Company-Network” so I set its IP, default GW, and the prefix length. Make sure you hit the Add button before hitting Create:

Ok, great, now we have an L2 Appliance Configuration:

Next, let’s get into the Plugin and establish our DC Extension.

4. Establish DC Extension in Extender plugin

Last step before we can start migrating! This is assuming you have your provider cloud established – let’s now go to DC Extensions

Click on New Extension –

Put in a name, source info (you can see I selected my Company-Network) and Destination will populate your vDC along with the subinterface Network. If you do not see any Network, something went awry with the trunk interface, check it out.

Also, note that we added the local Egress default gateway IP to enable the L2VPN-Client to route any packets sent towards the Egress Optimization IP address locally, and send everything else over the tunnel. If the default gateway for the virtual machines (belonging to the subnets you’re stretching) is same across the two sites you will need this setting to ensure traffic will be locally routed on each site. Should you need to migrate VM from one site to the other, you can do so without changing the network configuration on the guest VM and hence the mobility.

Now click that Start button…

Now we will see the status as Connecting…

So what’s happening in the background? Well, Extender is now deploying the L2VPN standalone edge and also creating a Distributed Port Group for the sink traffic:

On the Provider side, what do we have here? We have a new site config under the L2VPN-Server:

Really slick! So Extender automatically creates a new config based on this configuration.

Going back to the tenant, I see the IP was set to the L2VPN-Client Edge and the sink port was configured completely:

Taking a closer look at the DVS port Extender created, we can see Promiscuous Mode and Forged Transmits were set to Accept:

Oh, and look at that – it’s Connected! Woohoo!!!

On the Provider NSX side, we can see the tunnel is up and kicking:

COMPLETE! You are now ready for your first warm migration.

Again, big thanks to my buddy Wissam for the collaboration here.

Next – let’s do a warm migration!

vCD Extender – Warm Migration Setup – 1 of 2

Hey everybody! I appreciate all of the interest in vCloud Director Extender – today I will be walking through setting up vCD Extender for warm migrations. This is going to be a lengthy post, but very straightforward once you go through the steps.

First off, what is a “warm migration?” Well, a warm migration in vCD Extender allows you to continue to replicate a VM until you’re ready to cut it over. The goal of a warm migration is to minimize the downtime required for this migration. This is NOT a cross-vCenter vMotion: Extender replicates it using the H4 replication system and then requests you to cut it over. Again, the goal is to minimize the downtime to minutes, if not seconds.

Big thank you to my peer Wissam Mahmassani – he and I worked on this together to demonstrate functionality.

To start, I’m going to list out the high-level steps on setting this up.

  1. Advanced Gateway Services enabled on vCD Tenant
  2. Create trunked network inside of org VDC and setup L2VPN-Server
  3. Setup DC Extension placement inside of tenant Extender Manager
  4. Establish DC Extension in Extender plugin
  5. Warm Migrate away!

Before we get started, here’s what we are going to build out. If you are looking for how Extender is logically laid out, please check out my initial Extender post here.


This is the same Extender/vCD Topology as before – pretty flat demo environment, but I’m using Company as my tenant now.

1. Advanced Gateway Services Permissions Setup

The organization administrator will require advanced entitlements for setting up the L2VPN connection and Extender functionality. This is a little painful to do manually, so after further consideration, I decided to take Jon Waite’s PowerShell script and modify it (Thank you so much, Jon!) – read more here on how Jon created this.

Note that I am doing this in a vCD 8.20 environment. Permissions are similar but I believe 9.0 may have these by default (need to verify this)DCP 17Jan2018: Wrong – per my testing, I modified the below script for vCD 9.x environments here. As of this blog post, 9.0 does not have warm migration functionality – this will be added in a patch release.

Below is the PowerShell script I modified and will demonstrate how this works.

# vCD Extender Permissions Setup - initially created by KiwiCloud.Ninja - modified by Daniel Paluszek -
# Script to add new OrgRights options for administering advanced Edge Gateway to a vCloud Director organisation.
# Note that Organisation roles (e.g. Organizational Administrator) still need to be edited to add these rights once
# this script has been run against their org.
# NOTE: You must be connected to the vCloud API (Connect-CIServer) with a System administrative user prior to running the script for this to work.
# Add your Org name in line 7 while the vCD instance name is added in line 8
$OrgToUpdate = ''
$APIendpoint = ''

Function vCloud-REST(
[string]$Method = 'Get',
[string]$ApiVersion = '27.0',
[int]$Timeout = 40
$mysessionid = ($global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }).SessionId
$Headers = @{"x-vcloud-authorization" = $mysessionid; "Accept" = 'application/*+xml;version=' + $ApiVersion}
if (!$ContentType) { Remove-Variable ContentType }
if (!$Body) { Remove-Variable Body }
[xml]$response = Invoke-RestMethod -Method $Method -Uri $URI -Headers $headers -Body $Body -ContentType $ContentType -TimeoutSec $Timeout
Write-Host "Exception: " $_.Exception.Message
if ( $_.Exception.ItemName ) { Write-Host "Failed Item: " $_.Exception.ItemName }
Write-Host "Exiting."
return $response
} # Function vCloud-REST End

# The new vCloud Director API v27.0 OrgRights for vCD Extender Preparation and Advanced Networking:
$newrights = @{}
$newrights.Add("Hybrid Cloud Operations: View from-the-cloud tunnel", "629c90fd-78a4-3929-98bd-57e4747d067b")
$newrights.Add("Organization vDC Distributed Firewall: Enable/Disable", "a100f6a0-2c81-3b61-90c3-c4dbd721b3a8")
$newrights.Add("Organization vDC Gateway: Configure BGP Routing", "2c4eb5ac-15f5-33f0-8b4a-680b3a1d3707")
$newrights.Add("Organization vDC Gateway: Configure DHCP", "be1abe9a-7ddc-38f6-bdf3-94affb01e46b")
$newrights.Add("Organization vDC Gateway: Configure Firewall", "b755b050-772e-3c9c-9197-111c286f563d")
$newrights.Add("Organization vDC Gateway: Configure IPSec VPN", "209cde55-55db-33f1-8357-b27bba6898ed")
$newrights.Add("Organization vDC Gateway: Configure L2 VPN", "eeb2b2a0-33a1-36d4-a121-6547ad992d59")
$newrights.Add("Organization vDC Gateway: Configure Load Balancer", "27be9828-4ce4-353e-8f68-5cd69260d94c")
$newrights.Add("Organization vDC Gateway: Configure NAT", "c9e19573-3d54-3d4a-98f2-f56e446a8ef9")
$newrights.Add("Organization vDC Gateway: Configure OSPF Routing", "3b337aef-42a8-3ed1-8616-341152bc5790")
$newrights.Add("Organization vDC Gateway: Configure Remote Access", "72c5e652-c8d7-3f19-ab83-283d30cb679f")
$newrights.Add("Organization vDC Gateway: Configure SSL VPN", "92b7d500-6bb6-3176-b9eb-d1fda4ce444d")
$newrights.Add("Organization vDC Gateway: Configure Static Routing", "f72af304-97b0-379e-9d6d-68eb89bdc6cf")
$newrights.Add("Organization vDC Gateway: View BGP Routing", "d9dabcab-579e-33c5-807b-dc9232bf7eff")
$newrights.Add("Organization vDC Gateway: View DHCP", "8e16d30d-1ae3-3fff-8d4b-64c342b186a9")
$newrights.Add("Organization vDC Gateway: View Firewall", "7fee6646-ec0c-34c9-9585-aff6f4d92473")
$newrights.Add("Organization vDC Gateway: View IPSec VPN", "82beb471-ab7f-3e2b-a615-136ba6645525")
$newrights.Add("Organization vDC Gateway: View L2 VPN", "105191de-9e29-3495-a917-05fcb5ec1ad0")
$newrights.Add("Organization vDC Gateway: View Load Balancer", "2a097e48-f4c4-3714-8b24-552b2d573754")
$newrights.Add("Organization vDC Gateway: View NAT", "fb860afe-2e15-3ca9-96d8-4435d1447732")
$newrights.Add("Organization vDC Gateway: View OSPF Routing", "eb525145-08e5-3934-91ef-ec80837c9177")
$newrights.Add("Organization vDC Gateway: View Remote Access", "65439584-6aad-3c2c-916f-794099ee85bf")
$newrights.Add("Organization vDC Gateway: View SSL VPN", "cdb0edb0-9623-30a8-89de-b133db7cfeab")
$newrights.Add("Organization vDC Gateway: Configure Services", "b080bb50-cff1-3258-9683-842d34255a95")
$newrights.Add("Organization vDC Gateway: Configure Syslog", "84ddb40f-a49a-35e1-918e-3f11507825d7")
$newrights.Add("Organization vDC Gateway: Configure System Logging", "ff3fc70f-fd25-3c0a-9d90-e7ff82456be5")
$newrights.Add("Organization vDC Gateway: Convert to Advanced Networking", "9dc33fcb-346d-30e1-8ffa-cf25e05ba801")

$myendpoint = $global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }

if (!$myendpoint.IsConnected) {
Write-Host "Not connected to this vCloud endpoint, use 'Connect-CIServer' before running this script."

$org = Get-Org -Name $OrgToUpdate -Server $APIendpoint

if (!$org) {
Write-Host "Couldn't match organization with name $OrgToUpdate, exiting."

$rightsuri = 'https://' + $APIendpoint + "/api/admin/org/" + $org.Id.Substring($org.Id.LastIndexOf(':')+1) + "/rights"

[xml]$rights = vCloud-REST -URI $rightsuri -ContentType 'application/' -Method 'Get' -ApiVersion '27.0'

# Add the new API v27 'RightsReference' elements to the XML returned:
foreach($newrule in $newrights.Keys) {
$newright = $rights.CreateElement("RightReference", "")

# Update the Organization with the ammended rights:
vCloud-REST -URI $rightsuri -ContentType 'application/' -Body $rights.InnerXml -Method 'Put' -ApiVersion '27.0'

Well, why are we doing this? By default, the org admin does not have these permissions added to the default role. So we need to add them. In this example, I will walk through what you’ll see before and after.

In this example, I’ll be adding the required permissions to org ‘T1’

Before we run this PowerShell script, here are the current permissions of the Organization Administrator. As you can see, there’s no section for Gateway Advanced Services settings.

Let’s go ahead and execute the PowerShell Script. Just make sure you have the latest PowerCLI pack installed – you will need to connect using “Connect-CIServer” first before execution.

Now login into your vCD cell –

Change the PS script to your specific parameters, as I pointed in the arrows below:

Now execute!

Now I see a whole new mess of rights! Go ahead and grant the rights to the Org Admin by checking the boxes. 

2. Create trunked network inside Org VDC and Setup L2VPN-Server

On to Step 2 – let’s go ahead and create an Org VDC network, convert it to a subinterface, and set up our L2VPN-Server.

Let’s log into our Company and create a new Org VDC Network

Setting the gateway for my 172.16.10.x Web Network I will be using.

Naming it ‘company-network’

Go ahead and right click – Convert to subinterface:

Continue on…

Now we see company-network as a subinterface:

Now we are ready to configure our L2VPN-Server, right click and Configure Services:

Go to the VPN tab, L2 VPN – then I start with Server Sites:

Add your peer site information and ensure you select your stretched interface:

Ensure you are setting this as the Server, select your Algorithm, Enabled is green, and Save Changes!

Ok, great! We have 2 of the 5 steps complete. We can also see from the Provider vCenter the VPN is set up, but down right now.

On the next post, we will finish up the setup.

Next Post -> vCD Extender Warm Migration Setup – 2 of 2