vCD Extender – Warm Migration Demonstration

Continuation of vCD Extender Setup – Part 1 and Part 2

Now you’re ready for your first migration. Going back to my high-level diagram, I will be migrating WebApp2 to the Provider vCD environment and ensuring connectivity is available over the VPN tunnel.

Before we start, we can see WebApp1 can ping WebApp2:

Let’s go to Migrations – Warm Migration:

Let’s select WebApp2 – remember, it must be running for a warm migration:

Selecting the defaults – I only have one network here, so it defaulted to our L2VPN network of “company-network”

Changing the Target RPO to 5 minutes (this is the fastest RPO currently) and naming the migration job “WebApp2-Migration” – this will start immediately.

Okay, now we see the job running! Depending on your network connectivity, this will take some time to replicate over the WAN.

So let’s see what happens behind the scenes.

On my provider side, I see my cloud replication instance accepting the incoming replication:

We can also see on the provider vCenter, a shell VM is substantiated and then removed from inventory. I also see some kind of SSL update to one of the hosts (I presume this is to allow direct replication):

On the tenant side, I can also see a task in here to enable and start replication of WebApp2:

While the On-Prem replication instance shows an outbound replication ID:

OK, initial replication is complete! We are now ready to cut WebApp2 over.

Back to Workloads subtab -> hit the Start Cutover button

It will prompt you for the target and you can also choose to power it on after cutover. This is handy if you had multiple VM’s that were ready for cutover, but perhaps not ready to cut them over at the same time.

Hitting the start button, you are asked one last time if you’re ready to cut over…

Alright, we are cutting over…

We can see the pings stop…

We can see WebApp2 was shut down –

I start seeing some new tasks on the provider vCenter side…

Now we also see WebApp2 pop up in vCD –

Looks like we have power on!

And now waiting a few moments for the VM to fully boot…and we have connectivity! Note that I am double nested here, so this duplicate packet behavior is something I’ve seen with VPN tunnels. 

WebApp1 can see WebApp2, let’s see if it’s the reverse! Yes, we do.

By my count, this took about 4 minutes from me pushing the button, final synchronization and having network connectivity between my two WebApp servers.

I hope to have a demo video next – enjoy setting up vCD Extender and migrating your customers to your vCD Cloud! #longlivevCD

-Daniel

vCD Extender – Warm Migration Setup – 2 of 2

Continuation of vCD Extender – Warm Migration Setup – 1 of 2

3. Configure L2 Appliance configuration inside of tenant Extender Appliance

Alright, now we are ready to set up the DC Extension on the tenant side. First, we need to tell the tenant Extender appliance where to position the L2VPN-Client edge in the tenant environment.

First, let’s log into the Extender tenant appliance:

Click on DC Extensions and Add Appliance Configuration (top button) – remember, our tenant does not have NSX (but could as you see below!) so we will have Extender deploy a standalone L2VPN Client edge.

Now let’s put in the configuration parameters for the L2 Appliance. As you can see, this is for my “Company-Network” so I set its IP, default GW, and the prefix length. Make sure you hit the Add button before hitting Create:

Ok, great, now we have an L2 Appliance Configuration:

Next, let’s get into the Plugin and establish our DC Extension.

4. Establish DC Extension in Extender plugin

Last step before we can start migrating! This is assuming you have your provider cloud established – let’s now go to DC Extensions

Click on New Extension –

Put in a name, source info (you can see I selected my Company-Network) and Destination will populate your vDC along with the subinterface Network. If you do not see any Network, something went awry with the trunk interface, check it out.

Also, note that we added the local Egress default gateway IP to enable the L2VPN-Client to route any packets sent towards the Egress Optimization IP address locally, and send everything else over the tunnel. If the default gateway for the virtual machines (belonging to the subnets you’re stretching) is same across the two sites you will need this setting to ensure traffic will be locally routed on each site. Should you need to migrate VM from one site to the other, you can do so without changing the network configuration on the guest VM and hence the mobility.

Now click that Start button…

Now we will see the status as Connecting…

So what’s happening in the background? Well, Extender is now deploying the L2VPN standalone edge and also creating a Distributed Port Group for the sink traffic:

On the Provider side, what do we have here? We have a new site config under the L2VPN-Server:

Really slick! So Extender automatically creates a new config based on this configuration.

Going back to the tenant, I see the IP was set to the L2VPN-Client Edge and the sink port was configured completely:

Taking a closer look at the DVS port Extender created, we can see Promiscuous Mode and Forged Transmits were set to Accept:

Oh, and look at that – it’s Connected! Woohoo!!!

On the Provider NSX side, we can see the tunnel is up and kicking:

COMPLETE! You are now ready for your first warm migration.

Again, big thanks to my buddy Wissam for the collaboration here.

Next – let’s do a warm migration!

vCD Extender – Warm Migration Setup – 1 of 2

Hey everybody! I appreciate all of the interest in vCloud Director Extender – today I will be walking through setting up vCD Extender for warm migrations. This is going to be a lengthy post, but very straightforward once you go through the steps.

First off, what is a “warm migration?” Well, a warm migration in vCD Extender allows you to continue to replicate a VM until you’re ready to cut it over. The goal of a warm migration is to minimize the downtime required for this migration. This is NOT a cross-vCenter vMotion: Extender replicates it using the H4 replication system and then requests you to cut it over. Again, the goal is to minimize the downtime to minutes, if not seconds.

Big thank you to my peer Wissam Mahmassani – he and I worked on this together to demonstrate functionality.

To start, I’m going to list out the high-level steps on setting this up.

  1. Advanced Gateway Services enabled on vCD Tenant
  2. Create trunked network inside of org VDC and setup L2VPN-Server
  3. Setup DC Extension placement inside of tenant Extender Manager
  4. Establish DC Extension in Extender plugin
  5. Warm Migrate away!

Before we get started, here’s what we are going to build out. If you are looking for how Extender is logically laid out, please check out my initial Extender post here.

 

This is the same Extender/vCD Topology as before – pretty flat demo environment, but I’m using Company as my tenant now.

1. Advanced Gateway Services Permissions Setup

The organization administrator will require advanced entitlements for setting up the L2VPN connection and Extender functionality. This is a little painful to do manually, so after further consideration, I decided to take Jon Waite’s PowerShell script and modify it (Thank you so much, Jon!) – read more here on how Jon created this.

Note that I am doing this in a vCD 8.20 environment. Permissions are similar but I believe 9.0 may have these by default (need to verify this)DCP 17Jan2018: Wrong – per my testing, I modified the below script for vCD 9.x environments here. As of this blog post, 9.0 does not have warm migration functionality – this will be added in a patch release.

Below is the PowerShell script I modified and will demonstrate how this works.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
# vCD Extender Permissions Setup - initially created by KiwiCloud.Ninja - modified by Daniel Paluszek - paluszek.com
# Script to add new OrgRights options for administering advanced Edge Gateway to a vCloud Director organisation.
# Note that Organisation roles (e.g. Organizational Administrator) still need to be edited to add these rights once
# this script has been run against their org.
# NOTE: You must be connected to the vCloud API (Connect-CIServer) with a System administrative user prior to running the script for this to work.
# Add your Org name in line 7 while the vCD instance name is added in line 8
$OrgToUpdate = ''
$APIendpoint = ''

Function vCloud-REST(
[Parameter(Mandatory=$true)][string]$URI,
[string]$ContentType,
[string]$Method = 'Get',
[string]$ApiVersion = '27.0',
[string]$Body,
[int]$Timeout = 40
)
{
$mysessionid = ($global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }).SessionId
$Headers = @{"x-vcloud-authorization" = $mysessionid; "Accept" = 'application/*+xml;version=' + $ApiVersion}
if (!$ContentType) { Remove-Variable ContentType }
if (!$Body) { Remove-Variable Body }
Try
{
[xml]$response = Invoke-RestMethod -Method $Method -Uri $URI -Headers $headers -Body $Body -ContentType $ContentType -TimeoutSec $Timeout
}
Catch
{
Write-Host "Exception: " $_.Exception.Message
if ( $_.Exception.ItemName ) { Write-Host "Failed Item: " $_.Exception.ItemName }
Write-Host "Exiting."
Return
}
return $response
} # Function vCloud-REST End

# The new vCloud Director API v27.0 OrgRights for vCD Extender Preparation and Advanced Networking:
$newrights = @{}
$newrights.Add("Hybrid Cloud Operations: View from-the-cloud tunnel", "629c90fd-78a4-3929-98bd-57e4747d067b")
$newrights.Add("Organization vDC Distributed Firewall: Enable/Disable", "a100f6a0-2c81-3b61-90c3-c4dbd721b3a8")
$newrights.Add("Organization vDC Gateway: Configure BGP Routing", "2c4eb5ac-15f5-33f0-8b4a-680b3a1d3707")
$newrights.Add("Organization vDC Gateway: Configure DHCP", "be1abe9a-7ddc-38f6-bdf3-94affb01e46b")
$newrights.Add("Organization vDC Gateway: Configure Firewall", "b755b050-772e-3c9c-9197-111c286f563d")
$newrights.Add("Organization vDC Gateway: Configure IPSec VPN", "209cde55-55db-33f1-8357-b27bba6898ed")
$newrights.Add("Organization vDC Gateway: Configure L2 VPN", "eeb2b2a0-33a1-36d4-a121-6547ad992d59")
$newrights.Add("Organization vDC Gateway: Configure Load Balancer", "27be9828-4ce4-353e-8f68-5cd69260d94c")
$newrights.Add("Organization vDC Gateway: Configure NAT", "c9e19573-3d54-3d4a-98f2-f56e446a8ef9")
$newrights.Add("Organization vDC Gateway: Configure OSPF Routing", "3b337aef-42a8-3ed1-8616-341152bc5790")
$newrights.Add("Organization vDC Gateway: Configure Remote Access", "72c5e652-c8d7-3f19-ab83-283d30cb679f")
$newrights.Add("Organization vDC Gateway: Configure SSL VPN", "92b7d500-6bb6-3176-b9eb-d1fda4ce444d")
$newrights.Add("Organization vDC Gateway: Configure Static Routing", "f72af304-97b0-379e-9d6d-68eb89bdc6cf")
$newrights.Add("Organization vDC Gateway: View BGP Routing", "d9dabcab-579e-33c5-807b-dc9232bf7eff")
$newrights.Add("Organization vDC Gateway: View DHCP", "8e16d30d-1ae3-3fff-8d4b-64c342b186a9")
$newrights.Add("Organization vDC Gateway: View Firewall", "7fee6646-ec0c-34c9-9585-aff6f4d92473")
$newrights.Add("Organization vDC Gateway: View IPSec VPN", "82beb471-ab7f-3e2b-a615-136ba6645525")
$newrights.Add("Organization vDC Gateway: View L2 VPN", "105191de-9e29-3495-a917-05fcb5ec1ad0")
$newrights.Add("Organization vDC Gateway: View Load Balancer", "2a097e48-f4c4-3714-8b24-552b2d573754")
$newrights.Add("Organization vDC Gateway: View NAT", "fb860afe-2e15-3ca9-96d8-4435d1447732")
$newrights.Add("Organization vDC Gateway: View OSPF Routing", "eb525145-08e5-3934-91ef-ec80837c9177")
$newrights.Add("Organization vDC Gateway: View Remote Access", "65439584-6aad-3c2c-916f-794099ee85bf")
$newrights.Add("Organization vDC Gateway: View SSL VPN", "cdb0edb0-9623-30a8-89de-b133db7cfeab")
$newrights.Add("Organization vDC Gateway: Configure Services", "b080bb50-cff1-3258-9683-842d34255a95")
$newrights.Add("Organization vDC Gateway: Configure Syslog", "84ddb40f-a49a-35e1-918e-3f11507825d7")
$newrights.Add("Organization vDC Gateway: Configure System Logging", "ff3fc70f-fd25-3c0a-9d90-e7ff82456be5")
$newrights.Add("Organization vDC Gateway: Convert to Advanced Networking", "9dc33fcb-346d-30e1-8ffa-cf25e05ba801")

$myendpoint = $global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }

if (!$myendpoint.IsConnected) {
Write-Host "Not connected to this vCloud endpoint, use 'Connect-CIServer' before running this script."
Exit
}

$org = Get-Org -Name $OrgToUpdate -Server $APIendpoint

if (!$org) {
Write-Host "Couldn't match organization with name $OrgToUpdate, exiting."
Exit
}

$rightsuri = 'https://' + $APIendpoint + "/api/admin/org/" + $org.Id.Substring($org.Id.LastIndexOf(':')+1) + "/rights"

[xml]$rights = vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Method 'Get' -ApiVersion '27.0'

# Add the new API v27 'RightsReference' elements to the XML returned:
foreach($newrule in $newrights.Keys) {
$newright = $rights.CreateElement("RightReference", "http://www.vmware.com/vcloud/v1.5")
$newright.SetAttribute("href","https://$APIEndpoint/api/admin/right/$($newrights.Item($newrule))")
$newright.SetAttribute("name",$newrule)
$newright.SetAttribute("type","application/vnd.vmware.admin.right+xml")
$rights.OrgRights.AppendChild($newright)
}

# Update the Organization with the ammended rights:
vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Body $rights.InnerXml -Method 'Put' -ApiVersion '27.0'

Well, why are we doing this? By default, the org admin does not have these permissions added to the default role. So we need to add them. In this example, I will walk through what you’ll see before and after.

In this example, I’ll be adding the required permissions to org ‘T1’

Before we run this PowerShell script, here are the current permissions of the Organization Administrator. As you can see, there’s no section for Gateway Advanced Services settings.

Let’s go ahead and execute the PowerShell Script. Just make sure you have the latest PowerCLI pack installed – you will need to connect using “Connect-CIServer” first before execution.

Now login into your vCD cell –

Change the PS script to your specific parameters, as I pointed in the arrows below:

Now execute!

Now I see a whole new mess of rights! Go ahead and grant the rights to the Org Admin by checking the boxes. 

2. Create trunked network inside Org VDC and Setup L2VPN-Server

On to Step 2 – let’s go ahead and create an Org VDC network, convert it to a subinterface, and set up our L2VPN-Server.

Let’s log into our Company and create a new Org VDC Network

Setting the gateway for my 172.16.10.x Web Network I will be using.

Naming it ‘company-network’

Go ahead and right click – Convert to subinterface:

Continue on…

Now we see company-network as a subinterface:

Now we are ready to configure our L2VPN-Server, right click and Configure Services:

Go to the VPN tab, L2 VPN – then I start with Server Sites:

Add your peer site information and ensure you select your stretched interface:

Ensure you are setting this as the Server, select your Algorithm, Enabled is green, and Save Changes!

Ok, great! We have 2 of the 5 steps complete. We can also see from the Provider vCenter the VPN is set up, but down right now.

On the next post, we will finish up the setup.

Next Post -> vCD Extender Warm Migration Setup – 2 of 2

 

vCloud Director Extender – Installation Review

I’m happy to announce VMware has released vCloud Extender for vCloud Director. With that said, I was given the opportunity to provide feedback to our talented engineering and product management teams on installing and reviewing this new valuable solution set.

To start – what is vCloud Extender? vCloud Extender allows tenants to seamlessly migrate workloads to a vCloud Director environment. This is without any net new infrastructure or software purchase: the tenant just needs to add two vApps while the provider would add three – very streamlined. For a provider to get started with Extender, you just need vCloud Director for Service Providers.

Check out VMware’s Introduction to vCloud Director Extender video:

The installation of Extender is extremely streamlined for the tenant and provider environments. This post will go through the installation steps and requirements for an initial successful Extender installation.

Link for vCloud Director Extender download here 

vCloud Director Extender Documentation Link

First off, let’s talk about architecture. I covered this in a previous blog post, but it wouldn’t hurt to reiterate a few of the points. If you haven’t seen the previous blog post on vCloud Extender, check it out here.

On the provider side, we have the following workflow:

  1. Management vCenter deploys Extender Manager appliance. This provides the functionality for managing Extender.
  2. Extender Manager does the following…
    1. Registers to vCD and management vCenter instance
    2. Deploys and activates Replication Manager
    3. Deploys and activates Replication Instance
      1. Replication Instance is then pointed to Resource vCenter (think the Consumption environment where the tenant resides)
  3. A few points of interest:
    1. Proxy Server – you can access the Replication Manager and Replication Instance through a proxy server or a gateway. One of the requirements are to provide a proxy with a public endpoint and configure rules to route the network traffic to the replication components.
      1. This would allow you to have vCD, Extender Manager, and a Reverse Proxy in the DMZ while the replication instances and Replication Manager are in a private management network behind an Edge/FW.
    2. Control traffic – traffic between the Extender instances and replicator instances – all run over HTTPs / 443 traffic. It’s important to note that the on-prem replication instance and the Replication Manager must have bi-directional 443 communication. This is something to ensure your tenants are aware of when planning for installation.
    3. Replication Traffic goes over encrypted TCP which is on port 44045.

On the tenant side, it’s pretty straight forward. We have two appliances to deploy:

  1. Extender Appliance – manages the association with the tenant vCenter instance along with replication instance deployments.
  2. Replication Appliance – is deployed from Extender Appliance and controls the migration (warm or cold) of VM’s. This is based on our new H4 engine – next generation vSphere Replication engine.

From the tenant perspective, we would have something like this:

High Level Steps for installation:

  1. Provider
    1. Deploy Extender appliance in SP management vCenter environment.
    2. Bring up the Extender Manager UI (HTML access)
    3. Start the Configuration Wizard
      1. Associate it with the Management vCenter (where your other management appliances will reside)
      2. Register with vCD
      3. Register with Resource vCenter instance
      4. Deploy Replication Manager – then activate
      5. Deploy Replicator Instance – then activate
    4. Complete – verify Extender sees all connected resources.
    5. For L2 VPN Network Span, this is done by having administrator privileges to the L2 VPN configuration in vCD. From there, we need to establish a L2 VPN Server and L2 VPN Client (Client would be on the tenant environment). A future blog post will cover this in further detail.
  2. Tenant
    1. Deploy Extender appliance in tenant vCenter environment.
    2. Bring up the Extender UI (HTML)
    3. Start the Configuration Wizard
      1. Register tenant vCenter
      2. Register Plugin with vCenter instance
      3. Deploy Replicator instance – then activate
    4. Complete – from here, we could deploy a Network Stretch function (this will be covered in another blog post)

Demonstration Environment for Extender Installation

For the sake of this post and video, I wanted to create a streamlined installation process for vCD Extender. Therefore, my demonstration environment architecture looks like the following:

As you probably can imagine, this is not built for production state, but just to demonstrate the Extender installation. Initial recommendations:

  1. Separation of management/resource vCenter instances
  2. Separation of compute/management clusters for Provider environment
  3. I will be using internal IP’s/FQDN’s for this demonstration – a production environment would have externally facing resources and/or DNS addresses.
  4. Utilization of a Reverse Proxy to segment DMZ / Private Management connectivity. I will point out these options during the installation video.
  5. L2 VPN connectivity / warm migrations will be covered in a future blog post. L2 VPN is not required for cold migrations.

Provider Installation Video:

Tenant Installation Video:

Tenant Connection to Provider and Cold Migration:

 

Any feedback would be much appreciated.

-Daniel