A Deeper Look into NSX L2VPN with vCD Extender Orchestration

I’ve been rebuilding my vCloud Director (vCD) lab and running through a few connectivity scenarios. Moreover, I wanted to write and share my findings on orchestrating an on-prem NSX environment connecting to a vCD/Provider environment using vCloud Director Extender (VXLAN to VXLAN). In vCD Extender, this is also known as a DC Extension.

To back up, let’s talk about how NSX provides flexible architecture as it relates to Provider scalability and connectivity. My esteemed colleagues did some great papers from our vCloud Architecture Toolkit (vCAT):

Before I get started, I also think this is a good guide for planning out VXLAN <–> VXLAN VPN connectivity.

Let’s look at my lab design from a network / logical perspective –

As you can see above, I have my Acme-Cloud organization available on the left with a single VM in the 172.16.20.x/24 network that’s running on NSX/VXLAN.

On the right, we have “Acme DC” that’s also using NSX and has a logical switch named “Acme-Tier” with the same network subnet.

The orange Extender Deployed L2VPN Client is what’s deployed by vCD Extender on tunnel creation. We’re going to walk through how Extender creates this L2VPN tunnel within an on-prem NSX environment.

Provider

  1. This is very similar to my warm migration setup, so I’m going to try not to duplicate material.
  2. I have my Acme-Cloud-Tier Org VDC Network that was converted to a Subinterface inside of vCD: 
  3. We can see in the Edge Services, my L2VPN Server has been set up with a default site configuration. However, vCD Extender creates its own site configuration – 
  4. Extender generates a unique ID for the username and password. This is done when the DC Extension is executed by the tenant. I also established the egress optimization gateway address for local traffic. 

Tenant – vCD Extender Setup

  1. Before we can create a Data Center Extension, we need two required fields for NSX deployments.
  2. First, we need to give the required information to successfully deploy a standalone Edge that will be running our L2VPN client service. This would include the uplink network along with an IP address, gateway, and VMware-specific host/storage information.
  3. Second, we need to provide the required NSX Manager information. I’m sure this is to make the API calls required to deploy the Edge device(s) to the specified vCenter. 
  4. Once the DC Extension has been created, we would see a new Edge device under Networking & Security 

Tenant – DC Extension (L2VPN) Execution

  1. So what happens when we attempt to create a new DC Extension (or L2VPN Connection)? A few things:
    1. Creation of our trunk port for our specified subinterface
    2. Deployment of the new Edge device that will act as the L2VPN Client
    3. Reconfiguration of the trunk port (uses mcxt-tpg-l2vpn-vxlan-** prefix)
    4. Allowing NSX to do its magic along with L2VPN
  2. We can see within my task console what happened – 
  3. Voila, we have a connected L2VPN tunnel. As you can see, the blue “E” stipulates that we have a local egress IP set. I did this since I wanted to route traffic to its local interface for traffic optimization.
  4. So, what happens in the background? Well, let’s take a look at the Edge device. We can see the trunk interface was created while the subinterface is configured to point to my logical switch “Acme-Tier.” 
  5. Last of all, the L2VPN configuration was completed and established as the Client. We can now see the tunnel is alive too. 
  6. From the main vCD Extender page, we can also see traffic utilization over the tunnel. Pretty nice graph! 

Just a quick ping test, yep! WebVM can access WebVM2.

In summary, NSX to NSX DC Extensions within vCD Extender works pretty similar to Provider/VXLAN to On-Prem/VLAN. The key difference is the on-prem vCD Extender has the embedded Standalone Edge to deploy to vCenter.

Enjoy those cloud migrations!

-Daniel

Configure vCloud Director 9.x Org Rights needed for L2 VPN Stretching – vCloud Extender

NOTE – this is for vCD Extender 1.0 ONLY. If you need a permissions script for vCD Extender 1.1, please check it out here. 

I am currently working on a new lab environment that consists of vCloud Director (vCD) Extender and vCloud Availability.

Our documentation provides a method for writing out the permissions needed via curl/API commands. I decided to take a shortcut and just use a PowerShell script.

Configuring vCloud Director Organization Rights for L2 VPN Stretching – VMware Documentation

If you don’t know what the heck, I’m talking about, vCloud Director Extender is our plugin product to migrate workloads from an on-prem environment to a vCD environment. Check out my installation guides here:

vCloud Director Extender – Installation Review

vCD Extender – Warm Migration Setup

I *thought* that vCloud Director 9.x would have all of the advanced organization permissions required for L2 stretching. I was incorrect, it’s missing a few things (really actually two, but following our exact documentation).

Per my findings, my vCD 9.0.0.1 environment was missing the following from an Org Admin:

  • Organization vDC Gateway: View L2 VPN
  • Organization vDC Gateway: Configure L2 VPN

So I modified my previous script (see here) to write the required org permissions to establish and set up the L2 VPN / Data Center Connection for vCloud Director Extender.

Before I ran the script, I saw this:

Running the script….

After running the script, I now see the two new L2 VPN options available to my org admin.

Done! Now I can continue on with my L2 setup.

More to come and script is below, thanks!

-Daniel

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
# vCD Extender Permissions Setup - initially created by KiwiCloud.Ninja - modified by Daniel Paluszek - paluszek.com
# January 17th, 2018 - Modified for a vCloud Director 9.x Instance
# Script to add new OrgRights options for administering advanced Edge Gateway to a vCloud Director organisation.
# Note that Organisation roles (e.g. Organizational Administrator) still need to be edited to add these rights once
# this script has been run against their org.
# NOTE: You must be connected to the vCloud API (Connect-CIServer) with a System administrative user prior to running the script for this to work.
# Add your Org name in line 7 while the vCD instance name is added in line 8
$OrgToUpdate = 'T1'
$APIendpoint = 'vcd-01a.corp.local'

Function vCloud-REST(
[Parameter(Mandatory=$true)][string]$URI,
[string]$ContentType,
[string]$Method = 'Get',
[string]$ApiVersion = '27.0',
[string]$Body,
[int]$Timeout = 40
)
{
$mysessionid = ($global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }).SessionId
$Headers = @{"x-vcloud-authorization" = $mysessionid; "Accept" = 'application/*+xml;version=' + $ApiVersion}
if (!$ContentType) { Remove-Variable ContentType }
if (!$Body) { Remove-Variable Body }
Try
{
[xml]$response = Invoke-RestMethod -Method $Method -Uri $URI -Headers $headers -Body $Body -ContentType $ContentType -TimeoutSec $Timeout
}
Catch
{
Write-Host "Exception: " $_.Exception.Message
if ( $_.Exception.ItemName ) { Write-Host "Failed Item: " $_.Exception.ItemName }
Write-Host "Exiting."
Return
}
return $response
} # Function vCloud-REST End

# The new vCloud Director API v27.0 OrgRights for vCD Extender Preparation and Advanced Networking:
$newrights = @{}
$newrights.Add("Organization vDC Gateway: Convert to Advanced Networking", "9dc33fcb-346d-30e1-8ffa-cf25e05ba801")
$newrights.Add("Organization vDC Gateway: View L2 VPN", "105191de-9e29-3495-a917-05fcb5ec1ad0")
$newrights.Add("Organization vDC Gateway: Configure L2 VPN", "eeb2b2a0-33a1-36d4-a121-6547ad992d59")
$newrights.Add("Organization vDC Gateway: Configure Firewall", "b755b050-772e-3c9c-9197-111c286f563d")
$newrights.Add("Organization vDC Network: Edit Properties", "b0cfe989-521b-3d7f-9bc2-f23c74a99633")
$newrights.Add("Organization vDC Network: View Properties", "2c8d98ef-4acc-3be4-9214-fcb9682b7a19")

$myendpoint = $global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }

if (!$myendpoint.IsConnected) {
Write-Host "Not connected to this vCloud endpoint, use 'Connect-CIServer' before running this script."
Exit
}

$org = Get-Org -Name $OrgToUpdate -Server $APIendpoint

if (!$org) {
Write-Host "Couldn't match organization with name $OrgToUpdate, exiting."
Exit
}

$rightsuri = 'https://' + $APIendpoint + "/api/admin/org/" + $org.Id.Substring($org.Id.LastIndexOf(':')+1) + "/rights"

[xml]$rights = vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Method 'Get' -ApiVersion '27.0'

# Add the new API v27 'RightsReference' elements to the XML returned:
foreach($newrule in $newrights.Keys) {
$newright = $rights.CreateElement("RightReference", "http://www.vmware.com/vcloud/v1.5")
$newright.SetAttribute("href","https://$APIEndpoint/api/admin/right/$($newrights.Item($newrule))")
$newright.SetAttribute("name",$newrule)
$newright.SetAttribute("type","application/vnd.vmware.admin.right+xml")
$rights.OrgRights.AppendChild($newright)
}

# Update the Organization with the ammended rights:
vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Body $rights.InnerXml -Method 'Put' -ApiVersion '27.0'

vCloud Usage Meter 3.6.1 – Round Up Post

Updated October 22nd, 2019 – added vCAv 3.x Metering

Note that the below posts are still applicable for the latest version of Usage Meter 3.6.1. Please apply any necessary hot patches post-deployment.

vCloud Usage Meter was released almost two weeks ago and I wanted to provide a summary post for those of you that will be deployed/migrating to 3.6.1.

Release Notes for Usage Meter 3.6.1 

Download Page for 3.6.1

3.6.1 does have several enhancements for vROps and vSAN reporting – so I suggest migrating as soon as possible for accurate reporting.

Remember, continue to run your existing UM instance for at least two (2) months to verify billing and monitoring accuracy (i.e. ensure you aren’t missing a vCenter instance!).

This is a minor release, so my past posts all apply here.

VMware vCloud Usage Insight – Automatic Phone Home Reporting

VMware vCloud Usage Meter and Usage Insight Landing Page

Bundle Translation and Logic

VMware vCloud Usage Meter – BND to Bundle Translation Technical Discussion

Hot Patch 3 for Usage Meter 3.6.1 – vCAv 3.x Metering

VMware vCloud Usage Meter 3.6.1 – Metering of vCloud Availability 3.x and Logic Review

Migrate from existing Usage Meter to 3.6.1

Migrating to vCloud Usage Meter 3.6

End to End vCloud Usage Meter Installation (3 Parts)

Part 1 – Usage Meter Deployment

Part 2 – Usage Meter Deployment

Part 3 – Usage Meter Deployment

Miscellaneous

Automate retrieving the Horizon Usage Report for VMware Cloud Providers

Correcting vROps Reporting in Usage Meter

Handy Commands for Usage Meter 

Happy Deploying!

-Daniel

Updating vROps Instance in vCloud Usage Meter

Updated December 6th, 2018

On new deployments of VMware vCloud Usage Meter, sometimes the wrong vRealize Operations Instance (vROps) is propagated to Usage Meter on initial connection. Today, you cannot change or modify this vROps instance. This is frustrating for new users of Usage Meter.

Why?

Well, Usage Meter relies on the Managed Object Browser (MOB) to correlate to the connected vROps instance. In some cases, a VM Administrator may have a stale or older vROps instance still registered to the vCenter environment. For correct reporting and to remove any errors from Usage Meter, this needs to be resolved. 

How do I resolve this?

Three steps to solving this:

  1. If applicable, remove the vCenter extension/registration from vRealize Operations.
  2. Remove the incorrect vROps instance from the vCenter MOB.
  3. Register the correct vROps instance and synchronize the inventory in Usage Meter.

Before, any of this is done, I suggest snapshotting your vCenter and Usage Meter appliance. Don’t forget to remove the snaps post-completion!

Step 1 – If applicable, remove the vCenter extension/registration from vRealize Operations

  1. If you have an operating vROps instance that needs to be removed/modified, let’s remove this registration from the vCenter(s).
  2. In the vROps UI, navigate to Administration -> Solutions -> Select VMware vSphere -> click Configure (the gear wheel) and click Manage Registrations – 
  3. From there, we need to put in the credentials and click the Unregister button to complete this request. 
  4. Again, if your vROps instance is not available, skip ahead.

Step 2 – Remove the incorrect vROps instance from the vCenter MOB.

  1. Open your browser to “https://<vCenter-FQDN>/mob/?moid=ExtensionManager” – this is the direct link to the Extension Manager section. Type in administrative credentials to log in. 
  2. Verify you see the extensionList[“com.vmware.vcops”] extension in the list. Click on it. We are going to verify that we see the incorrect vROps instance before we remove it. 
  3. From here, click on “server.” We are going to verify that the incorrect vROps instance is showing up in the extension.
  4. Verify you see the incorrect vROps instance. 
  5. OK, press the back button twice and back to the original URL. Now we will unregister the incorrect vROps instance.
  6. Click on the UnregisterExtension Method at the bottom – 
  7. Now you’ll get a popup requesting the extension name that we will unregister. Type in “com.vmware.vcops” in the box and press the Invoke Method button. 
  8. This may take a few seconds to run. However, you will see a void message. Close the popup and refresh the main browser tab that has the MOB information. We should see that the vcops extension has been successfully removed. 

Step 3  – Register the correct vROps instance and synchronize the inventory in Usage Meter.

  1. We are now ready to register the right vROps instance to vCenter.
  2. Log into the vROps web console and navigate to Administration -> Solutions -> select the VMware vSphere name. You might see it collecting, but we need to register the plugin into vCenter. 
  3. Click on the wheel icon right under Solutions to open up the Configuration section. We will now click on Manage Registrations to re-register it to vCenter.
  4. Check the box to “Use collection credentials” and click the Register button. 
  5. This might take a moment to register the plugin inside of vCenter, but a successful message will look like the following – 
  6. Going back to my vCenter MOB tab, I can now see vcops under the ExtensionManager section. 
  7. Almost complete! Hop over to the Usage Meter console and click “Synchronize All vCenter Inventories” button right under the vCenter Server section. Then, click the Rebuild button under vRealize Operations Manager. 
  8. There we go! We now see our vROps instance. Click on Edit to put in the correct credentials. Remember, you need to utilize a LOCAL vROps read-only or administrator account. AD/LDAP accounts do NOT work!
  9. Once you put in the credentials, you will see a message stating the credentials are correct. Complete! On the next hourly run, we should see it fully activated and the “Not yet discovered” message will be removed. 

Complete! Now, you’ll be able to monitor your vROps instance and bill based on the usage.

-Daniel