A Deeper Look into NSX L2VPN with vCD Extender Orchestration

I’ve been rebuilding my vCloud Director (vCD) lab and running through a few connectivity scenarios. Moreover, I wanted to write and share my findings on orchestrating an on-prem NSX environment connecting to a vCD/Provider environment using vCloud Director Extender (VXLAN to VXLAN). In vCD Extender, this is also known as a DC Extension.

To back up, let’s talk about how NSX provides flexible architecture as it relates to Provider scalability and connectivity. My esteemed colleagues did some great papers from our vCloud Architecture Toolkit (vCAT):

Before I get started, I also think this is a good guide for planning out VXLAN <–> VXLAN VPN connectivity.

Let’s look at my lab design from a network / logical perspective –

As you can see above, I have my Acme-Cloud organization available on the left with a single VM in the 172.16.20.x/24 network that’s running on NSX/VXLAN.

On the right, we have “Acme DC” that’s also using NSX and has a logical switch named “Acme-Tier” with the same network subnet.

The orange Extender Deployed L2VPN Client is what’s deployed by vCD Extender on tunnel creation. We’re going to walk through how Extender creates this L2VPN tunnel within an on-prem NSX environment.


  1. This is very similar to my warm migration setup, so I’m going to try not to duplicate material.
  2. I have my Acme-Cloud-Tier Org VDC Network that was converted to a Subinterface inside of vCD: 
  3. We can see in the Edge Services, my L2VPN Server has been set up with a default site configuration. However, vCD Extender creates its own site configuration – 
  4. Extender generates a unique ID for the username and password. This is done when the DC Extension is executed by the tenant. I also established the egress optimization gateway address for local traffic. 

Tenant – vCD Extender Setup

  1. Before we can create a Data Center Extension, we need two required fields for NSX deployments.
  2. First, we need to give the required information to successfully deploy a standalone Edge that will be running our L2VPN client service. This would include the uplink network along with an IP address, gateway, and VMware-specific host/storage information.
  3. Second, we need to provide the required NSX Manager information. I’m sure this is to make the API calls required to deploy the Edge device(s) to the specified vCenter. 
  4. Once the DC Extension has been created, we would see a new Edge device under Networking & Security 

Tenant – DC Extension (L2VPN) Execution

  1. So what happens when we attempt to create a new DC Extension (or L2VPN Connection)? A few things:
    1. Creation of our trunk port for our specified subinterface
    2. Deployment of the new Edge device that will act as the L2VPN Client
    3. Reconfiguration of the trunk port (uses mcxt-tpg-l2vpn-vxlan-** prefix)
    4. Allowing NSX to do its magic along with L2VPN
  2. We can see within my task console what happened – 
  3. Voila, we have a connected L2VPN tunnel. As you can see, the blue “E” stipulates that we have a local egress IP set. I did this since I wanted to route traffic to its local interface for traffic optimization.
  4. So, what happens in the background? Well, let’s take a look at the Edge device. We can see the trunk interface was created while the subinterface is configured to point to my logical switch “Acme-Tier.” 
  5. Last of all, the L2VPN configuration was completed and established as the Client. We can now see the tunnel is alive too. 
  6. From the main vCD Extender page, we can also see traffic utilization over the tunnel. Pretty nice graph! 

Just a quick ping test, yep! WebVM can access WebVM2.

In summary, NSX to NSX DC Extensions within vCD Extender works pretty similar to Provider/VXLAN to On-Prem/VLAN. The key difference is the on-prem vCD Extender has the embedded Standalone Edge to deploy to vCenter.

Enjoy those cloud migrations!


Configure vCloud Director 9.x Org Rights needed for L2 VPN Stretching – vCloud Extender

I am currently working on a new lab environment that consists of vCloud Director (vCD) Extender and vCloud Availability.

Our documentation provides a method for writing out the permissions needed via curl/API commands. I decided to take a shortcut and just use a PowerShell script.

Configuring vCloud Director Organization Rights for L2 VPN Stretching – VMware Documentation

If you don’t know what the heck, I’m talking about, vCloud Director Extender is our plugin product to migrate workloads from an on-prem environment to a vCD environment. Check out my installation guides here:

vCloud Director Extender – Installation Review

vCD Extender – Warm Migration Setup

I *thought* that vCloud Director 9.x would have all of the advanced organization permissions required for L2 stretching. I was incorrect, it’s missing a few things (really actually two, but following our exact documentation).

Per my findings, my vCD environment was missing the following from an Org Admin:

  • Organization vDC Gateway: View L2 VPN
  • Organization vDC Gateway: Configure L2 VPN

So I modified my previous script (see here) to write the required org permissions to establish and set up the L2 VPN / Data Center Connection for vCloud Director Extender.

Before I ran the script, I saw this:

Running the script….

After running the script, I now see the two new L2 VPN options available to my org admin.

Done! Now I can continue on with my L2 setup.

More to come and script is below, thanks!


# vCD Extender Permissions Setup - initially created by KiwiCloud.Ninja - modified by Daniel Paluszek - paluszek.com
# January 17th, 2018 - Modified for a vCloud Director 9.x Instance
# Script to add new OrgRights options for administering advanced Edge Gateway to a vCloud Director organisation.
# Note that Organisation roles (e.g. Organizational Administrator) still need to be edited to add these rights once
# this script has been run against their org.
# NOTE: You must be connected to the vCloud API (Connect-CIServer) with a System administrative user prior to running the script for this to work.
# Add your Org name in line 7 while the vCD instance name is added in line 8
$OrgToUpdate = 'T1'
$APIendpoint = 'vcd-01a.corp.local'

Function vCloud-REST(
[string]$Method = 'Get',
[string]$ApiVersion = '27.0',
[int]$Timeout = 40
$mysessionid = ($global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }).SessionId
$Headers = @{"x-vcloud-authorization" = $mysessionid; "Accept" = 'application/*+xml;version=' + $ApiVersion}
if (!$ContentType) { Remove-Variable ContentType }
if (!$Body) { Remove-Variable Body }
[xml]$response = Invoke-RestMethod -Method $Method -Uri $URI -Headers $headers -Body $Body -ContentType $ContentType -TimeoutSec $Timeout
Write-Host "Exception: " $_.Exception.Message
if ( $_.Exception.ItemName ) { Write-Host "Failed Item: " $_.Exception.ItemName }
Write-Host "Exiting."
return $response
} # Function vCloud-REST End

# The new vCloud Director API v27.0 OrgRights for vCD Extender Preparation and Advanced Networking:
$newrights = @{}
$newrights.Add("Organization vDC Gateway: Convert to Advanced Networking", "9dc33fcb-346d-30e1-8ffa-cf25e05ba801")
$newrights.Add("Organization vDC Gateway: View L2 VPN", "105191de-9e29-3495-a917-05fcb5ec1ad0")
$newrights.Add("Organization vDC Gateway: Configure L2 VPN", "eeb2b2a0-33a1-36d4-a121-6547ad992d59")
$newrights.Add("Organization vDC Gateway: Configure Firewall", "b755b050-772e-3c9c-9197-111c286f563d")
$newrights.Add("Organization vDC Network: Edit Properties", "b0cfe989-521b-3d7f-9bc2-f23c74a99633")
$newrights.Add("Organization vDC Network: View Properties", "2c8d98ef-4acc-3be4-9214-fcb9682b7a19")

$myendpoint = $global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }

if (!$myendpoint.IsConnected) {
Write-Host "Not connected to this vCloud endpoint, use 'Connect-CIServer' before running this script."

$org = Get-Org -Name $OrgToUpdate -Server $APIendpoint

if (!$org) {
Write-Host "Couldn't match organization with name $OrgToUpdate, exiting."

$rightsuri = 'https://' + $APIendpoint + "/api/admin/org/" + $org.Id.Substring($org.Id.LastIndexOf(':')+1) + "/rights"

[xml]$rights = vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Method 'Get' -ApiVersion '27.0'

# Add the new API v27 'RightsReference' elements to the XML returned:
foreach($newrule in $newrights.Keys) {
$newright = $rights.CreateElement("RightReference", "http://www.vmware.com/vcloud/v1.5")

# Update the Organization with the ammended rights:
vCloud-REST -URI $rightsuri -ContentType 'application/vnd.vmware.admin.org.rights+xml' -Body $rights.InnerXml -Method 'Put' -ApiVersion '27.0'

Update VMware vCloud Director 9 in 5 Minutes

I sometimes hear that vCloud Director (vCD) is cumbersome and hard to install/upgrade. I just updated my lab environment vCD instance from 9.0 to 9.0.1 in about five minutes. The team has done an incredible job of making any patch/upgrade process seamless. Below are the steps for upgrading while documentation link is below.

Upgrade VMware vCloud Director Installation – Documentation

We can see I’m running the 9.0 base version. As stated above, we will upgrade to 9.0.1.

Download and check integrity

  1. Go to vmware.com and download the latest upgrade package 
  2. Upload via SCP (or your method of transfer) to your vCD cell 
  3. Now let’s SSH to our vCD cell and check the md5sum to ensure nothing happened from downloading it to copying it. Yes, md5 matches and we are good! 
  4. Let’s go ahead and make the bin file executable. “chmod u+x <filename>” makes it easy and now we can see it’s executable. 

Run the Installer

  1. We are now ready to run the vCD installer bin file. Let’s go ahead and dot slash it! 
  2. The installer will verify you have an upgradeable version of vCloud Director and ensure you want to proceed with the upgrade. Type “y” to continue on. 
  3. Now we can see the installer is stopping the vCD cell processes and continuing on with the installation.
  4. And we are complete! The last message we see is we need to perform the database upgrade to ensure the schema is up to date. 

Update the vCloud Director Database

  1. Okay, let’s go ahead and run “/opt/vmware/vcloud-director/bin/upgrade” – we see a message ensuring you want to continue with the upgrade. You’ll want to make sure all of your other cells are stopped to ensure there’s no database connections. 
  2. The upgrade will look at the database and ensure everything is acceptable. Again, you’ll one more time, just to make sure you have that backup! 🙂 
  3. we can see the upgrade task was completed and everything looks great. We are prompted to start the vCD services back up. Type y to continue on. 
  4. Services are started! 

Let’s log in and check the build version. There we go, on the latest version of vCloud Director 9.0!

That’s it. Very easy and straightforward. If you’re interested in further insight on architecting vCloud Director, do check out our free vCloud Architecture Toolkit papers:



My Deployment Experience with Ubiquiti Networks

I was the fortunate winner of the vBrownBag year-end contest and won Ubiquiti Networks gear. I wasn’t sure what I was going to get, but wow, was I surprised!

I’ve been wanting to pick up Ubiquiti gear for some time now – I was running an older Meraki deployment from my Cisco days, which has served me well. However, the MR18 was definitely showing its age from a channel utilization perspective.

I *finally* got a chance to sit down yesterday and start deploying out a simple design for now while also learning about Ubiquiti Networks gear.

I also jumped into this without reading much documentation and quite frankly, understanding of how the Ubiquiti model works. I wanted to see how easy it was to deploy based on my past network experience.

From a top-level topology perspective, I decided to lay out the deployment as such, pretty straight-forward.

Deployment Steps

  1. I first created an account on Ubnt.com, fairly straightforward and showed a demo controller. 
  2. I was a little confused on which step I should take next – do I need to set up the USG first or the Cloud Key? Well, I kind of did both, which was fine.
  3. I plugged in the USG and allowed the default DHCP settings while just setting up a hardwired connection from my laptop. 
  4. From here, I was able to start a setup wizard on the USG. Very straightforward while setting some initial defaults.
  5. I did initially set it up with a local Controller (not the Cloud Key) but was able to move over the USG pretty seamlessly.  
  6. From there, I started adding in the AP and the Switch. Adoption was easy, just a click of the button.
  7. I upgraded to the latest code and voila, complete with my initial basic setup!

Experience and Testing so far

  1. The deployment was very easy in my opinion and the usability is even easier, maybe even too easy? You can tell Ubiquiti spent a lot of time on the visuals of the UniFi dashboard along with what typical administrators would be configuring.
  2. I think the hardest part was the ramifications of the SSID channel and key change – had to reprogram quite a bit of devices!
  3. I’ve had wireless congestion issues in the past on 11B/G spectrum, so it was great to see some of their insights/stats on congestion. Again, changing the radio channel was very easy. 

Now, for my very scientific (sarcasm) test – before and after wireless speeds.

From my office where all of my gear is, I did a before and after throughput test on my iPhone.

While it’s not an apples to apples comparison as it relates to the gear (I realize the Ubiquiti gear is much newer), I’ve improved my downstream throughput by 2x which is outstanding.

I’m very satisfied with the quality of the Ubiquiti Networks gear and I can tell how it’s caught on in the industry. I plan on adding another 8 port switch and also another AP down the road. I’ll be also carving up VLANs and different networks based on use case. I’m interested to see what else can be done from the USG, looks like I can even configure terminal access.