vCloud Availability for vCloud Director – Tenant Walkthrough Video

I recently created a vCloud Availability demonstration video and wanted to share this out with others.

vCloud Availability (vCAv) for vCloud Director is a very powerful solution that provides Disaster Recovery as a Service (DRaaS) that’s built on top of vCloud Director. What’s great about vCAv is the ability to protect, migrate, and failover workloads from a tenant environment just from vCenter.

vCAv utilizes vSphere Replication as its replication engine while our Cloud Provider Business Unit built the architecture around vCD to provide scalable multi-tenancy. Granular selection of VM’s is possible that correlate to a DR-enabled VDC.

For VMware Cloud Providers that are interested in further details, here are some good links to start on vCAv:

Well, here’s the video. Enjoy!


VMware Cloud Provider – vSAN Best Practices

I’ve been getting a few requests recently on the best practices for vSAN inside of the Cloud Provider Program (VCPP). I’d like to spend some time on how VSAN works and on what works well within the consumption program.

First things first, how is vSAN licensed inside of VCPP?

vSAN consumes points just like any other offering inside of VCPP. However, compared to our bundles, vSAN is measured on Used Capacity on disk.

There are two base options of vSAN inside of VCPP:

  1. vSAN Standard
  2. vSAN Advanced

There are add-on packages for additional vSAN functionality. Essentially, this is added to the base option to provide additional feature sets. Two add-ons are available:

  1. Enterprise Add-On for the vSAN Standard base option
  2. Enterprise Add-On for the vSAN Advanced base option

This is all summarized in the Product Usage Guide (PUG as I call it) on page 45:

To summarize:

  1. vSAN Standard is 0.04 points/GB
  2. If I use vSAN Standard with the Enterprise Add-On: 0.06 points/GB (.02 uplift)
  3. vSAN Advanced is 0.06 points/GB
  4. If I use vSAN Advanced with the Enterprise Add-On: 0.08 points/GB (.02 uplift)

Now a single VCPP is equivalent to a specific cost, that’s all subject to your agreement. Moreover, Cloud Providers with higher commitments get additional discounts. This is on top of the current vSAN Promotion going on right now too. However, you can see how this can be attractive when we talk about consumption capacity and modeling.

vSAN Automatic Detection inside of vCloud Usage Meter

One of the great things about Usage Meter is it provides automatic detection of vSAN and will report on monthly consumption. This can be broken down to a specific tenant or customer based on your Usage Meter rule set.

There’s no additional configuration necessary to setup vSAN for Usage Meter reporting. Detected vSAN clusters will appear in the Cluster History report (depicted below).

Moreover, there are four features that are automatically detected based on the Usage Meter detection:

  1. Deduplication
  2. Erasure Coding (use of RAID5/6)
  3. Stretched Cluster
  4. IOPS Limit inside of the vSAN Storage Policy

There are a few things that are critical to accurate vSAN usage inside of VCPP:

  1. Ensure you are using the appropriate vSAN license key within the VCPP model. If you intend to use vSAN Advanced, use a vSAN Advanced key. Here’s why –
    1. Usage Meter will detect what license version of vSAN is being used. If an Enterprise key is detected, and either Dedupe/Erasure Coding features are being used, vSAN Advanced + Enterprise Add-On will be charged.
    2. This is important since if you are only intending to utilize vSAN Advanced (which includes Deduplication and Erasure Coding), you’ll want to make sure you pay for the appropriate amount.
  2. Stretch Cluster and Deduplication is applied at the cluster level. Hence, all workloads that reside on that vSAN datastore will be charged accordingly, even if you have workloads that may be tied to a PFTT=0 policy.
  3. Moreover, even individual features like IOPS limit and Erasure Coding (all within the SPBM) are enabled on a per-VM basis but scoped at the cluster level. If one or more VM’s are using these features, the feature is considered to be enabled for the entire cluster.

The below table showcases how vSAN is detected and the importance of using the accurate license.

vSAN Best Practices in VCPP

Here are a few things I’ve been messaging to my Providers regarding vSAN consumption inside of VCPP.

  1. vSAN is based on actual capacity used on disk. This is inclusive of the Fault Tolerance method used along with VM Memory Swap! Therefore, a FTT=2/RAID1 is going to consume a significant amount more storage than a FTT=1/RAID5EC policy.
    1. The chart that depicts storage capacity required for each FT method:
  2. VMs used the default Storage Policy are thin-provisioned while the VM Memory Swap is thick provisioned. Keep in mind when discussing this with eager tenants that like to over-provision their storage. vSAN will only consume the storage that has been written to disk.
  3. As for the VM Memory Swap, the Storage Policy does not impact the default setting. By default, VM Memory Swap is striped (or FTT=1) to another host. This can be overridden on a host basis by using the advanced setting of “esxcfg-advcfg -s 1 /VSAN/SwapThickProvisionDisabled”
    1. Review Duncan’s blog article more here:
    2. Update 10/23/2018 – Note that with vSAN 6.7, sparse swap is enabled by DEFAULT. Therefore, saves on the overall capacity utilization! 
  4. Deduplication and Compression can HELP from a cost perspective. Here’s why – Usage Meter will only detect what’s written to disk from the vSAN Cluster view. In the below example, I’ve written 101.72GB but with these efficiencies, I’ve written only 47.83GB to disk. From the VCPP program, I’d be only charged for about 47GB in space.
    1. ProTip: This is not a broad recommendation to use Deduplication and Compression just to save costs. Keep in mind the efficiencies only apply per disk group. Know your workload and I/O profile. This also applies to FTT policies – know your workload!

VCPP Sample Sizing for vSAN

So let’s go through some sample sizing on what an actual VM would be charged on a monthly basis.

Sample VM with FTT=1/RAID1 Policy

  • Sample VM: 2 vCPU, 4GB vRAM, 500GB HDD
  • SPBM – Standard Policy, FTT=1/RAID-1
  • Size Calculation
    • 500GB * 2 (FTT=1/RAID1) = 1000GB
    • 4GB Memory * 2 (Always mirrored) = 8GB
      • If you are using vSAN 6.7, you do not need to add this. 
    • 1000GB + 8GB = 1008GB in total storage required
  • VCPP Cost Calculation – let’s say we are using the vSAN Advanced license which is .06 points per GB.
    • 1008GB * .06 points = 60.48 points per month
    • Let’s say my cost for a point was $1USD, this workload would cost me about $60 dollars a month.

Now let’s calculate this with an FTT=1/RAID5EC Policy. Keep in mind, all design rules apply still: still need a minimum of 4 hosts, dedupe/comp requires All Flash, etc.

Sample VM with FTT=1/RAID5/EC Policy

  • Sample VM: 2 vCPU, 4GB vRAM, 500GB HDD
  • SPBM – Standard Policy, FTT=1/RAID-5 Erasure Coding
  • Size Calculation
    • 500GB * 1.33 (FTT=1/RAID5EC) = 665GB
    • 4GB Memory * 2 (Always mirrored) = 8GB
      • If you are using vSAN 6.7, you do not need to add this.
    • 665GB + 8GB = 673GB in total storage required
  • VCPP Cost Calculation – let’s say we are using the vSAN Advanced license which is .06 points per GB.
    • 673GB * .06 points = 40.38 points per month
    • Let’s say my cost for a point was $1USD, this workload would cost me about $40 dollars a month. This is about a 32% reduction in savings compared to a RAID-1 policy. 

Now let’s say I applied a 1:5:1 deduplication and compression ratio to this sample VM. This is a conservative estimate since our standard vSAN calculator utilizes a 2:1 ratio.

Sample VM with FTT=1/RAID5/EC Policy with Deduplication/Compression Ratio of 1.5:1

  • Sample VM: 2 vCPU, 4GB vRAM, 500GB HDD
  • SPBM – Standard Policy, FTT=1/RAID-5 Erasure Coding
  • Size Calculation
    • 500GB  / 1.5 (dedupe/comp ratio) *1.33 (FTT=1/RAID5EC) = 443GB
    • 4GB Memory * 2 (Always mirrored) = 8GB
      • If you are using vSAN 6.7, you do not need to add this.
    • 443GB + 8GB = 451GB in total storage required
  • VCPP Cost Calculation – let’s say we are using the vSAN Advanced license which is .06 points per GB.
    • 451GB * .06 points = 27.08 points per month
    • Let’s say my cost for a point was $1USD, this workload would cost me about $27 dollars a month. This is about a 55% reduction in savings compared to the original model and 33% savings compared to the RAID5EC policy. 


  1. As you can see, vSAN consumption can be extremely granular from a cost perspective and will vary based on the protection scheme.
  2. Using the efficiency capabilities of vSAN along with Erasure Coding does save capacity, but remember what your workload requires!
  3. Usage Meter auto detects usage and will roll it up in the monthly reports – no additional configuration needed.
  4. As for pricing, there is a current promotion going on for vSAN inside of the Cloud Provider Program – reach out to your aggregator or VMware Business Development Manager for further information. This makes the overall vSAN pricing even more attractive on a consumption basis.



A Deeper Look into NSX L2VPN with vCD Extender Orchestration

I’ve been rebuilding my vCloud Director (vCD) lab and running through a few connectivity scenarios. Moreover, I wanted to write and share my findings on orchestrating an on-prem NSX environment connecting to a vCD/Provider environment using vCloud Director Extender (VXLAN to VXLAN). In vCD Extender, this is also known as a DC Extension.

To back up, let’s talk about how NSX provides flexible architecture as it relates to Provider scalability and connectivity. My esteemed colleagues did some great papers from our vCloud Architecture Toolkit (vCAT):

Before I get started, I also think this is a good guide for planning out VXLAN <–> VXLAN VPN connectivity.

Let’s look at my lab design from a network / logical perspective –

As you can see above, I have my Acme-Cloud organization available on the left with a single VM in the 172.16.20.x/24 network that’s running on NSX/VXLAN.

On the right, we have “Acme DC” that’s also using NSX and has a logical switch named “Acme-Tier” with the same network subnet.

The orange Extender Deployed L2VPN Client is what’s deployed by vCD Extender on tunnel creation. We’re going to walk through how Extender creates this L2VPN tunnel within an on-prem NSX environment.


  1. This is very similar to my warm migration setup, so I’m going to try not to duplicate material.
  2. I have my Acme-Cloud-Tier Org VDC Network that was converted to a Subinterface inside of vCD: 
  3. We can see in the Edge Services, my L2VPN Server has been set up with a default site configuration. However, vCD Extender creates its own site configuration – 
  4. Extender generates a unique ID for the username and password. This is done when the DC Extension is executed by the tenant. I also established the egress optimization gateway address for local traffic. 

Tenant – vCD Extender Setup

  1. Before we can create a Data Center Extension, we need two required fields for NSX deployments.
  2. First, we need to give the required information to successfully deploy a standalone Edge that will be running our L2VPN client service. This would include the uplink network along with an IP address, gateway, and VMware-specific host/storage information.
  3. Second, we need to provide the required NSX Manager information. I’m sure this is to make the API calls required to deploy the Edge device(s) to the specified vCenter. 
  4. Once the DC Extension has been created, we would see a new Edge device under Networking & Security 

Tenant – DC Extension (L2VPN) Execution

  1. So what happens when we attempt to create a new DC Extension (or L2VPN Connection)? A few things:
    1. Creation of our trunk port for our specified subinterface
    2. Deployment of the new Edge device that will act as the L2VPN Client
    3. Reconfiguration of the trunk port (uses mcxt-tpg-l2vpn-vxlan-** prefix)
    4. Allowing NSX to do its magic along with L2VPN
  2. We can see within my task console what happened – 
  3. Voila, we have a connected L2VPN tunnel. As you can see, the blue “E” stipulates that we have a local egress IP set. I did this since I wanted to route traffic to its local interface for traffic optimization.
  4. So, what happens in the background? Well, let’s take a look at the Edge device. We can see the trunk interface was created while the subinterface is configured to point to my logical switch “Acme-Tier.” 
  5. Last of all, the L2VPN configuration was completed and established as the Client. We can now see the tunnel is alive too. 
  6. From the main vCD Extender page, we can also see traffic utilization over the tunnel. Pretty nice graph! 

Just a quick ping test, yep! WebVM can access WebVM2.

In summary, NSX to NSX DC Extensions within vCD Extender works pretty similar to Provider/VXLAN to On-Prem/VLAN. The key difference is the on-prem vCD Extender has the embedded Standalone Edge to deploy to vCenter.

Enjoy those cloud migrations!


Configure vCloud Director 9.x Org Rights needed for L2 VPN Stretching – vCloud Extender

NOTE – this is for vCD Extender 1.0 ONLY. If you need a permissions script for vCD Extender 1.1, please check it out here. 

I am currently working on a new lab environment that consists of vCloud Director (vCD) Extender and vCloud Availability.

Our documentation provides a method for writing out the permissions needed via curl/API commands. I decided to take a shortcut and just use a PowerShell script.

Configuring vCloud Director Organization Rights for L2 VPN Stretching – VMware Documentation

If you don’t know what the heck, I’m talking about, vCloud Director Extender is our plugin product to migrate workloads from an on-prem environment to a vCD environment. Check out my installation guides here:

vCloud Director Extender – Installation Review

vCD Extender – Warm Migration Setup

I *thought* that vCloud Director 9.x would have all of the advanced organization permissions required for L2 stretching. I was incorrect, it’s missing a few things (really actually two, but following our exact documentation).

Per my findings, my vCD environment was missing the following from an Org Admin:

  • Organization vDC Gateway: View L2 VPN
  • Organization vDC Gateway: Configure L2 VPN

So I modified my previous script (see here) to write the required org permissions to establish and set up the L2 VPN / Data Center Connection for vCloud Director Extender.

Before I ran the script, I saw this:

Running the script….

After running the script, I now see the two new L2 VPN options available to my org admin.

Done! Now I can continue on with my L2 setup.

More to come and script is below, thanks!


# vCD Extender Permissions Setup - initially created by KiwiCloud.Ninja - modified by Daniel Paluszek -
# January 17th, 2018 - Modified for a vCloud Director 9.x Instance
# Script to add new OrgRights options for administering advanced Edge Gateway to a vCloud Director organisation.
# Note that Organisation roles (e.g. Organizational Administrator) still need to be edited to add these rights once
# this script has been run against their org.
# NOTE: You must be connected to the vCloud API (Connect-CIServer) with a System administrative user prior to running the script for this to work.
# Add your Org name in line 7 while the vCD instance name is added in line 8
$OrgToUpdate = 'T1'
$APIendpoint = 'vcd-01a.corp.local'

Function vCloud-REST(
[string]$Method = 'Get',
[string]$ApiVersion = '27.0',
[int]$Timeout = 40
$mysessionid = ($global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }).SessionId
$Headers = @{"x-vcloud-authorization" = $mysessionid; "Accept" = 'application/*+xml;version=' + $ApiVersion}
if (!$ContentType) { Remove-Variable ContentType }
if (!$Body) { Remove-Variable Body }
[xml]$response = Invoke-RestMethod -Method $Method -Uri $URI -Headers $headers -Body $Body -ContentType $ContentType -TimeoutSec $Timeout
Write-Host "Exception: " $_.Exception.Message
if ( $_.Exception.ItemName ) { Write-Host "Failed Item: " $_.Exception.ItemName }
Write-Host "Exiting."
return $response
} # Function vCloud-REST End

# The new vCloud Director API v27.0 OrgRights for vCD Extender Preparation and Advanced Networking:
$newrights = @{}
$newrights.Add("Organization vDC Gateway: Convert to Advanced Networking", "9dc33fcb-346d-30e1-8ffa-cf25e05ba801")
$newrights.Add("Organization vDC Gateway: View L2 VPN", "105191de-9e29-3495-a917-05fcb5ec1ad0")
$newrights.Add("Organization vDC Gateway: Configure L2 VPN", "eeb2b2a0-33a1-36d4-a121-6547ad992d59")
$newrights.Add("Organization vDC Gateway: Configure Firewall", "b755b050-772e-3c9c-9197-111c286f563d")
$newrights.Add("Organization vDC Network: Edit Properties", "b0cfe989-521b-3d7f-9bc2-f23c74a99633")
$newrights.Add("Organization vDC Network: View Properties", "2c8d98ef-4acc-3be4-9214-fcb9682b7a19")

$myendpoint = $global:DefaultCIServers | Where { $_.Name -eq $APIendpoint }

if (!$myendpoint.IsConnected) {
Write-Host "Not connected to this vCloud endpoint, use 'Connect-CIServer' before running this script."

$org = Get-Org -Name $OrgToUpdate -Server $APIendpoint

if (!$org) {
Write-Host "Couldn't match organization with name $OrgToUpdate, exiting."

$rightsuri = 'https://' + $APIendpoint + "/api/admin/org/" + $org.Id.Substring($org.Id.LastIndexOf(':')+1) + "/rights"

[xml]$rights = vCloud-REST -URI $rightsuri -ContentType 'application/' -Method 'Get' -ApiVersion '27.0'

# Add the new API v27 'RightsReference' elements to the XML returned:
foreach($newrule in $newrights.Keys) {
$newright = $rights.CreateElement("RightReference", "")

# Update the Organization with the ammended rights:
vCloud-REST -URI $rightsuri -ContentType 'application/' -Body $rights.InnerXml -Method 'Put' -ApiVersion '27.0'