How is ECMP metered when configured in the Provider architecture within the VMware Cloud Provider Program?

Recently, I received a request from one of our aggregators regarding how Equal Cost Multipathing (ECMP) is metered within the VMware Cloud Provider Program (VCPP), specifically Tom Fojta’s recommendation on architecting Provider-managed NSX Edges and Distributed Logical Router (DLR) in ECMP mode, specifically this diagram from the Architecting a VMware vCloud Director Solution – 

As shown in the diagram above – How does Usage Meter handle bill these tenant virtual machines (VMs) when we have a provider NSX architecture that utilizes ECMP? 

For you TL;DR readers – any VM connected to a Tenant Edge / direct network that has ECMP enabled northbound, NSX Advanced will be charged for said VM. Read on if you want to learn how this is done. 

First off, let’s talk about why this matters. Per the Usage Meter Product Detection whitepaper (this can be found on VMware Partner Central), we can see how Usage Meter detects specific NSX features based on the pattern of usage. Regarding dynamic ECMP, it is metered by the “Edge gateway” which could be a little ambiguous. If one utilizes ECMP, they would be metered for NSX Advanced within VCPP. 

One of the scenarios from the whitepaper does show ECMP-enabled Edges but not an Edge that is abstracted away from the provider environment – 

My initial reaction was that Usage Meter would not look at the northbound provider configuration and the interconnectivity to vCloud Director. However, I was not confident and wanted to verify this explicit configuration and expected metering. Let’s review my findings. 

Lab Setup

In the above diagram, we can see I created a similar Provider managed NSX configuration with ECMP enabled from the DLR to the two Provider Edges with dynamic routing enabled (BGP). From there, I expose a LIF/Logical Switch named “ECMP-External-Network” to vCloud Director that is then exposed to my T2 organization as a new External Network.

From there, I created a dedicated Tenant Edge named “T2-ECMP-ESG” that will be attached to this newly created network along with a VM named “T2-ECMP-VM.” The goal is to verify how T2-ECMP-VM and T2-TestVM are metered by Usage Meter with this newly created Tenant Edge. 

Lab Configuration

My Edges are setup for BGP and reporting the correct routes from the southbound connected DLR (and tenant Edges) – 

From the DLR, we can see that I have two active paths to my Provider Edges (Provider-Edge-1 and 2) – 

Last of all, my T2-ECMP-ESG is operational and attached to the newly created ECMP External Network – 

Last of all, I have my VM’s created and powered on (remember, Usage Meter will only meter powered on VM’s). We can see T2-ECMP-VM is attached to a org routed network from T2-ECMP-ESG named “T2-ECMP-Network” – 

Findings

Let’s work from the north to south – start with the Provider Edges and show how Usage Meter detects and bills. 

Note – I have vROps Enterprise in my lab environment, so we will see Usage Meter picking up vROps registration and placing it in the appropriate bundle.

Provider Edges / DLR

As expected, the Provider Edges and DLR are detected along with registration to vROps. By design, NSX Edges are not charged for NSX usage as they are metered as a management component (minimum Advanced bundle / 7-point). However, in my case, we see detection, and then registration to vROps Enterprise. Therefore, since it’s a bundle ID (BND) of 12, this is correlated to Advanced Bundle with Management (10-point) – 


Tenant Edge – T2-ECMP-ESG

Just like the Provider Edges and DLR, we see T2-ECMP-ESG register to UM along with vROps Enterprise registration. Same billing model as above. 

Tenant VM – T2-TestVM

I would not expect any change to this VM, but wanted to showcase that having a separate Edge with standard networking (i.e. no ECMP) will bill based off the NSX SP Base level. As expected, T2-TestVM was handled by Usage Meter just as anticipated – we can see registration, NSX SP Base usage, along with registration to vROps Enterprise – 

Tenant VM – T2-ECMP-VM

Finally, let’s take a look at my T2-ECMP-VM – as discussed before, this is wired to a Tenant Edge that is connected to the ECMP-enabled DLR via an External Network. 

We see initial registration, registration to vROps Enterprise, then NSX Advanced usage! This would be metered at Advanced Bundle with Networking and Management due to the NSX Advanced usage (12-point). 

Summary of Findings

Here’s what we learned:

  1. Edges/DLR Control VM’s are not charged for NSX usage since UM handles them as a management component. If you are using vROps, it will place it in the most cost effective bundle.
  2. Utilizing ECMP at the provider-level DOES impact any southbound connected VM from a billing perspective, even if an Edge sits in between the ECMP enabled configuration and the tenant VM. Per the findings, NSX Advanced will be metered. 
  3. Therefore, be aware of any NSX provider architecture and the use of NSX specific features. 

Again, this shows the logic inside of Usage Meter and how it relates to metering for tenant workloads. Cheers!

-Daniel

Current UI Capabilities of VMware vCloud Director 9.5 in MindNode

As many of you may be aware, the conversion to HTML5 UI for VMware vCloud Director is a phased approach. With the release of vCD 9.5, this introduced additional H5 functionality for both the tenant and provider.

I’ve had prior requests on the “parity” of UI capabilities between the new H5 UI and the Flex UI. Well, I sat down with the UI team and created two Mind Maps that show the current state of the H5 UI inside of vCD 9.5.

Before I show these images, I’d like to discuss the process of UI development. This is not an easy task and takes a significant amount of time to develop. Why? Well, it’s not just about “converting” a function from vCD Flex to H5. What about re-thinking “how” operations are done in vCloud Director while focusing on making things easier for net new consumers? These decisions are not taken lightly and many internal employees, partners, and providers provide time and insight into future UI development decisions.

Next, I decided to build a legend that reviews the following:

  1. What’s available that was available in the Flex UI in some capacity – these are the blue lines
  2. What’s new to H5 UI that was not available in Flex – these are green
  3. Last, what’s still to come in a future version of vCD – these are gold

What I learned from this exercise is the amount of net-new items we see in the H5 UI that were not available (due to a new feature or an approach). This is quite exciting as they expose a lot of the recent work from our vCD development team.

Let’s start off with the Tenant UI capability – click on it to view the full graphic:

As I’ve stated in the past, the Tenant UI is “mostly” finished with a few last additions. What’s great about this view is the amount of net-new functionality – multi-site dashboard, browsing by VM’s, Role creation, etc.

As for the Provider UI, this is the next focus. We can see more “gold” here, especially under the System Settings.

Again, a lot of new capabilities like vRO integration, Access Control, and so forth.

I’d like to thank my peers and colleagues for the validation of these maps and hope this showcases the current UI capabilities inside of vCD 9.5. Below are the transparent PNGs if you need them. Thanks!

-Daniel

VMware vCloud Director Rights Correlation to VCPP NSX Bundles

I was recently asked by a colleague if we have any existing collateral on VMware vCloud Director (vCD) that maps to the VMware Cloud Provider Program (VCPP) NSX levels that are currently available to partners. Well, there wasn’t, until now. 🙂

First, let’s talk about the NSX bundles inside of VCPP –

There are three levels identified within VCPP:

  1. NSX-SP Base – this is is your fundamental level of NSX. It does include your normal Edge services, Edge Firewall, NAT, Load Balancing, Dynamic/Static Routing, IPSEC/SSL VPN+, and Distributed Routing and Switching. This is typically referred to as “vCNS” mode (callout to the vCD old days) but does use NSX.
  2. NSX-SP Advanced – this includes Base, plus ECMP and Distributed Firewall functionality. Service Insertion, AD Integrated Firewall, etc. are all functions that the Provider can consume from the backend management.
  3. NSX-SP Enterprise – this includes Advanced along with HW VTEP integration, cross-vCenter NSX functionality, along with the L2VPN (Remote Gateway) solution. The new addition here is vCD 9.5 Multi-Site and Cross-VDC capability.

To state the obvious – vCloud Usage Meter will take care of automatic metering based off of what NSX functionality is used, this has been available since version 3.6. Check out this post that discusses how Usage Meter detects NSX (and vROPs) usage.

Last of all, the “Convert to Advanced Gateway” inside of vCloud Director for organization Edges DOES NOT mean you will be using NSX Advanced right away! This is just a change in how vCD presents the Edge UI (with Advanced, it’ll show the H5) along with the API rights available. I demonstrate this in the above post too.

So let’s talk about the NSX levels and how they can pertain to vCD rights and role. I worked up the following roles in my vCD environment:

  1. NSX SP-Base Rights
  2. NSX SP-Advanced Rights
  3. NSX SP-Enterprise Rights

So what does one gain when using these rights? Well, they are now aligned to the VCPP NSX bundles and can be utilized as a starting to monetize NSX inside of a vCD environment.

Now, in my experience, these specific vCD permissions will apply to the VCPP NSX levels as stated above. The big thing I found in my testing is ECMP can be toggled with Static Routing, so I set this as “View Only” for any routing capability for SP-Base.

If you are using vCD 9.5, one could also create a rights bundle that is published to an organization along with utilizing Global Roles to make this *much* easier.

The steps for this would be: Creation of Rights Bundle(s) -> Publish to org(s) -> Creation of Global Roles -> Publish to org(s) -> Apply role to user(s)

Alright, here are the three exports for these rights required. This is not comprehensive for all vCloud permissions required, but gives you an idea of what to append to your existing role (which could be easily done via REST API). Note that the exports below show my vCD instance (vcd-01a.corp.local) along with the org UUID, so replace this if you are doing a POST.

NSX SP-Base:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Role xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="NSX SP-Base Rights" id="urn:vcloud:role:0fbafcf0-1cce-4457-9f1d-d7bbacc189fd" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/0fbafcf0-1cce-4457-9f1d-d7bbacc189fd" type="application/vnd.vmware.admin.role+xml">
<Link rel="edit" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/0fbafcf0-1cce-4457-9f1d-d7bbacc189fd" type="application/vnd.vmware.admin.role+xml"/>
<Link rel="remove" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/0fbafcf0-1cce-4457-9f1d-d7bbacc189fd"/>
<Description></Description>
<RightReferences>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9dc33fcb-346d-30e1-8ffa-cf25e05ba801" name="Organization vDC Gateway: Convert to Advanced Networking" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2a097e48-f4c4-3714-8b24-552b2d573754" name="Organization vDC Gateway: View Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d9dabcab-579e-33c5-807b-dc9232bf7eff" name="Organization vDC Gateway: View BGP Routing" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/209cde55-55db-33f1-8357-b27bba6898ed" name="Organization vDC Gateway: Configure IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/c9e19573-3d54-3d4a-98f2-f56e446a8ef9" name="Organization vDC Gateway: Configure NAT" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/84ddb40f-a49a-35e1-918e-3f11507825d7" name="Organization vDC Gateway: Configure Syslog" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/92b7d500-6bb6-3176-b9eb-d1fda4ce444d" name="Organization vDC Gateway: Configure SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/72c5e652-c8d7-3f19-ab83-283d30cb679f" name="Organization vDC Gateway: Configure Remote Access" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/a5f5fc99-9afc-347b-9a31-f65f61f4416b" name="Organization vDC Gateway: Distributed Routing" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d1c77fc0-a4b9-3d99-bd4b-d7fab35e4fae" name="Organization vDC Gateway: View" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/65439584-6aad-3c2c-916f-794099ee85bf" name="Organization vDC Gateway: View Remote Access" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/cdb0edb0-9623-30a8-89de-b133db7cfeab" name="Organization vDC Gateway: View SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/27be9828-4ce4-353e-8f68-5cd69260d94c" name="Organization vDC Gateway: Configure Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/82beb471-ab7f-3e2b-a615-136ba6645525" name="Organization vDC Gateway: View IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9740be24-4dd7-373c-9237-91896338c11e" name="Organization vDC Gateway: View Static Routing" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/fb860afe-2e15-3ca9-96d8-4435d1447732" name="Organization vDC Gateway: View NAT" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/ff3fc70f-fd25-3c0a-9d90-e7ff82456be5" name="Organization vDC Gateway: Configure System Logging" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/7fee6646-ec0c-34c9-9585-aff6f4d92473" name="Organization vDC Gateway: View Firewall" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/eb525145-08e5-3934-91ef-ec80837c9177" name="Organization vDC Gateway: View OSPF Routing" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/8e16d30d-1ae3-3fff-8d4b-64c342b186a9" name="Organization vDC Gateway: View DHCP" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/b755b050-772e-3c9c-9197-111c286f563d" name="Organization vDC Gateway: Configure Firewall" type="application/vnd.vmware.admin.right+xml"/>
<RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/be1abe9a-7ddc-38f6-bdf3-94affb01e46b" name="Organization vDC Gateway: Configure DHCP" type="application/vnd.vmware.admin.right+xml"/>
    </RightReferences>
</Role>

NSX SP-Advanced:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Role xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="NSX SP-Advanced Rights" id="urn:vcloud:role:1cde673c-3573-4e3a-a520-d03a83caef8d" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/1cde673c-3573-4e3a-a520-d03a83caef8d" type="application/vnd.vmware.admin.role+xml">
    <Link rel="edit" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/1cde673c-3573-4e3a-a520-d03a83caef8d" type="application/vnd.vmware.admin.role+xml"/>
    <Link rel="remove" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/1cde673c-3573-4e3a-a520-d03a83caef8d"/>
    <Description></Description>
    <RightReferences>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2cd03d47-38e1-337a-907c-8d5b6a5258f2" name="Organization vDC Distributed Firewall: Configure Rules" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/c9e19573-3d54-3d4a-98f2-f56e446a8ef9" name="Organization vDC Gateway: Configure NAT" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d1c77fc0-a4b9-3d99-bd4b-d7fab35e4fae" name="Organization vDC Gateway: View" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/b755b050-772e-3c9c-9197-111c286f563d" name="Organization vDC Gateway: Configure Firewall" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/fb860afe-2e15-3ca9-96d8-4435d1447732" name="Organization vDC Gateway: View NAT" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/4e61b5b8-0964-36b6-b021-da39aea724fc" name="Organization vDC Distributed Firewall: View Rules" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/209cde55-55db-33f1-8357-b27bba6898ed" name="Organization vDC Gateway: Configure IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/82beb471-ab7f-3e2b-a615-136ba6645525" name="Organization vDC Gateway: View IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9dc33fcb-346d-30e1-8ffa-cf25e05ba801" name="Organization vDC Gateway: Convert to Advanced Networking" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/3b337aef-42a8-3ed1-8616-341152bc5790" name="Organization vDC Gateway: Configure OSPF Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/8e16d30d-1ae3-3fff-8d4b-64c342b186a9" name="Organization vDC Gateway: View DHCP" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2a097e48-f4c4-3714-8b24-552b2d573754" name="Organization vDC Gateway: View Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2c4eb5ac-15f5-33f0-8b4a-680b3a1d3707" name="Organization vDC Gateway: Configure BGP Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9740be24-4dd7-373c-9237-91896338c11e" name="Organization vDC Gateway: View Static Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/84ddb40f-a49a-35e1-918e-3f11507825d7" name="Organization vDC Gateway: Configure Syslog" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/65439584-6aad-3c2c-916f-794099ee85bf" name="Organization vDC Gateway: View Remote Access" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/cdb0edb0-9623-30a8-89de-b133db7cfeab" name="Organization vDC Gateway: View SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/a5f5fc99-9afc-347b-9a31-f65f61f4416b" name="Organization vDC Gateway: Distributed Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/72c5e652-c8d7-3f19-ab83-283d30cb679f" name="Organization vDC Gateway: Configure Remote Access" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/92b7d500-6bb6-3176-b9eb-d1fda4ce444d" name="Organization vDC Gateway: Configure SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d9dabcab-579e-33c5-807b-dc9232bf7eff" name="Organization vDC Gateway: View BGP Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/be1abe9a-7ddc-38f6-bdf3-94affb01e46b" name="Organization vDC Gateway: Configure DHCP" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/ff3fc70f-fd25-3c0a-9d90-e7ff82456be5" name="Organization vDC Gateway: Configure System Logging" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/27be9828-4ce4-353e-8f68-5cd69260d94c" name="Organization vDC Gateway: Configure Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/eb525145-08e5-3934-91ef-ec80837c9177" name="Organization vDC Gateway: View OSPF Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/f72af304-97b0-379e-9d6d-68eb89bdc6cf" name="Organization vDC Gateway: Configure Static Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/7fee6646-ec0c-34c9-9585-aff6f4d92473" name="Organization vDC Gateway: View Firewall" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/a100f6a0-2c81-3b61-90c3-c4dbd721b3a8" name="Organization vDC Distributed Firewall: Enable/Disable" type="application/vnd.vmware.admin.right+xml"/>
    </RightReferences>
</Role>

Finally, NSX-SP Enterprise:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Role xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:ns9="http://www.vmware.com/vcloud/versions" name="NSX SP-Enterprise Rights" id="urn:vcloud:role:61d589ab-27ca-4e7a-be3e-656e0dcaa587" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/61d589ab-27ca-4e7a-be3e-656e0dcaa587" type="application/vnd.vmware.admin.role+xml">
    <Link rel="edit" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/61d589ab-27ca-4e7a-be3e-656e0dcaa587" type="application/vnd.vmware.admin.role+xml"/>
    <Link rel="remove" href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/role/61d589ab-27ca-4e7a-be3e-656e0dcaa587"/>
    <Description></Description>
    <RightReferences>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d6b35bfc-3309-3573-8e3d-6bdd1cb2b61f" name="vDC Group: Configure" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9dc33fcb-346d-30e1-8ffa-cf25e05ba801" name="Organization vDC Gateway: Convert to Advanced Networking" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/72c5e652-c8d7-3f19-ab83-283d30cb679f" name="Organization vDC Gateway: Configure Remote Access" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/eeb2b2a0-33a1-36d4-a121-6547ad992d59" name="Organization vDC Gateway: Configure L2 VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/f72af304-97b0-379e-9d6d-68eb89bdc6cf" name="Organization vDC Gateway: Configure Static Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/92b7d500-6bb6-3176-b9eb-d1fda4ce444d" name="Organization vDC Gateway: Configure SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2cd03d47-38e1-337a-907c-8d5b6a5258f2" name="Organization vDC Distributed Firewall: Configure Rules" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/65439584-6aad-3c2c-916f-794099ee85bf" name="Organization vDC Gateway: View Remote Access" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/c9e19573-3d54-3d4a-98f2-f56e446a8ef9" name="Organization vDC Gateway: Configure NAT" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/105191de-9e29-3495-a917-05fcb5ec1ad0" name="Organization vDC Gateway: View L2 VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/be1abe9a-7ddc-38f6-bdf3-94affb01e46b" name="Organization vDC Gateway: Configure DHCP" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/63c53fdf-80e5-3e31-ab26-ec7cc36ea759" name="Multisite: System Operations" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/cdb0edb0-9623-30a8-89de-b133db7cfeab" name="Organization vDC Gateway: View SSL VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/4e61b5b8-0964-36b6-b021-da39aea724fc" name="Organization vDC Distributed Firewall: View Rules" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/8e16d30d-1ae3-3fff-8d4b-64c342b186a9" name="Organization vDC Gateway: View DHCP" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/ff3fc70f-fd25-3c0a-9d90-e7ff82456be5" name="Organization vDC Gateway: Configure System Logging" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/a100f6a0-2c81-3b61-90c3-c4dbd721b3a8" name="Organization vDC Distributed Firewall: Enable/Disable" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2a097e48-f4c4-3714-8b24-552b2d573754" name="Organization vDC Gateway: View Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/b755b050-772e-3c9c-9197-111c286f563d" name="Organization vDC Gateway: Configure Firewall" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/eb525145-08e5-3934-91ef-ec80837c9177" name="Organization vDC Gateway: View OSPF Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/6ad5a05b-0d30-3bb5-acb1-02e7710a5ae6" name="vDC Group: View" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/209cde55-55db-33f1-8357-b27bba6898ed" name="Organization vDC Gateway: Configure IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/7fee6646-ec0c-34c9-9585-aff6f4d92473" name="Organization vDC Gateway: View Firewall" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/fb860afe-2e15-3ca9-96d8-4435d1447732" name="Organization vDC Gateway: View NAT" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/27be9828-4ce4-353e-8f68-5cd69260d94c" name="Organization vDC Gateway: Configure Load Balancer" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/84ddb40f-a49a-35e1-918e-3f11507825d7" name="Organization vDC Gateway: Configure Syslog" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/3b337aef-42a8-3ed1-8616-341152bc5790" name="Organization vDC Gateway: Configure OSPF Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/6edbfce1-4705-3cff-8dc7-8c03d36a6d45" name="Site: Edit" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/a5f5fc99-9afc-347b-9a31-f65f61f4416b" name="Organization vDC Gateway: Distributed Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/9740be24-4dd7-373c-9237-91896338c11e" name="Organization vDC Gateway: View Static Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/975d07ed-9c05-3277-a926-3c65933eb738" name="Site: View" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/82beb471-ab7f-3e2b-a615-136ba6645525" name="Organization vDC Gateway: View IPSec VPN" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/2c4eb5ac-15f5-33f0-8b4a-680b3a1d3707" name="Organization vDC Gateway: Configure BGP Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d9dabcab-579e-33c5-807b-dc9232bf7eff" name="Organization vDC Gateway: View BGP Routing" type="application/vnd.vmware.admin.right+xml"/>
        <RightReference href="https://vcd-01a.corp.local/api/admin/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9/right/d1c77fc0-a4b9-3d99-bd4b-d7fab35e4fae" name="Organization vDC Gateway: View" type="application/vnd.vmware.admin.right+xml"/>
    </RightReferences>
</Role>

I hope this helps others on aligning to the VCPP NSX levels and how to establish NSX capabilities inside of a vCD environment.

Thanks!

-Daniel

Setting up VMware vCloud Director 9.5 for Cross-VDC Networking

In this post, I will be reviewing the necessary steps to support Cross-VDC Networking inside of VMware vCloud Director 9.5. These are fairly straightforward since it aligns to the standard requirements set forth from Cross-vCenter NSX.

Pre-Requisites:

  1. Multi-Site management must be configured between the vCloud Director instances. I will try to add a post on establishing this at a later time.
  2. Ensure you have a unique vCloud Director installation ID. If you have duplicate IDs, this can lead to MAC address conflicts. Fojta did a blog post on updating your ID – please accomplish this before continuing.

Cross-vCenter NSX Configuration

vCD 9.5 does require a standard Cross-vCenter NSX configuration implemented between the resource/payload vCenters before we can do any configuration at the vCloud Director level.

There are many guides out there, but here’s a link to the official VMware documentation on setting up cross-vCenter NSX. 

This can be a single or multi-SSO domain topology. In my environment, here’s what I’ve configured between my two sites: Site-A and Site-B.

  1. From the Networking and Security plugin, I’ve assigned my Site-A NSX Manager while linking Site-B NSX Manager as the secondary instance – 
  2. From there, I need to establish my Universal Segment ID pool and Transport Zone.
  3. Keep in mind you do not want to overlap with an existing Segment ID pool, so pick a number that’s high enough (or out of reach from other pools) – 
  4. From the Transport Zone screen, I’ve created my new Transport Zone named “Universal-TZ” – 
  5. Now I’m ready to connect it to my respective clusters on Site-A and Site-B. Keep in mind I need to hit the drop down for the NSX Manager and attach the respective cluster at your secondary (or additional) location.
  6.  That’s it! Onto the next configuration which is at the vCloud Director level.

vCloud Director Initial Provider Setup

In this step, we need to assign the correlated NSX Manager to each vCenter instance that’s participating in the Cross-VDC networking solution. I will be showing how I’ve done this for my two sites, Site-A and Site-B, while establishing a fault domain.

  1. From my Site-A, navigate to System -> Manage & Monitor -> vSphere Resources -> vCenters – 
  2. We are going to right click, go to Properties of this vCenter – 
  3. From there, we need to go the NSX Manager tab. This is where we populate the following:
    1. Host/IP of NSX Manager
    2. Admin username
    3. Admin password
    4. Control VM Resource Pool vCenter Path – this can be either the MOref object id OR the ‘Cluster/RP’ path – I chose the former.
    5. Control VM Datastore Name – full name
    6. Control VM Management Interface Name – again, full name
    7. Network Provider Scope – now this is where we establish a fault domain. This Network Provider Scope could cover one or many vCenters in a single vCloud Instance. However, when we establish the vdc-Group, we must have a minimum of two different/unique fault domains (or network provider scope) inside of the created vdc-Group.
  4. Now, on my Site-B, I will configure my respective properties along with a Network Provider Scope of “region-b” – 
  5. Great! Next step is to add the Universal Transport Zone as a new network pool on each vCD instance. This is purely importing the created Universal-TZ and moving on, so very easy – 
  6. That’s it – now we are ready to enable a specific orgVDC for cross-VDC networking.

Enabling an orgVDC for Cross-VDC Networking

This is a very simple process – really just enable it on a per orgVDC basis.

  1. Go to your orgVDCs and right click on the orgVDC you want to enable cross-VDC Networking on. For example, I am enabling this on my Daniel oVDC’s – 
  2. Click on the Network Pool and Services sub-tab and you’ll see a new box below the Network Pool that states ‘Enable Cross VDC Networking (Using Network Pool “<Universal-TZ-Pool>”‘ Check this box.
    1. This still allows for local oVDC network creation using the traditional network pool as stated in the screenshot above – this is not a complete conversion to the Universal Transport Zone.
  3. Now, enabling this on my Site-B – 

Permissions/Rights required for Cross-VDC Networking

As discussed in the previous blog post, there are specific rights and roles required for Cross-VDC networking that are not enabled by default for the organization administrator. Please review these before the tenant utilizes Cross-VDC networking.

Cross-VDC Networking Permissions Review

Creation of the initial Cross-VDC Group

Now we are ready to test the creation of a new Cross-VDC group.

  1. Let’s log into the Tenant UI and we should see the Datacenter Groups from the context switching menu – 
  2. Now, I can create my first Cross-VDC group and start establishing my egress points. Awesome! 

More to come here on the Cross-VDC networking capabilities within vCD 9.5 from myself, Wissam Mahmassani, and Abhinav Mishra. Thanks!

-Daniel