One of the extremely exciting additions to VMware vCloud Director (vCD) is the ability to present vCenter instances securely to tenant organizations utilizing the vCD user interface – this is referred to as Central Point of Management, or what we abbreviate to as CPoM. Tom Fojta did a great job highlighting what’s new inside of vCD 9.7 here.
In this post, I am going to review the steps required to successfully deploy your first vCenter-SDDC to an organization inside of vCloud Director 9.7. You will need to utilize the CloudAPI, but do not be alarmed, I will walk you through these steps.
In this series of blog posts, I will be discussing the new release of VMware vCloud Availability 3.0 (vCAv). This is a very exciting release for the VMware Cloud Provider team. vCAv 3.0 will be out shortly (by end of our fiscal quarter) and I want to provide my insight into this platform. I will be focusing on the following points –
Introduction to vCAv 3.0
vCAv 3.0 Service Roles
Deployment Approach for Provider
Deployment for Tenant (On-Premises)
Introduction to vCAv 3.0
First off, let’s discuss what vCAv provides from a functionality perspective. vCAv is what I like to call “functional convergence.” In the past, we had many different products that provided some level of availability or migration capability –
In my opinion, this was a duplication of appliances and could be confusing to customers. The team has done a great job of putting forth a significant investment into vCAv 3.0 to simplify the architecture. Therefore, here’s where we are today –
Therefore, no need for multiple tools for migration, DR from/to Cloud, or between Clouds. We now have a single solution that answers all of the above.
vCAv 3.0 Key Functionality
So what does vCAv 3.0 provide?
Simple, Unified Architecture
There is a single OVF for the Provider and the Tenant. On the Provider side, each role can be easily deployed used by the vSphere Client or CLI.
Deployment is very intuitive and scalable – each role can be quickly deployed in a matter of minutes.
On-Premises Appliance is unified and provides vCenter UI integration for management.
On-Premises Migration and Protection
On-Premises appliance provides the same UI experience as connected to the Cloud instance, but built into vCenter.
Migration and/or protection of workloads can be done with a few clicks.
Allows for protection/migration to and from vCloud instances.
Cloud to Cloud Migration and Protection
Very similar behavior to C2C 1.5, we can protect workloads (vApps and VM’s) between vCD instances.
Network Mapping and Re-Addressing
One of the new additions is the ability to re-address and map out protected workflows for faster recovery at the destination site.
This maps to existing vCD oVDC Network constructs such as Static IP pools.
As discussed before, the team is aware of scalability requirements for Providers. For this version, here is the stated guidelines:
300 tenants with active protections paired to vCloud Director instance
20 vCAv Replicator instances per instance
500 active protections per vCAv Replicator instance
9,500 active protections across tenants to a Cloud
5TB protected VM size (contingent upon Cloud storage)
Direct and native vCloud Director integration for providers and tenants – with the ability to provide self-service, this provides a unique experience that meets DRaaS requirements and migration functionality.
Ease of Operationalization – I’ve done several deployments during the development process and it’s one of the easiest VMware Cloud Provider solutions to deploy. Once we review the roles and concepts, anyone should be able to operationalize this with ease.
Cost-Effective Approach – vCAv 3.0 will be part of the VMware Cloud Provider Program (VCPP). This is based on the monthly consumption of points, which is a very cost-effective solution that can be modeled and productized for DRaaS and migration offerings.
In the following diagram, we can see how this all comes together with a single, on-premises vCenter along with two vCD instances. One can pair this up with vCD multi-site federation capability.
Moreover, this pairs very well with existing vCD services such as CrossVDC Networking or L2VPN connectivity between on-premises and organization VDCs.
vCAv 3.0 Service Roles
Let’s review the distinct service roles of vCAv 3.0.
On the Provider side, we have the following –
Cloud Replication Management
This is a logical entity that consists of the core of vCAv 3.0.
vCloud Availability Portal – User Interface for the tenant and provider. All UI configuration ingresses from this service and applies all to all necessary connected components.
vCloud Availability vApp Replication Manager – communicates directly to vCD and understands tenancy constructs such as organizations, vApps, etc. Also responsible for enabling protections or migrations.
vCloud Availability Replication Manager – understands vCenter and ESXi concepts and will interoperate between the replicators and protected vCenters.
vCloud Availability Replicator – lightweight node responsible for executing on the host-based replication from a specific host. Typically, you deploy a replicator per vCenter.
vCloud Availability Tunnel – this is the tunneling service that is responsible for providing secure connectivity between on-premises vCenter(s) and connected vCD instances.
Each of these roles can be deployed separately or in a combined virtual appliance. For production deployments (which we will review later), the recommendation is standalone deployments for each role.
Tenant / On-Premises
On the tenant side, we have a single appliance that has a combined appliance approach –
vCloud Availability Replicator – just like on the Provider side, the Replicator is responsible for executing the host-based replication (HBR) process
vCloud Availability Tunnel – provides secure connectivity between on-premises and vCloud environment. All traffic securely ingresses and egresses through this service.
vCloud Availability Plugin – this plugin provides local vCenter UI management that is the same experience as connecting the vCAv Cloud environment.
While this blog series will cover the Provider and On-Premises side in further detail, we will have the following steps to execute on for a successful deployment.
Deployment of Cloud Replication Management (CRM) Instance
Deploy vCAv Replicator(s)
Deploy vCAv Tunnel
Configuration of CRM instance and start of site wizard
Configuration of Replicator
Configuration of Tunnel
Deploy vCAv On-Premises Appliance
Start configuration wizard
Connect to vCAv Cloud Tunnel
Configuration of local placement
Next up, I will review the Provider deployment process in further detail while providing the step-by-step procedures. Stay tuned!
Recently, I received a request from one of our aggregators regarding how Equal Cost Multipathing (ECMP) is metered within the VMware Cloud Provider Program (VCPP), specifically Tom Fojta’s recommendation on architecting Provider-managed NSX Edges and Distributed Logical Router (DLR) in ECMP mode, specifically this diagram from the Architecting a VMware vCloud Director Solution –
As shown in the diagram above – How does Usage Meter handle bill these tenant virtual machines (VMs) when we have a provider NSX architecture that utilizes ECMP?
For you TL;DR readers – any VM connected to a Tenant Edge / direct network that has ECMP enabled northbound, NSX Advanced will be charged for said VM. Read on if you want to learn how this is done.
First off, let’s talk about why this matters. Per the Usage Meter Product Detection whitepaper (this can be found on VMware Partner Central), we can see how Usage Meter detects specific NSX features based on the pattern of usage. Regarding dynamic ECMP, it is metered by the “Edge gateway” which could be a little ambiguous. If one utilizes ECMP, they would be metered for NSX Advanced within VCPP.
One of the scenarios from the whitepaper does show ECMP-enabled Edges but not an Edge that is abstracted away from the provider environment –
My initial reaction was that Usage Meter would not look at the northbound provider configuration and the interconnectivity to vCloud Director. However, I was not confident and wanted to verify this explicit configuration and expected metering. Let’s review my findings.
In the above diagram, we can see I created a similar Provider managed NSX configuration with ECMP enabled from the DLR to the two Provider Edges with dynamic routing enabled (BGP). From there, I expose a LIF/Logical Switch named “ECMP-External-Network” to vCloud Director that is then exposed to my T2 organization as a new External Network.
From there, I created a dedicated Tenant Edge named “T2-ECMP-ESG” that will be attached to this newly created network along with a VM named “T2-ECMP-VM.” The goal is to verify how T2-ECMP-VM and T2-TestVM are metered by Usage Meter with this newly created Tenant Edge.
My Edges are setup for BGP and reporting the correct routes from the southbound connected DLR (and tenant Edges) –
From the DLR, we can see that I have two active paths to my Provider Edges (Provider-Edge-1 and 2) –
Last of all, my T2-ECMP-ESG is operational and attached to the newly created ECMP External Network –
Last of all, I have my VM’s created and powered on (remember, Usage Meter will only meter powered on VM’s). We can see T2-ECMP-VM is attached to a org routed network from T2-ECMP-ESG named “T2-ECMP-Network” –
Let’s work from the north to south – start with the Provider Edges and show how Usage Meter detects and bills.
Note – I have vROps Enterprise in my lab environment, so we will see Usage Meter picking up vROps registration and placing it in the appropriate bundle.
Provider Edges / DLR
As expected, the Provider Edges and DLR are detected along with registration to vROps. By design, NSX Edges are charged for the Advanced SP Bundle as they are metered as a management component (minimum Advanced bundle / 7-point). However, in my case, we see detection, and then registration to vROps Enterprise. Therefore, since it’s a bundle ID (BND) of 12, this is correlated to Advanced Bundle with Management (10-point) –
Tenant Edge – T2-ECMP-ESG
Just like the Provider Edges and DLR, we see T2-ECMP-ESG register to UM along with vROps Enterprise registration. Same billing model as above.
Tenant VM – T2-TestVM
I would not expect any change to this VM, but wanted to showcase that having a separate Edge with standard networking (i.e. no ECMP) will bill based off the NSX SP Base level. As expected, T2-TestVM was handled by Usage Meter just as anticipated – we can see registration, NSX SP Base usage, along with registration to vROps Enterprise –
Tenant VM – T2-ECMP-VM
Finally, let’s take a look at my T2-ECMP-VM – as discussed before, this is wired to a Tenant Edge that is connected to the ECMP-enabled DLR via an External Network.
We see initial registration, registration to vROps Enterprise, then NSX Advanced usage! This would be metered at Advanced Bundle with Networking and Management due to the NSX Advanced usage (12-point).
Summary of Findings
Here’s what we learned:
Edges/DLR Control VM’s are not charged for NSX usage since UM handles them as a management component. If you are using vROps, it will place it in the most cost effective bundle.
Utilizing ECMP at the provider-level DOES impact any southbound connected VM from a billing perspective, even if an Edge sits in between the ECMP enabled configuration and the tenant VM. Per the findings, NSX Advanced will be metered.
Therefore, be aware of any NSX provider architecture and the use of NSX specific features.
Again, this shows the logic inside of Usage Meter and how it relates to metering for tenant workloads. Cheers!
I was recently asked by a colleague if we have any existing collateral on VMware vCloud Director (vCD) that maps to the VMware Cloud Provider Program (VCPP) NSX levels that are currently available to partners. Well, there wasn’t, until now. 🙂
First, let’s talk about the NSX bundles inside of VCPP –
There are three levels identified within VCPP:
NSX-SP Base – this is is your fundamental level of NSX. It does include your normal Edge services, Edge Firewall, NAT, Load Balancing, Dynamic/Static Routing, IPSEC/SSL VPN+, and Distributed Routing and Switching. This is typically referred to as “vCNS” mode (callout to the vCD old days) but does use NSX.
NSX-SP Advanced – this includes Base, plus ECMP and Distributed Firewall functionality. Service Insertion, AD Integrated Firewall, etc. are all functions that the Provider can consume from the backend management.
NSX-SP Enterprise – this includes Advanced along with HW VTEP integration, cross-vCenter NSX functionality, along with the L2VPN (Remote Gateway) solution. The new addition here is vCD 9.5 Multi-Site and Cross-VDC capability.
Last of all, the “Convert to Advanced Gateway” inside of vCloud Director for organization Edges DOES NOT mean you will be using NSX Advanced right away! This is just a change in how vCD presents the Edge UI (with Advanced, it’ll show the H5) along with the API rights available. I demonstrate this in the above post too.
So let’s talk about the NSX levels and how they can pertain to vCD rights and role. I worked up the following roles in my vCD environment:
NSX SP-Base Rights
NSX SP-Advanced Rights
NSX SP-Enterprise Rights
So what does one gain when using these rights? Well, they are now aligned to the VCPP NSX bundles and can be utilized as a starting to monetize NSX inside of a vCD environment.
Now, in my experience, these specific vCD permissions will apply to the VCPP NSX levels as stated above. The big thing I found in my testing is ECMP can be toggled with Static Routing, so I set this as “View Only” for any routing capability for SP-Base.
If you are using vCD 9.5, one could also create a rights bundle that is published to an organization along with utilizing Global Roles to make this *much* easier.
The steps for this would be: Creation of Rights Bundle(s) -> Publish to org(s) -> Creation of Global Roles -> Publish to org(s) -> Apply role to user(s)
Alright, here are the three exports for these rights required. This is not comprehensive for all vCloud permissions required, but gives you an idea of what to append to your existing role (which could be easily done via REST API). Note that the exports below show my vCD instance (vcd-01a.corp.local) along with the org UUID, so replace this if you are doing a POST.