VMware Cloud on AWS Management Exam 2019 5v0-31.19 – Exam Review and Tips

VMware and Amazon Web Services are making a significant investment in providing a vSphere experience in a hyperscaler environment and we continue to see adoption for Cloud Service Providers utilizing VMware Cloud on AWS as an offering for managed services.

In this post, I will review my approach, what to expect on the exam, and my raw study notes.

I decided to prepare for the VMware Cloud on AWS Management Exam 2019 (Exam 5v0-31.19) and sit for it. While there’s a lot of great material that I will review below, I wanted to provide my approach for others.

My Approach

  • Always start with the blueprint. I print this out and go through line to verify I have a base level understanding of the objective and what I need to focus on.
  1. First off, there is some great material already online that I reviewed:
    1. Paul McSharry‘s writeup on the material and his mindmap which is here.
    2. Manny Sidhu did a great writeup of his notes and approach here.
  2. I went through the VMware Cloud on AWS: Deploy and Manage – On Demand course on VMware Education.
    1. This was beneficial as it covered a lot of the fundamentals including some AWS concepts I was not aware of.
    2. I would highly recommend this for people even new to VMware vSphere as it reviews many of these concepts at a high-level.
  • Went through the VMware Hands on Lab for VMware Cloud on AWS (1987-01-HBD) – this is great to get your hands-on and experience setting up a SDDC. Walks through many of the same concepts I saw in the Deploy and Manage course.


It’s 30 questions and is done via the Pearson Vue site online. I thought it was very fair as a skill/badge exam and goes over many of the fundamental requirements. As expected, it’s true to the blueprint.

My Raw Notes

For me, I always prepare a final checklist of things I review before I take any examination. Below is my list of raw notes I used before I took the exam. As they are my raw notes, expect some abbreviations.

Anyway, enjoy the exam and I look forward to the expansion of VMware Cloud on AWS!

VMC on AWS Study Notes:

  1. Use Cases:
    1. Extension of onprem DC to the public cloud to expand resource capacity, increase disaster avoidance and recovery options, or localize application instances.
    2. Consolidation
    3. Peering the private and public cloud that allows for application mobility
  2. Global compliance – ISO, SOC1-3, GDPR, and HIPAA
  3. Many different AWS regions available plus GovCloud
  4. SDDC
    1. Minimum of 3 hosts and maximum of 32 hosts
    2. Up to 10 clusters can be added to a SDDC
    3. Stretched Cluster – between two AZ’s (min of 6 hosts and max of 28)
  5. Host Configuration
    1. 2 18 core sockets – Broadwell 
    2. 512 gibibytes of memory (550GB)
    3. 14.3TB of NVMe SSD’s – 3.6TB for flash, 10.7TB for capacity
    4. 1 AWS ENA – 25Gbps
  6. vSAN/Storage
    1. Two Disk Groups per host
    2. Dedupe/Compression is on by default
    3. Encryption is happening at the drive level
    4. Two different datastores – WorkloadDatastore (for workloads) and vsanDatastore (management VMs, cannot be modified)
    5. Default policy is PFTT 1, RAID1
    6. Can pick from PFTT of 1 to 3, RAID1/5/6 (if available hosts)
    7. Since All Flash, reads are from cap tier
  7. Stretched Cluster
    1. Sync writes between two AZ’s
    2. Witness host is added and not charged to customer
    3. Pre-reqs
      1. Requires a AWS VPC with two subnets, one subnet per AZ
      2. Smallest SC is 6, largest is 28
      3. Must grow in pairs
      4. Adding hosts trombones between AZs – first one is added to AZ1, next is AZ2, next is AZ1, and so on
    4. Site Disaster Tolerance – default is dual site mirroring 
  8. Network
    1. Traffic is separate between management and compute gateways
    2. Amazon Direct Connect allows for low-latency connections between on-prem and AWS. 
    3. ENA’s are highly redundant even though there’s a single pNIC per host
    4. Two types of VPCs
      1. One created and managed by VMware – underlying VPC that is created when the SDDC is created 
      2. Second – VPC that you create so you can peer with native AWS services 
    5. Firewall
      1. Default deny all
      2. Must add rules for vCenter access, IPsec, etc
      3. Firewall Rule Accelerator created a group of rules to accelerate successfully connecting a VPN tunnel
    6. Logical Networks
      1. Routed network – internal communication over a IPsec or Internet connection. Normal overlay network that we are used to. 
      2. External Network – utilized for L2VPN connectivity. Requires Tunnel ID. Think of this as a subint
    7. Inter-Networking Scenarios
      1. Compute GW – IPsec for guest OS connectivity
      2. Compute CW – L2VPN for vMotion, same L2 domain 
      3. Direct Connect with pub virtual interface – in conjunction with IPsec or L2VPN or Pub Internet. Used for AWS services
      4. Direct Connect with private virtual interface – secured to direct SDDC
  9. Hybrid Linked Mode
    1. Allows for a single management interface between on-prem vCenter and VMC
    2. Pre-req for migration from on-prem to VMC
    3. Same SSO is not needed 
    4. Configuration is only done from one of the vCenters to configure HLM. Will only be visible from this vCenter for future management. So, no bi-directional UI support. 
    5. Pre-reqs
      1. IPsec VPN connection between on-prem and SDDC management gateway
      2. Network connectivity between your VMC vCenter and on-prem vCenter server and identity source
      3. Same DNS 
      4. Less than 100ms RTT
      5. Misc ports needed for successful connectivity 
    6. vCenter Cloud Gateway Appliance configured HLM. 
  10. HCX
    1. Can do migrations between vSphere 5.1 to VMC
    2. No charge
  11. VPC
    1. Only one VPC can be connected to a SDDC
    2. VPC subnets can only reside in one AZ. 
    3. Elastic IP addresses are public IPv4 addresses mapped to the AWS account, not the resource.
    4. Connecting – 
      1. Must connect a Amazon VPC or if it’s a single node SDDC, can delay up to 14 days.
  12. Migrating VMs
    1. Cluster EVC and Per-VM EVC 
      1. In 6.7, can enable disable or change the EVC mode at the VM level. 
    2. Requirements for Hybrid Cold Migration
      1. vSphere 6.5 patch d or later, 6.0U3, vSphere 5.1/5.5
      2. IPsec VPN
      3. HLM but can use move-vm cmdlet 
    3. Hybrid Migration with vMotion
      1. Minimum bandwidth of 250Mbps and less than 100ms RTT
      2. vSphere 6.5 patch d / vSphere 6.0U3
      3. IPsec VPN
      4. vCSS/vDS 6.0 or 6.5
      5. AWS DC with a private virtual interface 
      6. HLM or move-vm cmdlet 
      7. L2VPN to extend VM networks between on-prem and VMC
      8. All FW rules in order. 
      9. VM hardware version 9, Cluster based EVC baseline on Broadwell
    4. Per-VM EVC
      1. Must be hardware version 14 or greater 
      2. VM must be powered off to change Per-VM EVC
  13. Permissions and Security
    1. CloudAdmin –
      1. Necessary privileges for creating/managing workloads in the SDDC
      2. Does not allow changing the configuration of management components that are supported by VMW
    2. CloudGlobalAdmin – 
      1. Associated with global privileges that allows you to create and manage content library objects and perform other global tasks. 
    3. cloudadmin@vmc.local is the default user generated during creation. 
    4. Other users cannot be created until HLM is configured. DO NOT modify solution users associated with the VMC created in an on-prem vSphere domain
  14. Elastic DRS
    1. Allows the SDDC to scale based on resource thresholds 
    2. Not supported for multi-AZ deployment or single host SDDC
    3. If a user adds or removes a host, current EDRS remediations are ignored 
  15. Licensing/Pricing
    1. On-Demand, One-Year and Three-Year Subscription models
    2. HLP discounts of up to 25%
    3. Site Recovery is an add-on cost
    4. All other AWS services are billed separately
  16. Cloud Services Roles
    1. Organization Owners – 
      1. Can have one or more
      2. Owners can invite additional owners and users, manage access
    2. Organization Users –
      1. Access VMware Cloud services
      2. Cannot invite users, change access, or remove
  17. Deployment
    1. Default Subnet CIDR is – reservations for other RFC1918 addresses
    2. is reserved for default compute
    3. Maximum hosts are dictated by the CIDR block you state
  18. Content Libraries 
    1. Onboarding Assistant is a java CLI tool for transferring to VMC
    2. Can still utilize subscribe functionality
    3. Utilize vSphere Client to upload files
  19. Site Recovery
    1. vSphere Replication based
    2. Supports Active-Active, Active-Passive, Bidirectional
    3. Pre-Reqs
      1. vCenter 6.7/6.5/6.0U3, ESXi 6.0U3 or later
      2. SRM 8.x on-prem

How does VMware vCloud Availability for Cloud-to-Cloud 1.5 interoperate with vCD Cross-VDC Networking?

I get this question quite a bit due to the new vCloud Director 9.5 Cross-VDC networking functionality – does vCloud Availability for Cloud-to-Cloud 1.5 (C2C) work with stretched networking inside of Cross-VDC networking?

The answer is: yes!

This is another great addition for recoverability considerations as one could fail over between vCloud Director instances without modifying the guest OS IP address. Furthermore, based on the application architecture, one could have active-active applications and ensure replication/failover in the event of a disaster.

Let’s go through my example high-level design I’ve worked up in my lab –

Example Cross-VDC setup with vCloud Availability for Cloud-to-Cloud 1.5

In the above diagram, we can see I have two active vCloud Director instances, Site-A and Site-B. I have two organizations, “Daniel” that resides in Site-A along with “Daniel-B” that resides in Site-B.

C2C is deployed on each site in the combined model and I have multi-site pairing completed so I can easily manage this between my two sites –

Within my Cross-VDC networking setup, I currently have my active egress setup to Site-A as depicted in the diagram above.

Last of all, I ran a protected workflow from Site-A to Site-B for my Stretched-vApp-VM –

From there, one can either migrate or failover the workload and without any guest OS IP changes. I am going to do a video shortly, but here’s a short GIF I created that shows the ease of use of failing over between my Site-A and Site-B –


After failover, I can then access Stretched-vApp-VM from the new NAT address on Site-B.

An Organization Administrator could also configure active/active or active/passive egress points for additional resiliency. This provides further capability inside of vCloud Director, especially with stretched networking and a complementary availability solution.



vCloud Director 9.5 Cross-VDC Networking Video Walkthrough and Discussion

Over the past few weeks, Wissam Mahmassani, Abhinav Mishra, and I have created a few walkthrough videos on setting up Cross-VDC networking inside of vCloud Director 9.5.

Unfamiliar with Cross-VDC networking in vCD? Check out this series of blog posts that review the capabilities and design considerations:

Intro and Use Cases

Getting Started with Cross-VDC

High-Level Provider Design

Design Considerations and Conclusion

The intent of these videos is to discuss setting up Cross-VDC networking in vCloud Director but also have a live chat on items we’ve learned along the way with working with it. Quite frankly, it was an open discussion between the team on the inner workings on vCD/NSX and what our development team has done in the backend.

Video Walkthrough

In the first video, we discuss the pre-requisites before we can start configuring vCloud Director for Cross-VDC networking. In essence, the assumption is cross-vCenter NSX has already been established and we have the primary and secondary NSX managers registered.

Next, we review the concept of creating a Datacenter group and what are the different egress options. This is very important as it explicitly controls how traffic exits the overlay environment.

Here, we discuss how BGP weights control our active/passive egress points and what vCD automates in the backend. The key is this is all done without provider/tenant configuration – vCD automates this process.

As a final wrap-up of the BGP weights, we review creation of the stretched networks inside of vCloud Director along with operational management inside of the vCD H5 UI.

Last of all, we demonstrate testing of Cross-VDC and failover of my “Daniel-App” between the two sites. What’s interesting is the ability to migrate egress points without any loss of connectivity. Unintended failover is managed by BGP weights, which the default timer is 60 seconds and could be revised if required.

As stated before, this shows the requirement of having a mirror Edge configuration, especially for NAT configuration and failover testing between sites.

This was a fun experience with the team while reviewing and having open discussions on Cross-VDC networking. We are hoping these are valuable for those of you that are interested in bringing this as a new service inside of vCloud Director.



How is ECMP metered when configured in the Provider architecture within the VMware Cloud Provider Program?

Recently, I received a request from one of our aggregators regarding how Equal Cost Multipathing (ECMP) is metered within the VMware Cloud Provider Program (VCPP), specifically Tom Fojta’s recommendation on architecting Provider-managed NSX Edges and Distributed Logical Router (DLR) in ECMP mode, specifically this diagram from the Architecting a VMware vCloud Director Solution – 

As shown in the diagram above – How does Usage Meter handle bill these tenant virtual machines (VMs) when we have a provider NSX architecture that utilizes ECMP? 

For you TL;DR readers – any VM connected to a Tenant Edge / direct network that has ECMP enabled northbound, NSX Advanced will be charged for said VM. Read on if you want to learn how this is done. 

First off, let’s talk about why this matters. Per the Usage Meter Product Detection whitepaper (this can be found on VMware Partner Central), we can see how Usage Meter detects specific NSX features based on the pattern of usage. Regarding dynamic ECMP, it is metered by the “Edge gateway” which could be a little ambiguous. If one utilizes ECMP, they would be metered for NSX Advanced within VCPP. 

One of the scenarios from the whitepaper does show ECMP-enabled Edges but not an Edge that is abstracted away from the provider environment – 

My initial reaction was that Usage Meter would not look at the northbound provider configuration and the interconnectivity to vCloud Director. However, I was not confident and wanted to verify this explicit configuration and expected metering. Let’s review my findings. 

Lab Setup

In the above diagram, we can see I created a similar Provider managed NSX configuration with ECMP enabled from the DLR to the two Provider Edges with dynamic routing enabled (BGP). From there, I expose a LIF/Logical Switch named “ECMP-External-Network” to vCloud Director that is then exposed to my T2 organization as a new External Network.

From there, I created a dedicated Tenant Edge named “T2-ECMP-ESG” that will be attached to this newly created network along with a VM named “T2-ECMP-VM.” The goal is to verify how T2-ECMP-VM and T2-TestVM are metered by Usage Meter with this newly created Tenant Edge. 

Lab Configuration

My Edges are setup for BGP and reporting the correct routes from the southbound connected DLR (and tenant Edges) – 

From the DLR, we can see that I have two active paths to my Provider Edges (Provider-Edge-1 and 2) – 

Last of all, my T2-ECMP-ESG is operational and attached to the newly created ECMP External Network – 

Last of all, I have my VM’s created and powered on (remember, Usage Meter will only meter powered on VM’s). We can see T2-ECMP-VM is attached to a org routed network from T2-ECMP-ESG named “T2-ECMP-Network” – 


Let’s work from the north to south – start with the Provider Edges and show how Usage Meter detects and bills. 

Note – I have vROps Enterprise in my lab environment, so we will see Usage Meter picking up vROps registration and placing it in the appropriate bundle.

Provider Edges / DLR

As expected, the Provider Edges and DLR are detected along with registration to vROps. By design, NSX Edges are charged for the Advanced SP Bundle as they are metered as a management component (minimum Advanced bundle / 7-point). However, in my case, we see detection, and then registration to vROps Enterprise. Therefore, since it’s a bundle ID (BND) of 12, this is correlated to Advanced Bundle with Management (10-point) – 

Tenant Edge – T2-ECMP-ESG

Just like the Provider Edges and DLR, we see T2-ECMP-ESG register to UM along with vROps Enterprise registration. Same billing model as above. 

Tenant VM – T2-TestVM

I would not expect any change to this VM, but wanted to showcase that having a separate Edge with standard networking (i.e. no ECMP) will bill based off the NSX SP Base level. As expected, T2-TestVM was handled by Usage Meter just as anticipated – we can see registration, NSX SP Base usage, along with registration to vROps Enterprise – 

Tenant VM – T2-ECMP-VM

Finally, let’s take a look at my T2-ECMP-VM – as discussed before, this is wired to a Tenant Edge that is connected to the ECMP-enabled DLR via an External Network. 

We see initial registration, registration to vROps Enterprise, then NSX Advanced usage! This would be metered at Advanced Bundle with Networking and Management due to the NSX Advanced usage (12-point). 

Summary of Findings

Here’s what we learned:

  1. Edges/DLR Control VM’s are not charged for NSX usage since UM handles them as a management component. If you are using vROps, it will place it in the most cost effective bundle.
  2. Utilizing ECMP at the provider-level DOES impact any southbound connected VM from a billing perspective, even if an Edge sits in between the ECMP enabled configuration and the tenant VM. Per the findings, NSX Advanced will be metered. 
  3. Therefore, be aware of any NSX provider architecture and the use of NSX specific features. 

Again, this shows the logic inside of Usage Meter and how it relates to metering for tenant workloads. Cheers!