vCloud Director – Key Differences in Edge Gateway Services (1 of 2)

Two-part vCD series since it was longer than I expected!

I had a question come in from a Cloud Provider on what are the actual key differences between a standard Edge Gateway Service and an Advanced Edge inside of the vCloud Director (vCD) User Interface (UI). While I could explain a few things on my own, I decided to do a little bit of legwork to confirm my suspicions. While some of you may already know the following, I thought this was an interesting exercise and wanted to share my results.

Before I get to that, I’m sure everyone is aware vCloud Director started off with vCloud Network and Security (VCNS) and this was the network backing before NSX. With recent versions of vCloud Director, everything is backed by NSX.

With that said, the Advanced Gateway experience is what VMware will eventually migrate to. Therefore, get used to the nice HTML5 intuitive and speedy UI! ūüôā


In my vCD 9.x instance, I have two edges deployed:

  1. SiteB-T1-ESG is my advanced edge. I can verify this by right-clicking on the edge and seeing that I do not have an option to Convert to Advanced Gateway 
  2. Moreover, you can see I am running version 9.x of vCD РI can convert it to a Distributed Logical Router! 
  3. However, with my SiteB-T1-ESG-2, I can see it’s not an Advanced Gateway as I’m able to convert it¬†

Let’s get to¬†the comparisons now. Again, this is going to be in the context of the UI – not going to talk about the API right now. Going to state the advantage based on service in the title.

Firewall Services – Advantage: Advanced Gateway

Advanced Gateway

  1. I can create granular firewall rules using grouping objects associated with the HTML5 interface.
  2. This provides a very similar experience to NSX within vCenter. To be honest, anyone that has used NSX should be able to figure this out very quickly.

Standard Gateway

  1. From the standard interface, I can only create rules from a IP/CIDR and key words such as “any, internal, external.”
  2. Pretty limited to say the least.

DHCP Services – Advantage: Advanced Gateway

Advanced Gateway

  1. From the DHCP subtab, I am able to establish pools, bindings, and relay configurations. Moreover, configuring IP Sets and DHCP Relay Agents.

Standard Gateway

  1. We have the ability to add a DHCP pool that’s applied on an internal network that’s connected to this ESG. Pretty basic capabilities, but works.

NAT Services – Advantage: Tie

Advanced Gateway

  1. Ability to establish Destination or Source NAT’s. I see the same options between both Advanced and the Standard gateway, so it’s hard to call an advantage either way.

Standard Gateway

  1. As stated with the Advanced Gateway, I have the ability to establish a DNAT or SNAT. Seems like the same options to me.

Routing Services – Advantage: Advanced Gateway

Advanced Gateway

  1. This seems like a night and day difference in routing options. I’m able to get an NSX-like experience from an HTML5 interface (that’s been around for over 1 year or so!)
  2. Ability to set ECMP, Routing ID’s, utilize OSPF, BGP, and Route Redistribution with prefixes to boot.
  3. If you’re used to NSX and applying routing configurations to an Edge, this is a very similar experience.

Standard Gateway

  1. Yeah, how do static routes sound to you? That’s all I can apply here from the UI.

Load Balancer – Slight Advantage: Advanced Gateway

Advanced Gateway

  1. The Advanced Gateway is very similar to what we see in NSX – just in an HTML5 format.
  2. We get to see our Global Configuration, Application Profiles, Monitoring, Rules, Pools and Virtual Servers.
  3. I also see we have additional algorithms available from an LB perspective. I wouldn’t say it’s a stark difference between Advanced and Standard, but more comprehensive than the Standard Gateway.

Standard Gateway

  1. Standard Gateway has very similar options as the Advanced UI, just in a different UI format.
  2. As stated above, we don’t have UDP available as a type and fewer algos for the Pool configuration. With that said, it’s very comparable, but giving a slight advantage to Advanced for some of the other options available.

More to come on Part 2 here.

A Deeper Look into NSX L2VPN with vCD Extender Orchestration

I’ve been rebuilding my vCloud Director (vCD) lab and running through a few connectivity scenarios. Moreover, I wanted to write and share my findings on orchestrating an on-prem NSX environment connecting to a vCD/Provider environment using vCloud Director Extender (VXLAN to VXLAN). In vCD Extender, this is also known as a DC Extension.

To back up, let’s talk about how NSX provides flexible architecture as it relates to Provider scalability and connectivity. My esteemed colleagues did some great papers from our vCloud Architecture Toolkit (vCAT):

Before I get started, I also think this is a good guide for planning out VXLAN <–> VXLAN VPN connectivity.

Let’s look at my lab design from a network / logical perspective –

As you can see above, I have my Acme-Cloud organization available on the left with a single VM in the 172.16.20.x/24 network that’s running on NSX/VXLAN.

On the right, we have “Acme DC” that’s also using NSX and has a logical switch named “Acme-Tier” with the same network subnet.

The orange Extender Deployed L2VPN Client is what’s deployed by vCD Extender on tunnel creation. We’re going to walk through how Extender creates this L2VPN tunnel within an on-prem NSX environment.


  1. This is very similar to my warm migration setup, so I’m going to try not to duplicate material.
  2. I have my Acme-Cloud-Tier Org VDC Network that was converted to a Subinterface inside of vCD: 
  3. We can see in the Edge Services, my L2VPN Server has been set up¬†with a default site configuration. However, vCD Extender creates its own site configuration –¬†
  4. Extender generates a unique ID for the username and password. This is done when the DC Extension is executed by the tenant. I also established the egress optimization gateway address for local traffic. 

Tenant – vCD Extender Setup

  1. Before we can create a Data Center Extension, we need two required fields for NSX deployments.
  2. First, we need to give the required information to successfully deploy a standalone Edge that will be running our L2VPN client service. This would include the uplink network along with an IP address, gateway, and VMware-specific host/storage information.
  3. Second, we need to provide the required NSX Manager information. I’m sure this is to make the API calls required to deploy the Edge device(s) to the specified vCenter.¬†
  4. Once the DC Extension has been created, we would see a new Edge device under Networking & Security 

Tenant – DC Extension (L2VPN) Execution

  1. So what happens when we attempt to create a new DC Extension (or L2VPN Connection)? A few things:
    1. Creation of our trunk port for our specified subinterface
    2. Deployment of the new Edge device that will act as the L2VPN Client
    3. Reconfiguration of the trunk port (uses mcxt-tpg-l2vpn-vxlan-** prefix)
    4. Allowing NSX to do its magic along with L2VPN
  2. We can see within my task console what happened –¬†
  3. Voila, we have a connected L2VPN tunnel. As you can see, the blue “E” stipulates that we have a local egress IP set. I did this since I wanted to route traffic to its local interface for traffic optimization.
  4. So, what happens in the background? Well, let’s take a look at the Edge device. We can see the trunk interface was created while the subinterface is configured to point to my logical switch “Acme-Tier.”¬†
  5. Last of all, the L2VPN configuration was completed and established as the Client. We can now see the tunnel is alive too. 
  6. From the main vCD Extender page, we can also see traffic utilization over the tunnel. Pretty nice graph! 

Just a quick ping test, yep! WebVM can access WebVM2.

In summary, NSX to NSX DC Extensions within vCD Extender works pretty similar to Provider/VXLAN to On-Prem/VLAN. The key difference is the on-prem vCD Extender has the embedded Standalone Edge to deploy to vCenter.

Enjoy those cloud migrations!


VMware VCIX6-NV Unlocked – Experience and Tips

I am happy to state that I passed my VCAP-NV and will be achieving VCIX6-NV.

This was an exhausting test – and sometimes frustrating. I even had a PSOD on one of my hosts that I had to resolve (pretty sure that was not part of the lab test)!

I have many people to thank but there have been some great guides out there. I posted a blog article on links I collected for preparing for my VCAP-NV.

I’ll be the first to admit this was my second attempt at my VCAP-NV: the first attempt one just got to me. Not to make any excuses, but my background isn’t networking/routing, so many of these topics were green to me.

I dusted myself off and said I’ll knock the second attempt out of the park. As you can see, VMware does document the blueprint objectives you missed on the test.

So what did I do? Before I start that, here’s a summary of my experience:


  1. This test will frustrate you. Yes, it is meant to. Accept it and embrace it.
    1. It is a nested vPOD/Lab environment. ProTip – for East Coasters, schedule your test first thing in the morning before the West starts working – much more responsive!
  2. It will attempt to confuse you. Take a deep breath and look at the overall question – what are you trying to accomplish?
    1. Also, sometimes the simplest answer is all you need. Don’t overcomplicate things (I have a tendency to do this).
  3. Do not spend too much time on a specific problem. If it’s not working as expected, make a note of it and move on.
    1. This was my problem with the first attempt. I spent too much time on a specific set of questions and ran out of time.
  4. Stay true to the Blueprint. If you practice and study using its guidelines, you will be okay.

First off, I wrote the entire Blueprint on my whiteboard with the main tasks. Excuse my handwriting, but you get the gist of what I was trying to accomplish here. I notated how many times I did a specific item.

After I didn’t make it the first attempt, I circled the sections I missed in black (even though half of them were because I just ran out of time). These were areas I focused on for the second attempt.

Focus Areas

  1. Lab, lab, and lab. This is a test you cannot dump your way through. I don’t see how it would be possible to pass this without hands-on experience, especially with routing protocols and how NSX interacts with them.
    1. Labs of Focus
      1. The VMware Hands-On Labs (HOL) is FREE! You would be crazy for not using these and getting some insight/guidance on how things work. I primarily used the 1803 and 1825 labs on VMware HOL Рlink here.
      2. VMware Education – NSX Install, Configure, Manage – Lab Connect.
        1. I thought this was a great, self-paced, approach to a vPOD environment with 22 guided labs. There is a cost to using this, but I believe there may discounts for Partners.
        2. VMware Education – Install, Configure Manage – Lab Connect Link
      3. Josh Andrews’ VCIX6-NV Practice Exam
        1. I thought this was a very nice addition from SOSTech / Josh Andrews.
        2. Josh provided 9 sample “test” questions using the 1703 (or 1803 which is the new version) lab.
        3. Definitely test yourself and document where you may struggle.
        4. Link to the SOSTech VCIX6 Practice Exam
    2. Existing Blog Guides
      1. I thought Clinton’s VCAP6-NV was the most comprehensive guide out there. I used this as a rule of thumb and mimicked many of the things he documented in his blog series. DEFINITELY spend some time here.¬†¬†
      2. Other Guides I went through:
        1. Chestin Hay / v4Real – Link
        2. – Study Guide ( a little dated, but still pertinent info)
        3. thecloudexpert / Chris Lewis’ guide – Link
      3. There are probably others I’m missing, but definitely leverage what’s out in the community – thank you to those of you that have published and spent a significant amount of time documenting!
  2. Practice and get involved. Read all of the publications out there on VMware’s site. Build out designs for customers. This will take time but provides further exposure and compounding of the topics.
  3. Break stuff. You need to see how NSX works when you start “pulling cables” if you will. I spent countless hours just doing this. There are several troubleshooting topics so you’ll need to understand what happens when things go awry.

What’s next? I will continue to focus on VMware NSX, but may attempt one of the VCAP-DCV’s. I hope this post will benefit others – cheers!



VMware NSX Reminder – With ECMP, no Edge Firewall!

This was something I ran into a week or so ago in an NSX design Рobviously not thinking right!

As a friendly reminder, disable the Edge firewall if you will be using ECMP mode on VMware NSX! There isn’t any message or warning if you enable ECMP mode with the Edge Firewall still on.

Here’s my understanding – since the firewall is a stateful service (this also applies to NAT/Load Balancing), it cannot work with asymmetric routing. For example, the 2nd Edge cannot be aware of a session that was started on the 1st edge (no SYN), so the traffic is dropped.

In my testing, it seems this impacts traffic traversing North to South, but routing South to North seems to work.

I did a quick video of my testing with my current lab environment to depict the results I see – which is the loss of network connectivity and pings from another routed segment.