This was something I ran into a week or so ago in an NSX design – obviously not thinking right!
As a friendly reminder, disable the Edge firewall if you will be using ECMP mode on VMware NSX! There isn’t any message or warning if you enable ECMP mode with the Edge Firewall still on.
Here’s my understanding – since the firewall is a stateful service (this also applies to NAT/Load Balancing), it cannot work with asymmetric routing. For example, the 2nd Edge cannot be aware of a session that was started on the 1st edge (no SYN), so the traffic is dropped.
In my testing, it seems this impacts traffic traversing North to South, but routing South to North seems to work.
I did a quick video of my testing with my current lab environment to depict the results I see – which is the loss of network connectivity and pings from another routed segment.