Prepare Virtual Switches for Nested ESXi and VyOS
Prepare virtual switches for nested ESXi and VyOS. Follow a homelab setup with a physical uplink port group and a separate internal switch.
Overview
This nested ESXi virtual-switch setup prepares the networks for a VyOS router between my physical and nested lab environments. I use standard switches on an HP DL360 Gen9, with an uplink port group and a separate internal switch.
Routed topology and security policy
Physical upstream LAN
| physical NIC uplink
Physical ESXi vSwitch0 — VyOS-WAN port group
| VyOS WAN interface
[ VyOS router: Layer 3 forwarding / optional NAT ]
| VyOS LAN interface
Internal standard switch — no physical NIC uplink
|
Nested ESXi VMs and their lab networksWAN means the router’s upstream side in this diagram; it does not necessarily mean a public Internet address. The LAN switch has no direct physical uplink. Return routes or the intended NAT policy must exist for traffic to traverse VyOS in both directions.
For this routed path, the WAN emits the router’s MAC address; the number of nested guests is not a reason to accept arbitrary MAC changes there. Nested ESXi carrying other guests’ MAC addresses is a different requirement on the nested-facing port group. Scope any promiscuous/forged-transmit or MAC-learning configuration to that isolated group and validate it with the actual topology.
Broadcom describes the security tradeoffs of accepting these policies in its port-group security guidance. Do not enable all three globally or rely on an unmeasured performance claim.
Plan the physical and nested networks
Just for simplicity, I will be using standard vswitches on my physical host since I will be using 1 HP DL360 gen9 to host all the nested esxi systems. The first configuration will be to add a virtual port group to the current switch which will act as the uplink port for my VyOS router.
Create the WAN port group
In the HP esxi host click on the hostname and select configure.
Click on Virtual Switches under Networking
As you can see I have 2 virtual ports already created, these are the defaults that get created during installation of esxi. Click on Add Networking..
Under select connection type, click the radio button next to Virtual Machine Port Group for a Standard Switch and click Next
Click the radio button for vSwitch0 on the Select target device screen and click Next
On the Connection settings screen, provide a network label and vlan ID for the uplink network on your VyOS Router. Then click Next
At the Ready to complete screen, review your settings and click Finish
Create the internal switch
This will create the virtual port group that will serve as the uplink to the physical network. Now we need to create another virtual switch that will be the nested side of the VyOS router.
Back at the Virtual switches screen, click Add Networking.. again
Select the radio button next to Virtual machine Port Group for a Standard Switch again and click Next
This time we are going to select New standard switch as the target device and for MTU enter in 9000. Then click Next
At the Create a Standard Switch screen, leave all the vmnics unclaimed and click Next
This will give you a warning that no active adapters are on this switch, just click Ok to continue
At Connection Settings, enter a name for your network label and click Next.
At the Ready to complete screen, review your changes and click Finish
Review port-group security
Review the security policy on the correct port group before changing it. VyOS routes between the WAN and LAN; guest MAC forwarding on a nested-facing port group is a separate question from the WAN router interface.
At the Virtual Switches screen, click the 3 dots next to that VyOS virtual port and select Edit Settings.
Click on Security in the left hand side of the window
The original screenshot shows MAC Address Changes set to Accept on the VyOS port group. Do not copy that setting solely because nested guests exist. Keep the WAN policy restrictive unless the router configuration has a demonstrated requirement.
For nested ESXi forwarding guest MAC addresses, determine the need for MAC learning or scoped promiscuous/forged-transmit settings on the nested-facing network. Test connectivity in both directions and document the exception. No performance penalty was measured in this walkthrough.
In my next post I will walkthrough the process of deploying a VyOS router.