Following the subject of previous post, I will continue with NSX routing LAB topology used by me in my NSX learning. LAB topology diagram are inserted below.
(click on picture for hi resolution image)
A brief description for lab topology:
- There are tree VXLAN logical switches: 5001, 5002 and 5003 corresponding to segments in a formal three-tier application (WEB/APP/DB). Each segment have a related subnet network: 10.128.20.0/24 for LS5003 (WEB), 10.128.21.0/24 for LS5002 (APP) and 10.128.22.0/24 for LS5001 (DB).
- Inter VXLAN routing are provided by a Distributed Logical Router (DLR) and an Edge Services Gateway (ESG). DLR act as a default gateway for DB and APP segments and ESG provide routing for WEB network segment. First IP from each subnet are reserved for default gateway.
- Both DLR and ESG are configured in HA configuration mode (active-standby) – that is why these services are represented by two VMs each (vShield Edge for ESG and Control VM for DLR) – x per Edge ESXi host.
- A group of two ESG appliances configured for Equal Cost Multi Path (ECMP) routing provides upstream communications. ECMP are enabled as well for DLR and ESG routers.
- All routers are interconnected trough a transit logical switch 5004. Routers in this segment will use IP addresses from 10.128.10.0/24 subnet: .5 for ESG, .254 and .253 for each of upstream routers. The DLR router use two IP addresses: first (.2) as a forwarding address and second (.3) as a control plane address (protocol address).
- DLR, ESG and upstream routers exchange routing information via OSPF. All routers forms OSPF neighbor relationships.
- ESG router are configured with both interfaces (internal and uplink) as part of the OSPF process (mapped to OSPF Area 0). DLR router have OSPF activated only on uplink interface and configured to redistribute in OSPF directly connected networks (APP and DB).
- Upstream routers use OSPF on its internal interfaces and form BGP peering with external physical routers via its uplink interfaces.
- External communications take place trough a dedicated dvPG (EDGE) with uplinks via physical NICs – DirectPath IO pass-through from physical host to EDGE nested ESXi.
- Upstream ESG routers will redistribute BGP routes in OSPF and conversely OSPF to BGP.
Presented below, are the configurations applied on physical routers (RTR1/2). VLAN 100 are used to interconnect with ESG upstream routers (EDGE/10.128.30.0/24 subnet) and VLAN 102 to reach next hop on Mikrotik router (subnet 192.168.1.0/24). For Internet access, a NAT overload function is configured on Mikrotik network appliance.
After all the configurations applied, we can check the routing protocol adjacencies and routing table information. For instance, here’s what DLR routers showing us:
OSPF adjacencies with ESG router (.5) and upstream routers (.253 and .254)
Respectively, the routing table:
We can observe two routes learned via OSPF: one to 10.128.20.0/24 (WEB) learned from ESG routers (.5) and a default route 0.0.0.0/0 as external redistribute route. All other routers are directly connected type. Btw, 172.16.22.121 is the IP address of the DLR’s router management interface and 169.254.1.8/30 are the automatically assigned network used for heartbeating between DLR router failover peers.
BGP peering can be checked on one of the upstream ESG router:
and corresponding, all known networks:
It’s interesting to verify the routes installed in routing table on physical routers. Bellow, are shown the routes on RTR1 physical routers.
Finally, we can check the connectivity to some internet resources from one of the VM connected to internal logical switch (5002/APP):