VXLAN Terminology

General:

========

Underlay:

• Existing L2/L3 network

Overlay:

• Fabric built over the Underlay
• L2 over L3

VNI/VNID:

• Virtual Network Identifier

L2VNI:

• Layer2 Virtual Network Identifier
• Identifies a L2 vlan/broadcast domain within the fabric
• Only extends L2 across fabric
• No routing

L3VNI:

• Layer3 Virtual Network Identifier
• Needed to route traffic in VXLAN fabric
• Each tenant VRF will need a unique L3VNI
• Same VRF-to-L3VNI mapping must exist on all VTEPs

IP forward:

• Required configuration under L3VNI SVI configuration
• Punts decapsulated packet to CPU (e.g. ARP)
• Forwards decapsulated packet down to host

Infra-vlan:

• Used for back up routing between VPC VTEPs in underlay
• Only needed on Cloudscale ASICs

Arp-suppression:

• Reduces the amount of broadcasts in a VXLAN segment
• Local VTEP will keep arp cache and respond to ARP request from local host if it has entry for remote host already, if not, it will flood out.
• Cannot be configured on VTEP where VNI is pure L2 (no SVI)

NVE:

• Network Virtual Interface
• VXLAN Tunnel Endpoint
• Performs encapsulation and decapsulation for VXLAN

BUM Traffic:

• Broadcast, Unknown Unicast, Multicast traffic

Advertise PIP/Virtual RMAC:

• Used on VPC VTEPs to advertise prefixes with the primary IP (PIP) of the NVE instead of the shared/secondary IP (VIP)
• Also advertises prefix with system-specific router mac address of the PIP instead of the system-specific router mac of the VIP

Node Types:

================

Spine:

• Connects VTEPs and Border VTEPs
• Typically a transit device for VXLAN encapsulated traffic (East/West)
• Route-Reflector (RR) for EVPN
• Rendezvous-Point (RP) for Underlay

VTEP(leaf):

• VXLAN Tunnel Endpoint
• VXLAN Edge Device
• Encapsulates CE traffic into VXLAN

Border Leaf:

• Provides External Connectivity
• VXLAN Edge-Device
• Routes traffic from an outside network and encapsulates it into VXLAN (North/South)

Border Gateway:

• Connects fabrics together
• Ingress replication used
• Does not have hosts connected to it

Border Spine:

• Acts as Spine as well as VTEP
• Can route traffic from an outside network into fabric (North/South)

Psuedo Border Gateway (need to validate):

• For legacy sites migrating to VXLAN
• Will allow for legacy site to connect to fabric

NDFC switch roles:

Switch Role	Description
Spine	Spine switches provide Layer-3 underlay inter-connection between leaf switches as well as BGP EVPN control plane functions. They form the backbone of the network and connect to leaf switches, but not directly to each other. This design helps minimize latency and ensures a more predictable and consistent performance across the network.The Cisco Nexus 9000 series can act as spine and leaf switches, but choosing this switch depends on the specific model and network design requirements.
Leaf	A Virtual Tunnel Endpoint (VTEP) for providing Layer-2 / Layer-3 connectivity point for workloads and Layer 4 to Layer 7 services. Leaf switches connect directly to servers and storage devices within the data center. In a spine-and-leaf setup, every leaf switch is connected to every spine switch, ensuring multiple paths for data to travel.Leaf switches provide VXLAN encapsulation and decapsulation and Anycast Gateway services. Endpoints can be connected using individual, port-channel or virtual port-vhannel interfaces.
Border	A VTEP acting as handoff point across VXLAN and IP domains. A border switch in a network typically refers to a device that connects the internal network to external networks. In data centers, this can mean connecting the internal fabric to external networks, such as other data centers, the internet, or enterprise networks.Typically used for VRF-LITE and MPLS North-to-South connectivity. Optionally, endpoints and Layer 4 to Layer 7 services can also be connected.
Border spine	Provides VXLAN VTEP and EVPN control-plane functions at the same time. Supports all functions that are natively provided by both a spine switch and a border switch. See the Support for Super Spine Switch Role for more information.
Border gateway	Border gateway generally refers to a router or switch that participates in routing protocols to manage data flow between different network domains. It plays a crucial role in determining the best path for data to travel. A border gateway provides the same function as a border but adds the ability to extend VXLAN tunnels to remote fabrics for VXLAN multi-site fabrics.Functions include VXLAN packet re-origination and re-writes for Layer-2/Layer-3 extensions. Supported as Anycast or VPC.
Border gateway spine	Supports all functions that are natively provided by both a spine switch and a border gateway. Only Anycast border gateway is supported when merged with a spine switch. See the Support for Super Spine Switch Role for more information.
Super spine	A super spine is an additional layer of spine switches used in very large data center networks. This layer sits above the regular spine layer in a multi-tier architecture and acts as a backbone for connecting multiple spine-and-leaf pods, effectively interconnecting them to create a larger, cohesive network. A super spine connects multiple groups of spine and leaf switches within a single VXLAN fabric. It helps inter-connect multiple spine layers to achieve full CLOS architecture.When spine and super spine switches are present in the same fabric, the EVPN control-plane functions are handled at the super spine layer while the spine acts as a Layer-3 transit.
Border super spine	Supports all functions that are natively provided by both a border and a super spine. See the Support for Super Spine Switch Role for more information.
Border gateway super spine	Supports all functions that are natively provided by both a border gateway and a super spine. Only Anycast border gateway is supported when merged with a super spine. See the Support for Super Spine Switch Role for more information.
Access	The access switch is used at the bottom layer in a traditional three-tier network architecture. It serves as the entry point for hosts (VMs) and end devices such as computers, printers, and IP phones to connect to the network. It provides Layer-2 connectivity for workloads in Classic Ethernet networks. Endpoints can be connected using individual, port-channel or virtual port-channel interfaces.
Aggregation	The aggregation switches serve as an intermediary between the core network (which handles high-speed data transport) and the access layer. It consolidates data from multiple access switches before forwarding it to the core layer, reducing the number of direct connections to the core. It provides Layer-3 gateway and FHRP services in Classic Ethernet networks. Additional functions include connecting Layer 4 to Layer 7 services and external IP domains.
Core router	The core router is the topmost layer in a traditional three-tier network architecture. It provides fast and reliable data transport across the network, connecting different distribution (aggregation) layers and ensuring seamless communication between various parts of the network.The core layer is designed for high-speed data transmission, ensuring data can travel quickly and efficiently across the network. It provides Layer-3 external IP inter-connectivity (ISN) across different domains. Typically used as an EVPN route server in VXLAN multi-site fabrics or as an MPLS-P router.
Edge router	The edge router is a specialized router located at a network boundary that connects an internal network to external networks, such as the internet or a wide area network (WAN). Its primary role is to manage data traffic between the internal network and external networks, ensuring efficient and secure data flow.An edge router provides Layer-3 external IP inter-connectivity across different domains, such as VXLAN and Classic Ethernet networks. Common inter-connectivity includes VRF-Lite.
Top of Rack (ToR)	A ToR switch connects to the servers within the same rack through short, direct connections, which reduces cabling complexity and enhances performance. The ToR switch aggregates traffic from all the servers in the rack and uplinks it to higher-level switches or routers, such as spine switches in a spine-and-leaf architecture.A ToR switch provides Layer-2 only connectivity for endpoints. Endpoints can be connected using individual, port-channel or virtual port-channel interfaces. A ToR role is supported for both VXLAN and Classic LAN networks. For VXLAN-based fabrics, a ToR is connected to the leaf switch.

Route-types:

============

Type1: Ethernet Auto-Discovery (A-D) route

• BGP based Multi-homing
• Mass Withdraw/Aliasing

Type2: MAC/IP advertisement route

• provides host reachability information
• L2VNI MAC or MAC-IP from L2 MAC Learning or ARP

Type3: Inclusive Multicast Route

• Dynamic Peer Discovery for EVPN Ingress Replication

Type4: Ethernet Segment Route

• BGP Based Multi-homing
• BUM DF election

Type5: IP Prefix-route

• Advertises IP Prefixes into fabric
• Source Active A-D route (TRM)
• Originated by VTEP where Source is directly connected (FHR)

Type6 Selective Multicast Ethernet Tag Route:

• Translates IGMP, (*,G), (S,G) join in TRM L2 Mode VTEP
• Shared tree join route
• Originated by VTEP where receiver is connected (LHR)

Type7 – IGMP Sync routes

• Source tree join route
• Originated by VTEP where receiver is connected (LHR)

BUM Traffic Forwarding:

===================

Multicast Replication:

• BUM traffic is sent to the multicast group for the VNI

Ingress Replication:

• Used where there is no multicast in the underlay
• BUM traffic is replicated to each VTEP via Unicast

Static Ingress Replication:

• Used wheree there is no multicast in the underlay
• Remote peers are statically defined
• Tunnel is up as long as peer is reachable