VXLAN Overview
VXLAN is a Layer 2 overlay scheme over a Layer 3 network. Overlays are called VXLAN segments and only VMs and physical machines (tenents) within the same segment have Layer 2 connectivity. VXLAN segments are uniquely identified using an identifier called the VXLAN Network Identifier (VNI). The VNI is a 24-bit identifier; therefore, an administrative domain can support up to 16 million overlay networks.
As the scope of the MACs originated by tenants is restricted by the VNI, overlapping MAC addresses across segments can be supported without traffic leaking between tenant segments. When a tenant frame traverses a VXLAN overlay network, it is encapsulated by a VXLAN header that contains the VNI. This frame is further encapsulated in a UDP header and L2/L3 headers.
VXLAN can add up to a 54-byte header to the tenant VM‘s frame. For VXLAN to work correctly, this requires that the IP MTU be set to at least 1554 bytes on the network-side interfaces. IP MTU of 1554 should also be set on all transit nodes which carry VXLAN traffic. The point at which a tenant frame is encapsulated (or decapsulated) is referred to as a VXLAN Tunnel Endpoint (or VTEP). VTEPs are typically located on hypervisors but may also be located on physical network switches. Network switches that act as a VTEP are referred to as VXLAN gateways.
The role to encapsulate/decapsulate a frame is performed by a VXLAN Tunnel Endpoint (VTEP), also referred to as a VXLAN gateway. A VXLAN gateway can be a Layer 2 gateway or Layer 3 gateway depending on its capacity. A Layer 2 gateway acts as a bridge connecting VXLAN segments to VLAN segments. A Layer 3 gateway performs all that of Layer 2 gateway, and capable of routing traffic between tenant VLANs/VMANs.
At tunnel initiation, a gateway looks up the destination MAC address of the frame received from the tenant VM. If the MAC address to remote VTEP IP binding is known, the gateway adds the VXLAN header and the IP/UDP header to the frame and forwards toward the DC network. A gateway node that terminates a tunnel removes the encapsulation headers from the packet and determines the bridge domain of the inner frame by examining the VXLAN network identifier (VNID) received in the VXLAN header. The gateway then looks up the inner MAC destination address (DA) in the tenant VLAN's/VMAN's filtering database and decides either to flood or forward the frame to tenant ports.
The VXLAN segments with the same virtual network ID form a virtual network with one Ethernet broadcast domain.

Note
Universal switches support Layer 3 gateways.Supported Platforms
VXLAN is supported on ExtremeSwitching Universal switches, and stacks with Universal slots only.
Limitations
The following capabilities are not supported in ExtremeXOS:
- Ability to assign more than one VNI to a virtual network.
- IPv6 addresses for local and remote VTEPs.
- Assigning source IP for VXLAN gateway encapsulation:
- Per virtual router
- Per virtual network or VNI
- Support for adding more than one tenant VLAN/VMAN per VNI.
- A physical port being part of both a tenant VLAN/VMAN and an underlay (Network) VLAN.
- Support for heterogeneous stack environments where at least one of the stack nodes is not VXLAN capable.
- More than one next-hop on a (network) port.
- Multicast underlay IP network, except PIM-SSM.
- Multiple VRs.
- With VXLAN for 5320 and 5420 series switches, MACSec-enabled shared ports cannot be used as a VXLAN tenant/network port.
Interactions with Existing ExtremeXOS Features
Feature/Capability | Tenant Network | Underlay Network | Rest of the Switch |
---|---|---|---|
VLAN with: Multiple C-Tags on the same port or different C-Tags on different ports | Future | Future | No new restrictions |
Port-Specific Tag (PSTag) | ExtremeXOS 31.2 or later | Not supported | No new restrictions |
MAC-based and Protocol-based VLANs | Not supported | Not supported | No new restrictions |
VMANs | Supported | Not supported | No new restrictions |
CEP | Not supported | Not supported | No new restrictions |
Configuring LAG on ports (static and LACP) | ExtremeXOS 21.1 or later | ExtremeXOS 21.1 or later | No new restrictions |
Configuring MLAG on ports | ExtremeXOS 21.1 or later | ExtremeXOS 21.1 or later | ExtremeXOS 21.1 or later |
Limit learning and MAC locking | ExtremeXOS 31.2 or later | Not supported | ExtremeXOS 21.1 or later |
Configuring IP (v4/v6) address on a VLAN | ExtremeXOS 21.1 or later | ExtremeXOS 21.1 or later | No new restrictions |
Enabling IP and IPMC forwarding |