Fabric Extend Considerations

Review the following restrictions, limitations, and behavioral characteristics that are associated with Fabric Extend.

Note

If your Fabric Extend configuration includes a VSP 4450 Series/ONA combination, see ONA Considerations for more information.

Tunnel source IP

Fabric Extend supports the tunnel source IP address using a brouter port interface, a CLIP IP, or a VLAN IP.

The following table shows lists the product support.


Product	Tunnel source IP
Product	Brouter port	CLIP IP	VLAN IP
VSP 4450 Series	Yes	Yes	Yes
VSP 4900 Series	Yes	Yes	Yes
5520 Series	Yes	Yes	Yes
VSP 7200 Series	Yes	Yes	Yes
VSP 7400 Series	Yes	Yes	Yes
VSP 8200 Series	Yes	Yes	Yes
VSP 8400 Series	Yes	Yes	Yes
VSP 8600 Series	Not supported	Not supported	Not supported
XA1400 Series	Yes	Yes	Yes

Extreme Fabric Orchestrator (EFO) —The Fabric Extend view within EFO is not required, but should be used.
Tunnel failover time—With IS-IS interface default values, tunnel failure detection can take up to 27 seconds. You can reduce the IS-IS interface hello timers to speed up logical link failure detection, but be careful to avoid link flapping due to values that are too low.
Note

If the number of IS-IS interfaces on a node is greater than 100, it is a good practice to set the hello timer not lower than 5 seconds.
ACL Filters over VXLAN—IP filters configured to match IP header fields in the headers of VXLAN encapsulated packets, work only when the switch acts as a transit router and does not participate in the initiation or termination of VXLAN traffic.
VLACP—VLACP is not supported over logical IS-IS interfaces.
CFM CCM—CFM Continuity Check Messages are not supported over logical IS-IS interfaces.
CFM traceroute and tracemroute—If CFM packets transit over a layer 3 tunnel (that is the CFM packets ingress a Fabric Extend layer 3 core tunnel and egress through another layer 3 core tunnel), the transit SPBM nodes do not display as intermediate hops in the output for CFM l2 traceroute and l2 tracemroute.

This is because the CFM packets are encapsulated in the outer layer 3 header as part of VXLAN encapsulation, and the transit SPBM nodes cannot look into the payload of the VXLAN packet and send a copy of the CFM packet to local CPU for processing.
CFM L2 ping—CFM L2 ping to MCoSPB source mac is not supported and can fail if they are reachable via Fabric Extend tunnel.
MACsec—Switch-based MAC Security (MACsec) encryption is Layer 2 so it cannot be used with Fabric Extend IP, which is Layer 3.
MTU minimum in Layer 2 Pseudowire core networks—Service provider Layer 2 connections must be at least 1544 bytes. In this type of deployment the tunnels are point-to-point VLAN connections that do not require VXLAN encapsulation. The default MTU value is 1950.
Logical IS-IS interfaces—Layer 2 core and Layer 3 core logical IS-IS interfaces are not supported on the same switch at the same time.
Fragmentation/reassembly—There is no fragmentation/reassembly support in Layer 2 core solutions.
If a tunnel was initially UP between a VSP 4450 Series and another VOSS switch with MTU 1950 and then the VSP 4450 Series was later configured for fragmentation, the following behavior occurs:
- If the ONA MTU is less than 1594, the tunnel to the VOSS switch will go DOWN.
- If the ONA MTU is 1594 and above, the tunnel will stay UP, but any fragmented packets received from the VSP 4450 Series will be lost at the VOSS switch site.
Fragmented traffic can only be sent with an XA1400 Series or VSP 4450 Series/ONA combination on both ends with the same MTU configured on each end.
RFC4963 and RFC4459 considerations:

The ONA 1101GT provides for the IP MTU of the Network port to be reduced from the default setting of 1950 bytes to 1500 bytes or lower. The MTU reduction feature with Fabric Extend is provided to facilitate the connection of two Fabric Connect networks over an IP network with any MTU without requiring end stations on the networks to reduce their MTU. The ONA 1101GT with the IP MTU of the network port set to 1500 bytes will fragment Fabric Extend VXLAN tunnel packets exceeding 1500 bytes. The ONA 1101GT will also reassemble fragmented Fabric Extend VXLAN tunnel packets at the tunnel termination point. The IP fragmentation and reassembly RFC 791 describes the procedure for IP fragmentation, and transmission and reassembly of datagrams and RFC4963 and RFC4459 detail limitations and network design considerations when using fragmentation to avoid out of order packets and performance degradation.
Factors that can impact performance are —
- The link speed per VXLAN IP address should be slower than 1G to avoid reassembly context exhaustion.
- ECMP and link aggregation algorithms in the IP core should be configured not to use UDP port hashing that could send IP fragments after the first fragment on different paths causing out of order packets. This is due to the fact that subsequent fragments do not have UDP port information.
Important

Different MTU sizes on each end can result in traffic drops.

Layer 2 logical IS-IS interfaces—Layer 2 logical IS-IS interfaces are created using VLANs. Different Layer 2 network Service Providers can share the same VLAN as long as they use different ports or MLT IDs.

Note

Exception: Layer 2 logical IS-IS interfaces are not supported on XA1400 Series.
MTU minimum in Layer 3 core networks—Service provider IP connections must be at least 1594 bytes to establish IS-IS adjacency over FE tunnels. The 1594 bytes includes the actual maximum frame size with MAC-in-MAC and VXLAN headers. If this required MTU size is not available, a log message reports that the IS-IS adjacency was not established. MTU cannot be auto-discovered over an IP tunnel so the tunnel MTU will not be automatically set. The default MTU value is 1950.

If the maximum MTU size has to be fewer than 1594 bytes, then you require fragmentation and reassembly of packets. The XA1400 Series and VSP 4450 Series/ONA combination supports fragmentation and reassembly, but you must have either an XA1400 Series or VSP 4450 Series with ONAs at BOTH ends of the IP WAN connection.
IP Shortcuts—The tunnel destination IP cannot be reachable through an IP Shortcuts route.

Important

If you enable IP Shortcuts and you are using the GRT as the tunnel source VRF, you must configure an IS-IS accept policy or exclude route-map to ensure that tunnel destination IP addresses are not learned through IS-IS.

If you enable IP Shortcuts and you are using a VRF as the tunnel source VRF, this is not an issue.
Layer 3 over Layer 2 Limitation—The VOSS switches require a single next hop (default gateway) for all tunnels.
- Over a layer 3 core network, on a given outgoing port or MLT, there is no issue as the one router next hop can support multiple VXLAN tunnels to one or more remote sites.
- For layer 3 tunneling over a layer 2 core, the VOSS switch without any specific configuration supports only one Fabric Extend tunnel to one remote site. The workaround for this single next hop issue is to create an additional VRF, VLAN, and loopback interface.
For a configuration example of this workaround, see Shortest Path Bridging (802.1aq) Technical Configuration Guide.

Note

This limitation does not apply to VSP 4450 Series.
You cannot establish a Virtual IST (vIST) session over a logical IS-IS interface. IST hellos cannot be processed or sent over a logical IS-IS interface if that is the only interface to reach BEBs in vIST pairs.

Assume that vIST is established over a regular NNI interface and the NNI interface goes down. If the vIST pairs are reachable through a logical IS-IS interface, then the vIST session goes down in up to 240 seconds (based on the IST hold down timer). During this time, the error message IST packets cannot be sent over Fabric Extend tunnels, vist session may go down is logged.

Caution

Expect traffic loss when the vIST session is down or when the error message is being logged.
Port Mirroring Resources:

Port mirroring resources are limited to four ports simultaneously (where each mirroring direction counts as one). For example, if two mirroring ports are designated to mirror both ingress and egress traffic then all four mirroring ports are consumed.
Port mirroring shares these four resources with other applications such as port mirroring RSPAN, Fabric Extend, Application Telemetry, IPFIX, and ACL with mirror action. Each one of these applications consumes at least one port mirroring resource. (port mirroring RSPAN consumes two if you configure both Ingress and Egress modes.)
Important
- To enable any one of the above applications, you must have at least one free mirroring resource. If all four port mirroring resources are already in use, the switch displays a Resource not available error message when you try to enable the application.
- The VSP 8600 uses the four reserved resources for port mirroring and ACLs that have a mirroring action. For the other applications, this restriction does not apply because the VSP 8600 uses mirroring resources that do not come out of the four reserved port mirroring resources.
Fabric Extend over IPsec limitations
- Fabric Extend over IPsec is only supported on XA1400 Series devices.
- Only pre-shared authentication key IPsec parameters are user configurable. Other, third-party solutions are not configurable.
- IKEv2 protocol key exchange only.
- IPsec support is only added for Fabric Extend tunnels.
- IPsec is not supported for regular layer 3 routed packets.