Fabric Extend Considerations

Review the following restrictions, limitations, and behavioral characteristics that are associated with Fabric Extend.
Note

Note

If your Fabric Extend configuration includes a VSP 4450 Series/ONA combination, see ONA Considerations for more information.

Tunnel Source IP

Fabric Extend supports the tunnel source IP address using a brouter port interface, a CLIP IP, or a VLAN IP.

The following table shows the product support.

Product Tunnel source IP
Brouter port CLIP IP VLAN IP
5420 Series Yes Yes Yes
5520 Series Yes Yes Yes
VSP 4450 Series Yes Yes Yes
VSP 4900 Series Yes Yes Yes
VSP 7200 Series Yes Yes Yes
VSP 7400 Series Yes Yes Yes
VSP 8200 Series Yes Yes Yes
VSP 8400 Series Yes Yes Yes
VSP 8600 Series Not supported Not supported Not supported
XA1400 Series Yes Yes Yes

Tunnel Failover Time

With IS-IS interface default values, tunnel failure detection can take up to 27 seconds. You can reduce the IS-IS interface hello timers to speed up logical link failure detection, but be careful to avoid link flapping due to values that are too low.

Note

Note

If the number of IS-IS interfaces on a node is greater than 100, it is a good practice to set the hello timer not lower than 5 seconds.

ACL Filters over VXLAN

IP filters configured to match IP header fields in the headers of VXLAN encapsulated packets, work only when the switch acts as a transit router and does not participate in the initiation or termination of VXLAN traffic.

VLACP

VLACP is not supported over logical IS-IS interfaces.

CFM CCM

CFM Continuity Check Messages are not supported over logical IS-IS interfaces.

CFM traceroute and tracemroute

If CFM packets transit over a layer 3 tunnel (that is the CFM packets ingress a Fabric Extend layer 3 core tunnel and egress through another layer 3 core tunnel), the transit SPBM nodes do not display as intermediate hops in the output for CFM l2 traceroute and l2 tracemroute.

This is because the CFM packets are encapsulated in the outer layer 3 header as part of VXLAN encapsulation, and the transit SPBM nodes cannot look into the payload of the VXLAN packet and send a copy of the CFM packet to local CPU for processing.

CFM L2 Ping

CFM Layer 2 ping to MCoSPB source mac is not supported and can fail if they are reachable via Fabric Extend tunnel.

MACsec

Switch-based MAC Security (MACsec) encryption is Layer 2 so it cannot be used with Fabric Extend IP, which is Layer 3.

MTU Minimum in Layer 2 Pseudowire Core Networks

Service provider Layer 2 connections must be at least 1544 bytes. In this type of deployment the tunnels are point-to-point VLAN connections that do not require VXLAN encapsulation. The default MTU value is 1950.

Logical IS-IS Interfaces

Layer 2 core and Layer 3 core logical IS-IS interfaces are not supported on the same switch at the same time.

Fragmentation and Reassembly

There is no fragmentation and reassembly support in Layer 2 core solutions.

If a tunnel was initially UP between a VSP 4450 Series and another VOSS switch with MTU 1950 and then the VSP 4450 Series was later configured for fragmentation, the following behavior occurs:
  • If the ONA MTU is less than 1594, the tunnel to the VOSS switch will go DOWN.

  • If the ONA MTU is 1594 and above, the tunnel will stay UP, but any fragmented packets received from the VSP 4450 Series will be lost at the VOSS switch site.

RFC4963 and RFC4459 Considerations

The ONA 1101GT provides for the IP MTU of the Network port to be reduced from the default setting of 1950 bytes to 1500 bytes or lower. The MTU reduction feature with Fabric Extend is provided to facilitate the connection of two Fabric Connect networks over an IP network with any MTU without requiring end stations on the networks to reduce their MTU. The ONA 1101GT with the IP MTU of the network port set to 1500 bytes will fragment Fabric Extend VXLAN tunnel packets exceeding 1500 bytes. The ONA 1101GT will also reassemble fragmented Fabric Extend VXLAN tunnel packets at the tunnel termination point. The IP fragmentation and reassembly RFC 791 describes the procedure for IP fragmentation, and transmission and reassembly of datagrams and RFC4963 and RFC4459 detail limitations and network design considerations when using fragmentation to avoid out of order packets and performance degradation.

The following list identifies factors that can impact performance:
  • The link speed per VXLAN IP address should be slower than 1G to avoid reassembly context exhaustion.

  • ECMP and link aggregation algorithms in the IP core should be configured not to use UDP port hashing that could send IP fragments after the first fragment on different paths causing out of order packets. This is due to the fact that subsequent fragments do not have UDP port information.

Important

Important

Different MTU sizes on each end can result in traffic drops.

Layer 2 Logical IS-IS Interfaces

Layer 2 logical IS-IS interfaces are created using VLANs. Different Layer 2 network Service Providers can share the same VLAN as long as they use different ports or MLT IDs.

Note

Note

Exception: Layer 2 logical IS-IS interfaces are not supported on XA1400 Series.

MTU Minimum in Layer 3 Core Networks

Service provider IP connections must be at least 1594 bytes to establish IS-IS adjacency over FE tunnels. The 1594 bytes includes the actual maximum frame size with MAC-in-MAC and VXLAN headers. If this required MTU size is not available, a log message reports that the IS-IS adjacency was not established. MTU cannot be auto-discovered over an IP tunnel so the tunnel MTU will not be automatically set. The default MTU value is 1950.

If the maximum MTU size has to be fewer than 1594 bytes, then you require fragmentation and reassembly of packets. The XA1400 Series and VSP 4450 Series/ONA combination supports fragmentation and reassembly, but you must have either an XA1400 Series or VSP 4450 Series with ONAs at BOTH ends of the IP WAN connection.

IP Shortcuts

The tunnel destination IP cannot be reachable through an IP Shortcuts route.

Important

Important

If you enable IP Shortcuts and you are using the GRT as the tunnel source VRF, you must configure an IS-IS accept policy or exclude route-map to ensure that tunnel destination IP addresses are not learned through IS-IS.

If you enable IP Shortcuts and you are using a VRF as the tunnel source VRF, this is not an issue.

Layer 3 over Layer 2 Limitation

Port Mirroring Resources

Port mirroring resources are limited to four ports simultaneously (where each mirroring direction counts as one). For example, if two mirroring ports are designated to mirror both ingress and egress traffic then all four mirroring ports are consumed.

Port mirroring shares these four resources with other applications such as port mirroring RSPAN, Fabric Extend, Application Telemetry, IPFIX, and ACL with mirror action. Each one of these applications consumes at least one port mirroring resource. (port mirroring RSPAN consumes two if you configure both Ingress and Egress modes.)
Important

Important

  • To enable any one of the above applications, you must have at least one free mirroring resource. If all four port mirroring resources are already in use, the switch displays a Resource not available error message when you try to enable the application.

  • The VSP 8600 uses the four reserved resources for port mirroring and ACLs that have a mirroring action. For the other applications, this restriction does not apply because the VSP 8600 uses mirroring resources that do not come out of the four reserved port mirroring resources.

Fabric Extend over IPsec Limitations