Traffic Mirroring

EFA supports traffic monitoring on both Clos and small data center (non-Clos) fabrics, for troubleshooting issues with applications and fabrics. EFA performs traffic monitoring by means of packet mirroring in a cloud native infrastructure solution and network functions virtualization in infrastructure deployments.

Following are the ports from which you can mirror the ingress and egress traffic:

Leaf ports connecting to the compute devices
Leaf ports connecting to the neighboring MCT leaf device (ICL ports)
Border leaf ports connecting to the external gateway
Border leaf ports connecting to the neighboring MCT border leaf device (ICL ports)
Spine ports connecting to leaf devices (Fabric non-ICL ports)
Spine ports connecting to super-spine devices (Fabric non-ICL ports)
Super-Spine ports connecting to spine and border-leaf devices (Fabric non-ICL ports)

There are two types of traffic mirroring:

In-band traffic mirroring
Out-of-band traffic mirroring

Following table describes the comparison between In-band and Out-of-band traffic mirroring solution:


In-band Mirroring	Out-of-band Mirroring
No additional hardware or ports	One additional switch, one reserved port on all leaf and border leaf switches
All configuration by EFA, no separate devices to be managed	Separate configuration on mirror switch through OOB mechanisms
All ingress information, including test access point (TAP) and VLAN, can be retained and used for classification	Ingress port information and possibly VLAN information, is not retained
Fabric needs to be measured for expected extra mirror traffic	Mirroring traffic has minimal impact on normal traffic and fabric capacity, no extra measurement needed
All functionality needs to be present in ingress leaf top of rack (ToR) switch	Minimal configuration needed on EFA, and dataplane support needed in the fabric
Extra tunnel configuration in fabric underlay	Fabric underlay is unmodified
Configuration of underlay tunnels to sink app breaks underlay/overlay separation	Tunnels to sink apps are outside the domain of fabric, and do not overlap
Cannot be applied for control port mirroring	Partial reuse possible for a common mirroring solution also on control network
Fabric has to be programmed for creating additional headers and remote destination reachability, underlay or overlay separation is lost	No fabric dependency on final encapsulation and forwarding toward sink
Egress ACL rule support minimal	Two level filtering possible, once in ingress switch, and once in the dedicated mirror switch, More complicated mirror rules can be cascaded.
QoS support needed on tenant and mirrored traffic streams because they share the same fabric links	No QoS support needed, because links are separate
Cannot be leveraged for troubleshooting fabric issues, due to reliance on fabric	Can be leveraged for troubleshooting fabric issues
Fabric admin needs to do all configuration because underlay routing modifications are needed	EFA tenant admin can create TAP sessions on the fabric switches, with pre-provisioning and custom provisioning of the configuration on mirror switch by fabric admin

Note

For information about commands and supported parameters to configure traffic mirroring, see Extreme Fabric Automation Command Reference, 3.1.0 .