IEEE 802.1Qbb Priority Flow Control

Priority flow control (PFC) as defined in the IEEE 802.1Qbb standard is an extension of IEEE 802.3x flow control, which is discussed in IEEE 802.3x Flow Control.

When buffer congestion is detected, IEEE 802.3x flow control allows the communicating device to pause all traffic on the port, whereas IEEE 802.1Qbb allows the device to pause just a portion of the traffic while allowing other traffic on the same port to continue.

For PFC, when an ingress port detects congestion, it generates a MAC control packet to the connected partner with an indication of which traffic priority to pause and an associated time for the pause to remain in effect. The recipient of the PFC packet then stops transmission on the priority indicated in the control packet and starts a timer indicating when traffic can resume.

Traffic can resume in two ways:

On the transmitting side, when the timer expires, traffic on that priority can resume.
On the receiving side, once congestion is relieved, another PFC packet is generated to un-pause the priority so that traffic can resume.

Limitations

The following limitations are associated with this feature:

PFC must be explicitly configured by the user.
In order to support the signaling of congestion across the fabric, an enhanced fabric mode is required. This enhanced mode is not available on some older models of Summits (see the following supported platforms section). Also, this enhanced mode reduces the effective bandwidth on the fabric by a small amount (less than 5%).
The fabric flow control packets take up some small amount of bandwidth on the fabric ports.

Supported Platforms

PFC is currently supported several port types, depending on the platform:

ExtremeSwitching 5320 switches: 10G uplink ports.
ExtremeSwitching 5420F switches: 10G ports
ExtremeSwitching 5420M switches: 10G, 25G ports
ExtremeSwitching 5520 switches: 5G, 10G, 25G, 40G ports
ExtremeSwitching 5720 switches: 5G, 10G, 25G, 40G, 50G, 100G ports
Extreme 7520 switches: 5G, 10G, 25G, 40G, 50G, 100G ports
Extreme 7720 switches: 5G, 10G, 25G, 40G, 50G, 100G ports

If you attempt to enable PFC on unsupported ports, an error message is displayed. (See Abnormal Configuration Examples.)

Setting the Priorities

Priority is established for reception of PFC packets with a QoS profile value on the ExtremeXOS switch and for transmission with a priority value added to the PFC packet.

QoS profile—Ingress traffic is associated with a QoS profile for assignment to one of eight hardware queues in the system that define how the traffic flows with respect to bandwidth, priority, and other parameters. By default, there are two QoS profiles (QP1 and QP8) defined in these supported platforms and PFC works with this default. To segregate the ingress traffic with more granularity, you will want to define other QoS profiles. The traffic that will be paused on reception of the PFC packet is associated with the hardware queue of the QoS profile that you specify.

The QoS profile is also used to configure the fabric ports.
Priority—When the hardware transmits a PFC packet, it uses the priority bits in the VLAN header on the ingress packet to determine the priority to pause, if the ingress packet is tagged. You can specify this transmit priority independently from the QoS profile to associate it with the reception of a PFC packet thus giving flexibility in the configuration of the network. For untagged ingress packets, the hardware queue determines the priority in the transmitted PFC packet. (For additional information, see QoS Profiles.

It is suggested that the priority in the VLAN header match the QoS profile priority when traffic ingresses at the edge of the network so that the traffic can be more easily controlled as it traverses through the network.