configure sys-recovery-level slot

configure sys-recovery-level slot [all | slot_number] [none | reset | shutdown]

Description

Configures a recovery option for instances where an exception occurs on the specified SummitStack slot.

Syntax Description

all Specifies all slots of the SummitStack.
slot_number Specifies the slot. Values are 1 through 8 indicating master, backup, or standby slots.
none Configures the SummitStack slot to maintain its current state regardless of the detected fault. The offending slot is not reset. For more information about the states of a slot, see the show slot command.
reset Configures the offending slot to reset upon fault detection. For more detailed information, see the Usage Guidelines.
shutdown Configures the switch to shut down all slots configured for shutdown upon fault detection. On the slots configured for shutdown, all ports in the slot are taken offline in response to the reported errors; however, the slots remain operational for debugging purposes only. ExtremeXOS logs fault, error, system reset, system reboot, and system shutdown messages to the Syslog.

Default

The default setting is reset.

Usage Guidelines

Use this command for system auto-recovery upon detection of problems. You can configure the SummitStack slots to take no action, automatically reset, shutdown, or failover from the master to the backup slot, if the switch detects a faulty master slot. This enhanced level of recovery detects faults in the ASICs, as well as packet buses.

You must specify one of the following parameters for the system to respond to slot failures:

  • none—Configures the slot to maintain its current state regardless of the detected fault. The offending slot is not reset. ExtremeXOS logs fault and error messages to the Syslog and notifies you that the errors are ignored. This does not guarantee that the slot remains operational; however, the stack does not reboot the slot.
  • reset—Configures the offending slot to reset upon fault detection. ExtremeXOS logs fault, error, system reset, and system reboot messages to the syslog.
  • shutdown—Configures the stack to shut down all slots configured for shutdown upon fault detection. On the slots configured for shutdown, all ports in the slot are taken offline in response to the reported errors; however, the master and backup slots remain operational for debugging purposes only. You must save the configuration, using the save configuration command, for it to take effect. ExtremeXOS logs fault, error, system reset, system reboot, and system shutdown messages to the Syslog.

Depending on your configuration, the switch resets the offending slot if fault detection occurs. An offending master/backup is reset any number of times, and the master/backup is not permanently taken offline. Other offending slots are reset a maximum of five times. After the maximum number of resets, the slot is permanently taken offline.

Messages Displayed

If you configure the system recovery setting to either none (ignore) or shutdown, the switch prompts you to confirm this action. The following is a sample shutdown message:

 Are you sure you want to shutdown on errors? (y/n) 

Enter y to confirm this action and configure the system recovery level. Enter n or press [Enter] to cancel this action.

Taking Ports Offline

Beginning with ExtremeXOS 11.5, you can configure the switch to shut down one or more slots upon fault detection by specifying the shutdown option. If you configure one or more slots to shut down and the switch detects a fault, all ports belonging to all of the configured-for-shutdown slots are taken offline in response to the reported errors. (Masters/backups are available for debugging purposes only.)

The affected slot remains in the shutdown state across additional reboots or power cycles until you explicitly clear the shutdown state. If a slot enters the shutdown state, the slot actually reboots and the show slot command displays the state of the slot as Initialized; however, the ports are shut down and taken offline. For more information about clearing the shutdown state, see the clear sys-recovery-level command.

Module Recovery Actions

The following table describes the actions recovery takes based on your recovery setting. For example, if you configure a recovery setting of reset for a slot, the slot is reset a maximum of five times before it is taken permanently offline.

Click to expand in new window

Slot Recovery Actions

Slot Recovery Setting Slot Type Action Taken
none Master

The master slot remains powered on in its current state.

Master/backup

The master/backup slot remains powered on in its current state.

This does not guarantee that the slot remains operational; however, the stack does not reboot the slot.

Regular slot

The slot remains powered on in its current state. The stack sends error messages to the log and notifies you that the errors are ignored.

This does not guarantee that the slot remains operational; however, the stack does not reboot the slot.

reset Master Resets the master.
Master/backup For the master, resets the master and fails over to the backup.
Regular slot Resets the slot a maximum of five times. After the fifth time, the slot is permanently taken offline.
shutdown Master The master is available for debugging purposes only (the regular slot ports also go down); however, you must clear the shutdown state using the clear sys-recovery-level command for the master/backup to become operational.

After you clear the shutdown state, you must reboot the stack.

For more information see the clear sys-recovery-level command.

Master/backup

The master/backup is available for debugging purposes only (the regular slot ports also go down); however, you must clear the shutdown state using the clear sys-recovery-level command for the master/backup to become operational.

After you clear the shutdown state, you must reboot the stack.

For more information see the clear sys-recovery-level command.

Regular slot

Reboots the slot. When the slot comes up, the ports remain inactive until you clear the shutdown state using the clear sys-recovery-level command for the slot.

After you clear the shutdown state, you must reset each affected slot or reboot the stack.

For more information see the clear sys-recovery-level command.

Displaying the Module Recovery Setting

To display the recovery setting, use the show slot command.

Beginning with ExtremeXOS 11.5, the show slot output has been modified to include the shutdown configuration. If you configure the slot recovery setting to shutdown, the output displays an “E” flag that indicates any errors detected on the slot disables all ports on the slot. The “E” flag appears only if you configure the slot recovery setting to shutdown.

Note

Note

If you configure one or more slots for shutdown and the stack detects a fault on one of those slots, all of the configured slots enter the shutdown state and remain in that state until explicitly cleared.

If you configure the recovery setting to none, the output displays an "e" flag that indicates no corrective actions will occur for the specified slot. The "e" flag appears only if you configure the slot recovery setting to none.

The following sample output displays the slot recovery action. In this example, notice the flags identified for slot 2:

???need Summit example????
Slots    Type                 Configured           State       Ports  Flags
-------------------------------------------------------------------------------
Slot-1   8900-G96T-c          8900-G96T-c          Operational   96   MB
Slot-2   8900-10G24X-c        8900-10G24X-c        Operational   24   MB   E
Slot-3   8900-40G6X-xm        8900-40G6X-xm        Operational   24   MB
Slot-4   G48Xc                G48Xc                Operational   48   MB
Slot-5   G8Xc                 G8Xc                 Operational    8   MB
Slot-6                                             Empty          0
Slot-7   G48Te2(PoE)          G48Te2(PoE)          Operational   48   MB
Slot-8   G48Tc                G48Tc                Operational   48   MB
Slot-9   10G4Xc               10G4Xc               Operational    4   MB
Slot-10                                            Empty          0
MSM-A    8900-MSM128                               Operational    0
MSM-B    8900-MSM128                               Operational    0
Flags : M - Backplane link to Master is Active
B - Backplane link to Backup is also Active
D - Slot Disabled
I - Insufficient Power (refer to "show power budget")
e - Errors on slot will be ignored (no corrective action initiated)
E - Errors on slot will disable all ports on slot

Displaying Detailed Module Recovery Information

To display the slot recovery setting for a specific port on a slot, including the current recovery mode, use the following command: show slot slot

In addition to the information displayed with show slot, this command displays the recovery setting configured on the slot. The following truncated output displays the recovery setting, displayed as "Recovery Mode," (shown in bold) for the specified slot:

# show slot 2 detail 

Slot-2 information:
     State:               Operational
     Download %:          100
     Restart count:       0 (limit 5)
     Serial number:       800601-00-02 1510G-00165
     Hw Module Type:      X450G2-48p-10G4
     SW Version:          21.1.0.29
     SW Build:            21.1.0.29
     Configured Type:     X450G2-48p-10G4
     Ports available:     52
     Recovery Mode:       Reset
     Debug Data:          Peer=Operational
     Node MAC:            02:04:96:97:F4:F1
     Current State:       BACKUP (In Sync)
     Image Selected:      secondary
     Image Booted:        secondary
     Primary ver:         21.1.0.18
     Secondary ver:       21.1.0.29
     Config Selected:     primary.cfg

Troubleshooting Slot Failures

If you experience a slot failure, use the following troubleshooting methods when you can bring the slot offline to solve or learn more about the problem:
  • Restarting the slot—Use the disable slot slot command followed by the enable slot slot command to restart the offending slot. By issuing these commands, the slot and its associated fail counter is reset. If the slot does not restart, or you continue to experience slot failure, please contact Extreme Networks Technical Support.
  • Running diagnostics—Use the run diagnostics normal slot command to run operational diagnostics on the offending slot to ensure that you are not experiencing a hardware issue. If the slot continues to enter the failed state, please contact Extreme Networks Technical Support.

Example

The following example configures a slot to not take an action if a fault occurs:

configure sys-recovery-level slot none

History

This command was first available in ExtremeXOS 11.3.

The shutdown parameter was added in ExtremeXOS 11.5.

Platform Availability

Summit X450-G2, X460-G2, X670-G2, X770, and Extreme Switching X440-G2, X620.