BlackTree Security · Infrastructure · Automation · AI

A Nexus Upgrade Can Corrupt the Configuration a Reboot Will Not Restore

An NX-OS change can corrupt a Cisco Nexus 3000 or 9000 switch’s configuration. Cisco field notice FN74371 identifies upgrades from 10.4(6)M or later 10.4(x)M to 10.5(1)F, 10.5(2)F or 10.5(3)F and downgrades from those 10.5 releases to 10.4(6)M or later 10.4(x)M. It was published on 2 February 2026 and revised on 24 September 2026 to update release numbers. This is a maintenance warning, not a new 28 September vulnerability.

Both disruptive and non-disruptive upgrades can be affected. Upward installs from 10.4(7)M or later into those early 10.5 releases are blocked. An attempted install may stop at compatibility check; a completed change may leave VLAN/SVI configuration damaged and interfaces down. Cisco recommends 10.5(4)M or later for affected 10.4 upgrades. An ordinary reload or downgrade cannot repair existing corruption. There is no non-disruptive repair: ASCII reload and reapply missing configuration, or erase and rebuild from backup.

The change window therefore needs a route that avoids the affected versions and a recovery plan fit for rebuilding a production switch if one has already taken the hazardous path.

The release pair matters more than the label on the image

The release crossing in the lead is the one to screen. A blocked install is a useful guard for that attempted path, but it does not validate a different upgrade or downgrade. Record both the starting image and the intended destination before approving a target.

Planning depends on the installed image, the target image, and whether the maintenance procedure moves through a prohibited pair. A fleet report that groups all 10.4 releases together can hide the one hop that matters. For Nexus 9000, Cisco’s 10.5 upgrade and downgrade guide points administrators to its supported upgrade and ISSU matrix and recommends install all because it performs compatibility checks. Changing boot variables and simply reloading skips those checks. Check the field notice’s release pair as well as the compatibility result.

Triage the outcome before returning traffic

If the installer stops, preserve the compatibility output and replan the route. If the switch boots but forwarding is wrong, do not assume that a displayed software version or a reachable management interface means the switch is healthy. Compare the live VLAN, SVI and interface state with a pre-change baseline before traffic is returned to the device.

That check matters most where a top-of-rack switch carries more than one service. A missing SVI can appear as an application outage, a routing problem or an isolated host failure. The switch’s recent maintenance history and exact source-target pair can turn a scattered set of symptoms into a single change incident. Capture that history before another reload destroys useful logs or changes the system state again.

Build the recovery route before the maintenance window

For a site already on an early 10.5 release, check the supported route in the current field notice and upgrade guide before planning a downgrade. Treat an emergency rollback as a new change request with its own release-pair check and service-impact assessment.

The preflight should record show version output for every switch, the intended image filename, the current and target entries in the applicable Cisco matrix, and the result of the supported installer impact check. Export both startup and running configurations to a location outside the switch and verify that the files can be retrieved. Record the VLAN, SVI, vPC and uplink state that the post-change team will compare. Arrange console or other out-of-band access, including credentials and physical or remote-hands availability. Then decide how much traffic the redundant peer can carry while one switch is unavailable.

These BlackTree operational checks make the decision reviewable and turn a backup into a recovery asset. A configuration file stored only on the switch being rebuilt is not a usable fallback.

There is also a security reason not to improvise an unsafe hop. BlackTree previously reported a separate Nexus 9000 switch flaw for which operators may have a strong reason to apply fixed software. That article covers a different issue and set of affected models. Patch urgency still requires an approved path, or the maintenance itself can interrupt the network needed to finish remediation.

Plan the rebuild as a service event

Treat recovery as a service rebuild with a topology-specific plan. Check which services depend on the switch, how the peer will carry them, and who can reach the console if remote management disappears. Use Cisco’s recovery section with TAC to select and sequence the procedure.

An ASCII reload has its own risks. Cisco’s separate technical note says it replays the text startup configuration line by line rather than using the normal binary configuration. It generally advises using the procedure with TAC direction, warns that boot may take substantially longer, and calls for a complete post-boot configuration comparison. Cisco describes how configuration order can briefly create inconsistent states. That is why a switch that answers on its console is not necessarily ready to carry production traffic.

If recovery is required, protect the peer and dependent services from a partly configured switch, retain the corrupted state and installer logs where possible, and coordinate the recovery sequence with Cisco TAC. Confirm that backups include all relevant virtual device contexts and access details before any procedure that erases configuration. Maintain console or another independent access route throughout recovery. These are high-impact actions, not routine rollback steps.

Verify service, not just the version banner

After an approved upgrade or recovery, compare the installed image and configuration against the change record. Check that expected VLANs exist with the right names and segment IDs, SVIs are up, uplinks and vPC peers are stable, and representative hosts can reach the networks they need. Recheck after the next controlled reload and preserve the results with the maintenance record.

For the next change window, verify the exact hop, take a usable backup, secure an independent way in, and do not let the word “rollback” disguise a second hazardous transition.

Sources

Leave a Reply

Your email address will not be published. Required fields are marked *