Supply Chain Blunder Means Cisco Servers Could Lose Data

Oops! There’s been a snafu in the Cisco supply chain that’s resulted in UCS servers being shipped that could lose data under a power loss. The problem is with hard drives from Seagate with an incorrect setting. The drive maker has already issued a mea culpa, which might’ve absolved Cisco of responsibility if not for the fact that Cisco installed and shipped the drives without checking to make sure they were configured properly. Sometimes one hand dirties the other.

The problem isn’t complicated. It seems that some Serial Attached SCSI 7.2K RPM Large Form Factor drives had drive write cache enabled. As Cisco put it with deadpan simplicity in its field notice: “If drive write cache is enabled during a power loss it can result in loss of data.” The issue affects Cisco UCS C220-M3/M4L, C240-M3/M4L and UCSC-C3X60 servers.

“Cisco ships all of their hard drives from manufacturing with drive write cache disabled.” Cisco said. “During a quality audit, select units were found to have the drive write cache enabled. The issue has been remediated in the manufacturing process. Users of potentially affected devices are recommended to change the drive cache configuration.”

The problem is that with drive write cache enabled the cache is not flushed prior to a power loss, meaning data in the cache will be lost. The workaround is to disable write cache, which is simple enough — although when dealing with technology, “simple” is a relative term.

“Users will generally have two types of setups with their hard drives; just a bunch of disks (JBOD) and redundant array of independent disks (RAID). The procedure to change the drive write cache settings differs depending on the OS and which setup the drive is in. In order to use the correct tool, you will have to know which OS you have and which storage volume setup is configured.”

The good news is that Cisco includes a chart to determine the proper tool to use, along with step-by-step instructions for resolving the issue. The bad news is that the disabling process could be a little time consuming, especially for those running the affected model UCSC-C3X60, which can house up to 56 disks.

What a way for an admin to start the week, eh?