The SolarWinds Breach Is Shaking Up Incident Response

The massive SolarWinds breach exposed some significant weaknesses in companies’ incident response practices.

Responding to a cyberattack like SolarWinds, where a software update process in a network management tool was compromised and attackers were able to delve deeply into targeted networks, requires the ability to analyze traffic and behavior logs that are often incomplete, or completely missing.

That hinders the incident response team's ability to track down the source of the attack and shut it down, cut off the attackers' communication channels, and determine how far the attack has spread.

Data centers too often put their full trust in their management software, allowing it full access to the enterprise and unfettered communications with the outside world, said Jerry Bessette, head of Booz Allen’s Cyber Incidence Response Program.

The SolarWinds breach has shown how dangerous of a mistake that is.

Check the Logs

When an attack like SolarWinds has occurred, and a company suspects it may have been compromised, there are several steps it can take to determine the scope of the problem.

"Look into unusual behavior, unusual network connections," Bessette told DCK. "Check your logging, look for IP addresses that shouldn't be there."

Bessette was previously chief of the FBI cyber division’s technical operations section, where he managed the national cyber incident response team.

In his experience, enterprises often fail to have the logging in place that can help them do a full investigation after an incident occurs. The SolarWinds breach is an opportunity to change that.

“If I was a CISO, I would adhere to the principle of never letting a good crisis go to waste," he said. There are plenty of organizations that don't have multi-factor authentication, network and behavior analytics, zero-trust or network segmentation, endpoint defenses, and adequate logging.

Logging was a particularly acute pain point in the SolarWinds breach because even companies that collect logs don't always keep them for a long enough period. The SolarWinds compromises lasted for months.

"Logging is storage, and storage is expensive," Bessette said. "But if you don't have the history to go back to, it's going to be hard to determine what you have on your network."

Stop the Chatter

Most malware, save for malware designed to do nothing but damage, needs to communicate back to its makers. It exfiltrates data, sends decryption keys, sends back descriptions of the environment it's in, downloads other attack tools, and gets instructions from the attackers about what to do next.

When these communications take place inside the official channels of data center management software, they can be very difficult to detect or stop.

Many enterprises don't do much scanning of outbound communications at all, much less those of trusted tools, said Bryan Sartin, chief services officer at eSentire, a cybersecurity firm. Previously, he was the executive director of global security services at Verizon Enterprise Solutions.

"I feel like an undertaker -- I'm only seeing situations when something bad happens," he told DCK. "And four times out of five, there's very little or zero screening of information leaving a victim's network."

Usually, if there's indications of compromise, one of the first steps is to cut off the attackers' communications. "If you can limit the outbound traffic, you can limit the impact of the breach," he said.

Before the SolarWinds breach was exposed, vendor channel communications likely would have been overlooked, he said. With all the attention that the attack has received, that might change, he said.

Cut Off Lateral Movement

In addition to wanting to call home, malware wants to spread, to get into as many systems as it possibly can.

The SolarWinds attackers did just that, spreading into multiple systems and taking over accounts.

To prevent things like that from happening, data centers have been increasingly turning to zero-trust architectures and network segmentation, least-privilege access, and multi-factor authentication.

If those preventative measures weren't in place before, they need to be put in place soon, since attackers quickly emulate the successful strategies of others.

The SolarWinds breach has shone a spotlight on the lateral movement detection gaps in existing security controls, said Ofer Israeli, founder and CEO of cybersecurity firm Illusive Networks.

"The richer the access footprint, the more pathways an attacker has to reach the crown jewels – and the faster damage can be done," he told DCK.

After an attack has taken place, data centers should take immediate steps to keep it from spreading further, he said.

That includes increased monitoring for suspicious lateral movements and unusual endpoint activity, said Booz Allen Hamilton's Bessette. "If you don't have an endpoint protection solution deployed already, we always recommend that you get one deployed immediately."

Attackers who have compromised an environment will also look for vulnerabilities that they can use to expand further, he added, so a data center should ensure that all patching is up to date.

"I would also order a wholesale password reset across the environment, and if you do not have multi-factor deployed, I would immediately deploy multi-factor authentication," he said.

And this is no time to slack off on monitoring for intrusion attempts. Yes, the attackers are already in the system – but other bad actors read the news and can smell blood in the water.

"News of an attack almost always leaks out," he said. "And then you'll see an overwhelming increase of scanning and attacks on your network. You should also monitor for bad login attempts – if you have a large volume, it could be someone trying to brute force the network or do credential stuffing."

But the biggest need, he said, is that defenders must get proactive.

"You have to actively hunt the environment for malicious activity," he said. "That is really where the change is coming."