When the blue team needs to mount a network defense, they must answer some very common questions:

• What type of environment needs to be defended?
• What types of threats can we expect to encounter?
• What is the baseline status of my environment?

And if bad things are found, they must quickly assess the situation, asking:

• What happened, and when, and how?
• What assets were impacted and in what ways?
• Do I have all the details needed to respond with confidence?

As part of a recent proof-of-concept, we were asked by the datacenter operators of a friendly hosting service provider, to answer these questions. They were interested to understand the cyber threat landscape in their environment and if they needed to address anything serious and imminent.

Going in, we had zero knowledge of the environment. We did not know who was friendly or who was a foe. We had no knowledge of the network at all, and we had no knowledge of what's considered “good” or “trusted” endpoints.

In this article we explain what we found.

NDR required where EDR is not possible

Because the nature of the hosting services in this environment does not give the hosting provider access to the actual endpoints, deploying an EDR solution is not possible. So a network monitoring solution is the only practical way to gain insights into the security posture.

So, with no control over endpoints, server installations and hosting setups and plenty of encrypted traffic transferring among various public sector and commercial organizations present, we were presented with an ideal scenario for our network detection and response (NDR) solution.

Additionally, because an NDR is a passive network monitoring system, it is completely non-intrusive to the business. That is, it does not need to be installed on every single host but instead on one or more congestion points in the network - simply sniffing traffic off a TAP or SPAN/mirror port.

For this proof of concept, our Stamus Network Probes were positioned outside the firewall, so we had an unfiltered view of the inbound and outbound network activity. The traffic included a mixture of business critical applications, web hosting setups, databases, email hosting, consumer-facing applications, and others.

The setup

The setup we deployed was fairly typical.

We installed a Stamus Network Probe, and the customer directed a subset of their network traffic using a 10 Gbps mirror port to the probe. Stamus Security Platform was installed on the customer’s hardware and the entire system was deployed with the full Stamus Network Detection and Response (NDR) license package.

We configured the system with the Stamus threat intelligence for high confidence events (Declarations of Compromise™), self-learning anomaly detection (Stamus Sightings™) as well as a third party ruleset (ETPro) and IP/Domain reputation lists.

Now what?

After setting up the Stamus Solution we were faced with two possible approaches:

• Rely on automation to triage the massive volume of events
• Proactively hunt to uncover everything possible about the environment

Due to our limited access and resource constraints (not unlike a typical security team), we chose to rely entirely on automation for the one week period.

After one week: Holy cow!

Over the course of one week, we were able to get a pretty good idea of the big picture situation, and collect a treasure trove of supporting forensic network log evidence.

We observed the following with a single Stamus Network Probe monitoring a 10G mirror port connection

• Small percentage of the total datacenter traffic
• 37,665,850 hosts/endpoints tracked
• 16 Billion security events (at peak hours Stamus enrichment and detection on the probe was processing 780,000-1,000,000 events per second)
• 28 Declarations of Compromise™
• 95 impacted assets
• 8 different tunneling techniques observed
• Numerous Stamus Sightings™ - potential beacons detected

See the Stamus Security Platform summary screen below

Threat families

During this one week, we uncovered activities in nearly all of the major threat families tracked by Stamus Security Platform, including:

• Phishing
• Data Theft
• APT
• RAT
• Trojan
• Botnet
• Offensive Tools
• Ransomware

Naturally, the APTs presented the most worrying of all these for the hosting provider because many of their customers are high-profile business and government agencies.

Malware activity - distinguishing between friend and foe

Stamus Security Platform is particularly good at detecting malware activity on the network and ascertaining which systems are the offenders and which assets are being targeted. During this one week, we uncovered activities from more than two dozen malware types, including:

• Sutra
• njRAT- v2
• DNS TunnelProxy
• RedLine
• Cobalt Group
• Gh0st
• Obfus
• Android Agent.BQX
• Hiddad
• Mirai
• Graftor
• QuasarRAT
• OceanLotus
• AsyncRAT
• GandCrab
• Igexin
• Kryptik
• Meterpreter
• Trickbot
• Trojan Agent
• Ursnif
• Hiddapp
• Injector
• Joker
• Loadmoney
• NanoCore
• Zeus
• njRAT
  
  

After identifying who was friend and who was foe, naturally a bit more information was needed to determine how things evolved with each specific threat/threat actor. For example, is there a connection between different groups? In what stage are each of the attacks associated with the assets?

In the screenshot below we see an example of the relationship of the threat activities among different affected assets.

These relationships show malware spreading laterally within this otherwise friendly environment.

So, what happened and when?

One of the more powerful investigative features of Stamus Security Platform is its ability to show a timeline of the affected assets and the progression of an an attack throughout the stages of the cyber kill chain.

While it’s always a little unnerving, I never get tired of discovering the various ways that the different malware and threat actors work together and the communications channels they use.

Forensic evidence and details

Unfortunately, we cannot publicly share any of the specific details, but I must proudly acknowledge that even Docker, Docker swarm and Kubernetes installations and keystrokes are not off limits to the Stamus Security Platform automated detection. And in this case the system was able to provide impressive forensic log evidence in both volume and quality.

Here are just a few examples of what Stamus NDR observed during this week:

Conclusion

This specific deployment was not typical for us, as it was very different from our usual big corporate organization setups. As such, it brought with it a number of challenges. As I mentioned at the beginning, we had no idea who was friend or foe, no idea of what endpoints we might encounter and no idea of what to expect in terms of the potential threat landscape.

Additionally, the blue team on site was faced with many constraints, including time, resources, costs, and privacy. And because the environment did not lend itself to endpoint solutions, there are limited possibilities in terms of monitoring solutions they could deploy.

In just one short week, the Stamus Networks team was able to help out the blue team at one of our partner data centers. We are proud to have been able to uncover several serious and imminent threats, and identify a few nefarious threat actors associated with an APT actor group and support it all with detailed timeline of events and with substantial evidence.