In a previous blog post, we announced the release of Open NRD from Stamus Networks - a set of threat intelligence feeds for Suricata users based on newly-registered domains.
Our team has created additional resources on this topic:
- BLOG: Threat Hunting for Unknown Actors & Threats using NRD and Sightings
- VIDEO: Webinar using the NRD threat intel - "Network Threat Hunting with Suricata and SELKS"
This blog post is one of five blogs introducing Open NRD and sharing various ways it can be used with Suricata for threat hunting and investigation. To skip to the other blog posts in the series, click on one of the following links:
- Introducing Open NRD: Newly Registered Domain Threat Intel Feeds for Suricata
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 2
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 3
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 4
In this blog we will explore some techniques that can be used to hunt for malicious TTP activity on the network using Suricata generated protocol data and/or context to an alert event.
What tools are needed?
The tools we will use in this exercise are:
- Open NRD threat intel feeds - free subscription for any Suricata user
- Suricata - the article assumes Suricata 6.0.0+ or 7.0.0+ installed
- Kibana or Splunk - only needed if you wish to reproduce the example visualizations.
You can read more here on how to enable the NRDs feeds.
Before we look into the actual hunting examples, we must understand three major Suricata hunting concepts. They are:
- The wealth of data generated by Suricata (especially beyond alerts)
- How correlation is performed with Flow ID
- How flow labeling with Flowbits can impact context
Suricata-generated data - more than just alerts
One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow records, anomalies, alerts, protocol transactions, and file transaction logs, plus file extraction and packet capture (PCAP).
Here is a full list and details of what those logs and transactions look like.
You also might be interested in the many new things available in the recently released Suricata 7.
Suricata data correlation with Flow ID
Suricata produces all relevant network security monitoring logs: protocol, flow, file transaction, and anomaly logs, including the ones related to an alert - but also independent of alerts. In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged - to an alert event and that alert’s metadata.
The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014 by Suricata lead developer, Victor Julien
https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3
Flow labeling with Flowbits can provide context
One of the real benefits of using Suricata is that it allows for streaming and continuous matching with flow labeling.
This provides more context than simply correlating alerts to flows and protocol logs. Suricata can insert these labels into different flows and protocol transaction logs when producing the logs. This allows for much more elaborate detection logic.
Any flow record produced by Suricata and its relevant / related protocol, file transaction, protocols, anomaly and alert (if present) logs can be marked/labeled. This allows Suricata users to connect multiple sessions / flow logic to our hunt.
Signatures add the label to a flow which will be checked later by other signatures. And this label will appear in the JSON logs for the flow event and application layer events
Here’s an example signature:
alert smb any any -> $HOME_NET any (msg:"ETERNALBLUE"; flow:to_server,established; content:"|ff|SMB|32 00 00 00 00 18 07 c0|"; flowbits:set,eternal_malware;)
And here’s the resulting JSON log showing Flowbits and Flow ID
What does this mean?
Keeping in mind the abundance and availability of this data, we can consider a few examples of how to highlight malicious activity based on newly registered domains matching on a number of different event types produced (some that we will use are listed below out of over 30 types available in Suricata) :
- Flow logs
- TLS logs
- HTTP logs
- DNS logs
- Alert logs
Let's look at some simple examples of how malicious actors use newly-registered domains (NRD). It is important to note that not all NRD are necessarily malicious on their own. However, malware actors are known to register new domains and immediately use them for phishing, command and control, and other stages of an attack. To see this we need context – which Suricata provides by default.
Here are several simple examples that usually yield interesting results when hunting based on the presence of NRD communication in the infrastructure (depending on the type of deployment).
The following behaviors can be observed through alerts triggered by dedicated Suricata signatures:
- Large transfers to and from NRD
- Clear text executable downloaded / transferred from NRD
- New sshhash with large flow to or from NRD with Entropy
- Never seen before metadata on an NRD (dataset sightings)
These are behaviors that may be uncovered using search & analytics among Suricata protocol and flow logs:
- SSH/FTP/SMTP communication to and from NRD hosts
- Production servers doing TLS/HTTP beaconing to a newly registered domain
- Large transfers to and from NRD
Let’s explore some of these behaviors in hands-on examples that can help us uncover malicious activity on the network with Suricata. The queries below are for Elasticsearch/Kibana, but very similar ones can be executed with Splunk too.
TLS encrypted beaconing
The query below can be executed on Suricata’s “event_type”:”flow” logs. Remember, these logs are generated regardless if there is an alert or not.
Here is the Suricata TLS events Kibana/ES search query and the resulting output:
event_type:flow AND metadata.flowbits:*stamus.nrd.entropy* AND app_proto.keyword:tls
The first two results above display a large number of check-ins by already-infected host(s) via TLS encrypted connections.
The visualization above is present in the SN-FLOW dashboard (based on Suricata “event_type”:”flow” log data) as part of free and Open Source Dashboards that are available in GitHub or as part of SELKS.
You can download it at this link →
https://github.com/StamusNetworks/suricata-analytics/tree/main/kibana/7
Large HTTP transfers
The query below can be executed on Suricata’s “event_type”:”http” logs. They are generated regardless if there is an alert or not.
event_type:http AND metadata.flowbits:*stamus.nrd* AND app_proto.keyword:http AND flow.bytes_toclient:>10000
In this hunt, the query above gave us results that showed examples of infected hosts transferring data to and from NRD hosts.
A snippet of the Suricata “event_type”:”flow” record shown below:
HTTP beaconing (C&C)
The query below can be executed on Suricata’s “event_type”:”http” logs. They are generated regardless if there is an alert or not.
event_type:http AND metadata.flowbits:*stamus.nrd.entropy* AND http.status:(+>=400 +<410)
The results show us a beacon-like behavior of an infected host, sending “check ins” to its Command and Control server while sending 404 HTTP status codes.
The visualization above is present in the SN-HTTP dashboard (based on Suricata “event_type”:”http” log data) as part of free and Open Source
Dashboards that are available in GitHub or as part of SELKS.
You can download it at this link →
https://github.com/StamusNetworks/suricata-analytics/tree/main/kibana/7
Clear text executable download from newly registered domain
The query below can be executed on Suricata’s “event_type”:”alert” logs.
This is the Kibana/ES query on NRD labeled flow and logs combined with clear text executable download alert from NRD and http status code 200:
event_type:alert AND metadata.flowbits:*stamus.nrd* AND alert.signature:*exe* AND http.status:200
This input can return interesting results for review in the context of Newly Registered Domains. For example, we can see a JSON view of the default Suricata alert. Protocol file transaction, flow, and PCAP data are also all produced by Suricata and made available.
We can see that at the time the alert triggered the domain was 15 days old.
Suricata produces all the relevant network security monitoring logs such as protocol, flow, file transaction, and anomaly logs, including the ones related to an alert. However, Suricata also produces these logs independent of alerts.
In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged to an alert event and that alert’s metadata.
The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014 by Suricata lead developer, Victor Julien. You can see the first iteration of that at this link →
https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3
We also have access to the file-related information from the transfer including hash, name, size, magic, and others in the Suricata “event_type”:”fileinfo” log produced by Suricata (which will be produced regardless if there is an alert or not):
{
"size": 410520,
"sha256": "9a353d4b85b3097762282703f1807c2b459698966b967280c8e4e13cc56d2e28",
"type": "PE32 executable (DLL) (GUI) Intel 80386",
"tx_id": 0,
"magic": "PE32 executable (DLL) (GUI) Intel 80386, for MS Windows",
"gaps": false,
"sid": [],
"stored": false,
"filename": "44251798532407400000.dat",
"state": "CLOSED"
}
Furthermore, each protocol, flow, anomaly, or file transaction log will have extra labeling inserted in the standard JSON log by way of flowbits:
{
"flowbits": [
"ET.zbot.dat",
"et.IE7.NoRef.NoCookie",
"et.MS.XMLHTTP.no.exe.request",
"stamus.nrd",
"ET.http.binary"
]
}
In other words , all records to and from NRD communication that are found in either DNS query (“event_type”:”dns”), HTTP hostname (“event_type”:”http”), and/or TLS sni (“event_type”:”tls”) will have the flowbit/marker inserted in them as well as any corresponding “event_type”:”fileinfo” , “event_type”:”flow” , “event_type”:”anomaly” event logs from those sessions/flows.
Conclusion
Some people might still consider Suricata a “legacy” intrusion detection system (IDS), but the data produced by Surcata speaks for itself. It is not only a highly capable IDS but has evolved into an impressive tool for gathering NSM data and full protocol, file transaction, flow, and anomaly logs along with file extraction and PCAP logging. And as we’ve shown at Stamus Networks, Suricata is in fact a powerful foundation on which to build a world-class network detection and response (NDR) system.
We hope this article helps to dispel some of these misconceptions and can help users see the many ways to use and optimize Suricata beyond the basic alerts and signatures.
Additional resources
Here is a full list of the Suricata event types and they information each includes.
To read part 2 of this series, please visit this link.
Further reading on this topic can be found in our free book, “Suricata for Analysts.”” the world’s first practical guide to threat detection and hunting using Suricata.
Please make sure you check out our free and Open Source contributions to Suricata on our Stamus Labs page.
If you wish to be notified when we publish similar content, please subscribe to the Stamus Networks blog.
Finally, to receive near-real time updates, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.
