Before beginning any sort of threat hunt, it is important to consider the tools you are using. This is especially true in the case of Suricata. Some people still consider Suricata to be a “legacy” intrusion detection system (IDS), but those that are more familiar with the tool know otherwise. Suricata has evolved into an impressive tool for gathering NSM data and full protocol, file transaction, flow, and anomaly logs along with file extraction and PCAP logging.
With this in mind, there are three Suricata fundamentals that must be understood before beginning a threat hunt:
- The wealth of data generated by Suricata (especially beyond alerts)
- How correlation is performed with Flow ID
- How flow labeling with Flowbits can impact context
By understanding and implementing these principals, you can have a more successful and productive hunt that uses Suricata to its full ability.
Let’s see how it all comes together.
Suricata-generated data - more than just alerts
One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow records, anomalies, alerts, protocol transactions, and file transaction logs, plus file extraction and packet capture (PCAP).
Here is a full list and details of what those logs and transactions look like.
Suricata data correlation with Flow ID
Suricata produces all relevant network security monitoring logs: protocol, flow, file transaction, and anomaly logs, including the ones related to an alert - but also independent of alerts. In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged - to an alert event and that alert’s metadata.
The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014.
https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3
Flow labeling with Flowbits can provide better context on what to look at first
One of the real benefits of using Suricata is that it allows for streaming and continuous matching with flow labeling.
This provides more context than simply correlating alerts to flows and protocol logs. Suricata can insert these labels into different flows and protocol transaction logs when producing the logs. This allows for much more elaborate detection logic.
Any flow record produced by Suricata and its relevant / related protocol, file transaction, protocols, anomaly and alert (if present) logs can be marked/labeled. This allows Suricata users to connect multiple sessions / flow logic to our hunt.
Signatures add the label to a flow which will be checked later by other signatures. And this label will appear in the JSON logs for the flow event and application layer events
Hands-on example
Let’s take all this information and see how it can actually help us in a threat hunt. Assuming you have the latest stable version of Suricata 6 or Suricata 7 (at the time of this writing) with a regular ETOpen ruleset, you can recreate this example at home.
We are going to do a quick review of a malicious FormBook example PCAP from AnyRun that also has an OpenDir (open directory).
In terms of malware, OpenDir simply means that it is an open/unprotected access folder that allows anyone to download its contents. Usually, malware actors populate these types of folders with ready-to-use loaders, downloaders, executables, and other similar files.
For this review, we will be using some of the jq queries found in our blog post “Malware PCAP Analysis Made Easy”.
Traditionally, we have a number of challenges when it comes to security monitoring. There is a lot of data that needs to be sorted through to eliminate false positives. Here are some examples of how we can perform this process.
First, we have a hypothesis that executable downloads have a high risk of being something unwanted. Second, because this actually happens quite often, we need to look at the executables that have a high likelihood of being malicious.
Our example PCAP is exactly this type of situation.
We simply can run it through Suricata like so:
sudo suricata -l logs/ -k none -r 2f004127-f264-44b4-83d0-7b3f78915ba2.pcap
With the first command below we can look at a signature output, for example:
jq 'select(.event_type=="alert")' logs/eve.json | jq 'select(.alert.signature=="ET MALWARE Likely Evil EXE download from dotted Quad by MSXMLHTTP M2")' | more
{
"timestamp": "2023-12-19T13:43:28.271528+0100",
"flow_id": 1994419372754444,
"pcap_cnt": 258,
"event_type": "alert",
"src_ip": "198.46.178.135",
"src_port": 80,
"dest_ip": "192.168.100.85",
"dest_port": 49165,
"proto": "TCP",
"pkt_src": "wire/pcap",
"metadata": {
"flowbits": [
"exe.no.referer",
"http.dottedquadhost",
"et.IE7.NoRef.NoCookie",
"et.MS.XMLHTTP.ip.request",
"ET.http.binary"
]
},
"ether": {
"src_mac": "52:54:00:36:3e:ff",
"dest_mac": "06:b2:99:6d:78:fe"
},
"tx_id": 0,
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 2022051,
"rev": 2,
"signature": "ET MALWARE Likely Evil EXE download from dotted Quad by MSXMLHTTP M2",
"category": "A Network Trojan was detected",
"severity": 1,
"metadata": {
"created_at": [
"2015_11_09"
],
"former_category": [
"CURRENT_EVENTS"
],
"updated_at": [
"2015_11_09"
]
}
},
"http": {
"hostname": "198.46.178.135",
"url": "/2545/wlanext.exe",
"http_user_agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
"http_content_type": "application/x-msdownload",
"http_method": "GET",
"protocol": "HTTP/1.1",
"status": 200,
"length": 44584,
"http_response_body_printable": "MZ......................@.............
……
……
[clipped]
You can see full protocol, flow, file transaction, and alert logs from that specific flow here.
We have 6 total alerts (“event_type”:”alert”), 1 http log (“event_type”:”http”), 1 file transaction log (“event_type”:”fileinfo”) and 1 flow log (“event_type”:”flow”).
One thing to note from the above section is the flowbits section. Many of the signatures created by Suricata actually insert labels into the log. Thus, if we look at the specific file transaction ("event_type":"fileinfo") part, we can see a few interesting pieces of information.
jq 'select(.event_type=="fileinfo" and .flow_id==1994419372754444 )' logs/eve.json
{
"timestamp": "2023-12-19T13:43:17.261811+0100",
"flow_id": 1994419372754444,
"event_type": "fileinfo",
"src_ip": "198.46.178.135",
"src_port": 80,
"dest_ip": "192.168.100.85",
"dest_port": 49165,
"proto": "TCP",
"pkt_src": "stream (flow timeout)",
"metadata": {
"flowbits": [
"exe.no.referer",
"http.dottedquadhost",
"et.IE7.NoRef.NoCookie",
"et.MS.XMLHTTP.ip.request",
"ET.http.binary"
]
},
"http": {
"hostname": "198.46.178.135",
"url": "/2545/wlanext.exe",
"http_user_agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)",
"http_content_type": "application/x-msdownload",
"http_method": "GET",
"protocol": "HTTP/1.1",
"status": 200,
"length": 888384
},
"app_proto": "http",
"fileinfo": {
"filename": "/2545/wlanext.exe",
"magic": "PE32 executable (GUI) Intel 80386, for MS Windows, 5 sections",
"gaps": false,
"state": "CLOSED",
"md5": "c810e663dd2ada28c1bb8ee928f1372f",
"sha1": "1a6bba568ae6a4b5df50db9b4f7ec8adc463773a",
"sha256": "82f300971534143367e928f8df3b520cb497f503deff537e1094118ce3df2982",
"stored": true,
"file_id": 1,
"size": 888384,
"tx_id": 0
}
}
Here we can see three things:
- 1. The flow_id allows us to correlate all the events in the flow.
- 2. We have an extracted file for further analysis if needed.
- 3. We have flow and a file transaction that have been labeled by 5 flowbits, mainly:
- - "exe.no.referer",
- - "http.dottedquadhost",
- - "et.IE7.NoRef.NoCookie",
- - "et.MS.XMLHTTP.ip.request",
- - "ET.http.binary"
All this gives us an idea that this is a binary/executable file downloaded directly from IPv4 without references or cookies and that it is MSXML.
So far so good. Now how can we do a better job of looking at the flows and executable files that have the highest likelihood of being interesting for investigation ?
We can try an idea and put all the above observations to work in Kibana or Splunk.
Let’s look at all the unique flows that have executable file transfers via HTTP with status code 200 and sort them by the number of unique flowbits in each flow. We want to look at the highest number of unique flowbits in the flow, which highlights more things happening. We can see this in the Kibana visualization below:
Additional Resources
And there we have it: an example of our three Suricata threat hunting fundamentals in action. These concepts are not only useful for reviewing malicious PCAPs, but for any threat hunt or investigation performed with Suricata.
To learn more about how you can optimize your Suricata use, read the free book “The Security Analyst’s Guide to Suricata” – the world’s first practical guide to threat detection and hunting with Suricata.
Please make sure you check out our free and Open Source contributions to Suricata on our Stamus Labs page.
For more examples of threat hunting with Suricata, read our series on “Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD)”. You can download the free newly-registered domain (NRD) threat intel lists from Stamus Labs here.
If you wish to be notified when we publish similar content, please subscribe to the Stamus Networks blog.
Finally, to receive near-real time updates, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.
