When a threat researcher is investigating malware behavior and traces on the network, they need a fast way to analyze malware PCAPs. In this blog post, we have compiled some useful JQ command routines for fast malware PCAP network analysis using Suricata.
I often find myself stumbling upon interesting social media posts by other malware researchers. These posts frequently include a PCAP file as an artifact from a malware binary that was detonated in a controlled environment such as a sandbox.
In many cases, I don’t have much extra time, so one of the first questions I need to quickly answer is, “Is this interesting?”. When I come across these files I want to know what they look like from a high level, so I can find out if the research is relevant to me.
There are of a few caveats of course – the PCAP itself or the recording of the network trace might not be long or complete – but in many cases it is good enough to provide ideas for hunting formulas, which I have found to be essential. By devising a quick way to review these PCAPs, which I will share today, I can break up my daily routine while also inviting the possibility to discover something novel.
What tools are needed?
The tools we will use in this exercise are:
- A sample malware PCAP from Triage
- Suricata
- Cyberchef
- Proofpoint (PFPT) Suricata ruleset
- JQ
Triage - (developed by Recorded Future) offers “dynamic analysis for Windows, Linux, Mac, and Android files. It runs the files in a cloud-hosted environment and provides you with a detailed report on the behavior of the malware that includes malicious scoring of the file.” The basic system is free to use.
Suricata - is a “high performance, open source network analysis and threat detection software used by most private and public organizations, and embedded by major vendors to protect their assets”. It is an open source tool.
Cyberchef - known as “The Cyber Swiss Army Knife” is an open-source web app for encryption, encoding, compression and data analysis.
ET Pro - “Proofpoint ET Pro is a timely and accurate rule set for detecting and blocking advanced threats using your existing network security appliances, such as next generation firewalls (NGFW) and network intrusion detection/prevention systems (IDS/IPS).” There is also a free version.
JQ - “JQ is a lightweight and flexible command-line JSON processor”. It is readily available and packaged on many OSs.
One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow, anomaly, alert, protocol, and file transaction logs, plus file extraction and packet capture (PCAP).
Here is a full list and details of what those logs and transactions look like.
The sample PCAP
So let’s have a look at an example. In this case we have an Arkei/Vidar type of malware stealer.
During any regular day researchers working for vendors, in academia, or for other organizations publish new findings and reports of malware and behaviors on social media and blogs. Usually, a fast initial analysis (non-detailed) can reveal behaviors and help assess if this specific malware is using a novel network technique of communication or not.
Recently, we discovered one such example – a PCAP from this sandbox detonation: https://tria.ge/230902-yzsd6afd3t/behavioral2
There are many relevant examples that become available daily. I’ve found that reading these isolated samples with the Suricata “-r” option and then reviewing the output has been a very fast way to get a top-level picture of what’s going on. Let’s walk through this process with the above malware example.
Reviewing the PCAP with Suricata
First, we need to conduct a quick initial north-south network analysis with Suricata.
Note: This setup assumes Suricata 7 is installed on the system.
Here is the command, and you can copy and paste the actual text below:
sudo suricata -S "rules/*.rules" -l logs/ -k none -r e06a97f4714b6d0e62c51b3395a64a6e.pcap ; \
echo "Suricata event types:" ; jq -r .event_type logs/eve.json | sort | uniq -c |sort -rn ; \
echo "Alerts:" ; grep '"event_type":"alert"' logs/eve.json | jq .alert.signature | sort -rn | uniq -c | sort -rn ; \
echo "TLS SNIs:" ; jq 'select(.event_type=="tls")' logs/eve.json | jq .tls.sni | sort -rn | uniq -c | sort -rn ; \
echo "TLS Versions:"; jq 'select(.event_type=="tls")' logs/eve.json | jq .tls.version | sort -rn | uniq -c | sort -rn; \
echo "HTTP Hostnames:" ; jq 'select(.event_type=="http")' logs/eve.json | jq .http.hostname | sort -rn | uniq -c | sort -rn ; \
echo "DNS Queries:" ; jq 'select(.event_type=="dns" )' logs/eve.json | jq .dns.rrname | sort | uniq -c | sort ; \
echo "Filenames:" ; jq 'select(.event_type=="fileinfo" )' logs/eve.json | jq .fileinfo.filename | sort -rn | uniq -c | sort -rn ; \
echo "File magic:" ; jq 'select(.event_type=="fileinfo" )' logs/eve.json | jq .fileinfo.magic | sort -rn | uniq -c | sort -rn
This is the output. Below you will find a breakdown of the findings with explanations:
Suricata start and stop stdout info:
Info: conf-yaml-loader: Configuration node 'DC_SERVERS' redefined. [ConfYamlParse:conf-yaml-loader.c:329]
Notice: suricata: This is Suricata version 7.0.1-dev (becb8cefc 2023-08-11) running in USER mode [LogVersion:suricata.c:1148]
Notice: threads: Threads created -> RX: 1 W: 16 FM: 1 FR: 1 Engine started. [TmThreadWaitOnThreadRunning:tm-threads.c:1890]
Notice: suricata: Signal Received. Stopping engine. [SuricataMainLoop:suricata.c:2815]
Notice: pcap: read 1 file, 7528 packets, 5663308 bytes [ReceivePcapFileThreadExitStats:source-pcap-file.c:388]
Below is a list of different unique event fields that could give us a hunting idea during an initial review.
This is a list of the different unique event fields that could give us some hunting ideas during our initial review. These unique protocol and other log event types were all produced by Suricata after reading the PCAP. Any of these Suricata event types can be expanded and reviewed in full detail, however we have them summed up here for the purpose of brevity and for the sake of getting a “bird’s eye” view.
Suricata event types:
70 dns
61 flow
18 tls
10 http
6 fileinfo
2 alert
1 stats
2 alerts were produced:
Alerts:
1 "ETPRO MALWARE Arkei Stealer Exfil"
1 "ET INFO Dotted Quad Host ZIP Request"
This is the encrypted connection's unique server name indication. This field is displayed from the “event_type:tls” protocol log produced by Suricata.
TLS SNIs:
3 "login.live.com"
1 "www.doi.org"
1 "nexusrules.officeapps.live.com"
1 "i.imgur.com"
1 "doi.org"
1 "arc.msn.com"
Here we can see which TLS versions are present in the communication. This field is displayed from the “event_type:tls” protocol log produced by Suricata.
TLS Versions:
18 "TLS 1.2"
We can also see the unique HTTP hostnames involved. This field is displayed from the “event_type:http” protocol log produced by Suricata.
HTTP Hostnames:
5 "ctldl.windowsupdate.com"
3 "79.137.206.192"
1 "ocsp.digicert.com"
1 "dl.delivery.mp.microsoft.com"
Here we can identify which type of DNS rrnames/queries are there in the DNS logs. This field is displayed from the “event_type:dns” protocol log produced by Suricata.
DNS Queries:
2 "arc.msn.com"
2 "doi.org"
2 "i.imgur.com"
2 "ocsp.digicert.com"
2 "www.doi.org"
It is important to note that the file transaction events are generated by Suricata (“event_type:fileinfo”) based on communication through FTP, HTTP, HTTP2, NFS, SMB, and SMTP protocols by Suricata. In this case, we have file transfer over HTTP only.
Filetransfer protocols:
6 "http"
Suricata also identifies the filenames from the file transaction logs. This field is displayed from the “event_type:fileinfo” protocol log produced by Suricata.
Filenames:
1 "/msdownload/update/v3/static/trustedr/en/disallowedcertstl.cab"
1 "/MFEwTzBNMEswSTAJBgUrDgMCGgUABBTrjrydRyt+ApF3GSPypfHBxR5XtQQUs9tIpPmhxdiuNkHMEWNpYim8S8YCEA6Nvl6mEObLtWnHNvbXAEs="
1 "/filestreamingservice/files/315ca852-f356-4361-8815-086f38f13875/siheng_25090.1002.cab"
1 "/dark.zip"
1 "/86277575c381e1fefb9bf290d771e6f1"
1 "/"
We can also see the filemagic of the files from the file transaction logs. This field is displayed from the “event_type:fileinfo” protocol log produced by Suricata.
File magic:
2 "ASCII text, with no line terminators"
1 "Zip archive data, at least v2.0 to extract, compression method=deflate"
1 "Microsoft Cabinet archive data, Windows 2000/XP setup, 4774 bytes, 1 file, at 0x2c last modified Sun, Jul 19 2023 17:14:08 +A \"disallowedcert.stl\", number 1, 1 datablock, 0x1 compression"
1 "Microsoft Cabinet archive data, single, 157294 bytes, 1 file, at 0x44 last modified Sun, Apr 04 2022 19:08:06 +A \"siheng.dll\", flags 0x4, number 1, extra bytes 20 in head, 12 datablocks, 0x1 compression"
1 "data"
Exploring the PCAP with Suricata
In this case, one of the interesting things that stuck out was the clear text zip file download "/dark.zip". We should investigate this further.
First we have to enter the command to fetch that file transaction record filename specifically. This can be done with the following command:
jq 'select(.event_type=="fileinfo" and .fileinfo.filename=="/dark.zip")' logs/eve.json | jq .
Below is a fully displayed file transaction event by Suricata. It contains any relevant application layer protocol information plus file hashes,filemagic ,filename ,filesize, and more flow traffic specifics.
Part of the exfil highlighted by the PFPT signature is also interesting because it uses base64 encoding:
To fetch one of the alerts that highlights exfiltration, we can use this command:
jq 'select(.event_type=="alert")' logs/eve.json | jq 'select(.alert.signature=="ETPRO MALWARE Arkei Stealer Exfil")'
The resulting alert log has metadata like the http connection specifics,response code ,user agent, request response body, and more:
Using Cyberchef to decode the log, it seems the exfil is a zip file that also contains a file called “information.txt” in the archive.
The server also replies with “b2s=”, which means “ok” decoded from base64:
All of this can happen in just a couple of minutes, giving us a very fast, high-level view of the north-south network communication with Suricata.
This is a very basic, although very useful, quick analysis of a PCAP trace from a live malware sandbox detonation to give us an idea of what is happening in this example.
Conclusion
Some people might still consider Suricata a “legacy” intrusion detection system (IDS), but at Stamus Networks, we don’t see it that way. It is not only a highly capable IDS but also an impressive tool for gathering NSM data. And Suricata is, in fact, a powerful foundation on which to build a full-featured network detection and response (NDR) system.
Many Suricata users remain unaware of how they can use and optimize Suricata beyond the basic alerts and signatures. For more JQ tips and tricks you could also review this video recording of a webinar by Open Information Security Foundation (OISF). Additionally, further reading on this topic can be found in our free book, “Suricata for Analysts.”” the world’s first practical guide to threat detection and hunting using Suricata.
Make sure to subscribe to the Stamus Networks blog, because we will soon be releasing a blog post on how to perform east-west SMB, KRB5, and FTP analysis using Suricata. You can also receive updates by following us on Twitter, LinkedIn, and Facebook, or by joining our Discord.
