In a previous blog post, we compiled a number of useful JQ command routines for fast malware PCAP network analysis using Suricata. In this post, we decided to use some new PCAPs to share additional command routines that you might find useful.
As before, I often find myself stumbling upon interesting social media posts by other malware researchers. These posts frequently include a PCAP file as an artifact from a malware binary that was detonated in a controlled environment such as a sandbox.
In many cases, I don’t have much extra time, so one of the first questions I need to quickly answer is, “Is this interesting?”. When I come across these files I want to know what they look like from a high level, so I can find out if the research is relevant to me.
There are of a few caveats of course – the PCAP itself or the recording of the network trace might not be long or complete – but in many cases it is good enough to provide ideas for hunting formulas, which I have found to be essential. By devising a quick way to review these pcaps, which I will share today, I can break up my daily routine while also inviting the possibility to discover something novel.
What tools are needed?
The tools we will use in this exercise are:
- A sample malware PCAP from AnyRun
- Suricata
- Cyberchef
- Proofpoint (PFPT) Suricata ruleset
- JQ
AnyRun - Innovative cloud-based sandbox with full interactive access.
Suricata - is a “high performance, open source network analysis and threat detection software used by most private and public organizations, and embedded by major vendors to protect their assets”. It is an open source tool.
Cyberchef - known as “The Cyber Swiss Army Knife” is an open-source web app for encryption, encoding, compression and data analysis.
ET Pro - “Proofpoint ET Pro is a timely and accurate rule set for detecting and blocking advanced threats using your existing network security appliances, such as next generation firewalls (NGFW) and network intrusion detection/prevention systems (IDS/IPS).” There is also a free version.
JQ - “JQ is a lightweight and flexible command-line JSON processor”. It is readily available and packaged on many OSs.
One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow, anomaly, alert, protocol, and file transaction logs, plus file extraction and packet capture (PCAP).
Here is a full list and details of what those logs and transactions look like.
The sample PCAP
Let’s have a look at an example. Today we are reviewing a scenario where we have SMTP data from publicly available network captures of Agent Tesla exfiltration malware. Please NOTE: This PCAP file can contain still active or live malware artifacts like binary and/or executables or configs and domains, so please handle with care.
During any regular day researchers working for vendors, in academia, or for other organizations publish new findings and reports of malware and behaviors on social media and blogs. Usually, a fast initial analysis (non-detailed) can reveal behaviors and help assess if this specific malware is using a novel network technique of communication or not.
There are many relevant examples that become available daily. I’ve found that reading these isolated samples with the Suricata “-r” option and then reviewing the output has been a very fast way to get a top-level picture of what’s going on. Let’s walk through this process with the above malware example.
Reviewing the PCAP with Suricata
First, we need to conduct a quick initial north-south network analysis with Suricata.
Note: This setup assumes Suricata 7 is installed on the system.
In this specific example we are looking at an AgentTesla SMTP Exfiltration snapshot in time:
Here is the command, and you can copy and paste the actual text below:
sudo suricata -S "rules/*.rules" -l logs/ -k none -r 45527614-2f1d-4f74-b3bd-2dc42608beb4.pcap ; \
echo "Suricata event types:" ; jq -r .event_type logs/eve.json | sort | uniq -c |sort -rn ; \ echo "Alerts:" ; grep '"event_type":"alert"' logs/eve.json | jq .alert.signature | sort -rn | uniq -c | sort -rn ; \
echo "TLS SNIs:" ; jq 'select(.event_type=="tls")' logs/eve.json | jq .tls.sni | sort -rn | uniq -c | sort -rn ; \
echo "TLS Versions:"; jq 'select(.event_type=="tls")' logs/eve.json | jq .tls.version | sort -rn | uniq -c | sort -rn; \
echo "HTTP Hostnames:" ; jq 'select(.event_type=="http")' logs/eve.json | jq .http.hostname | sort -rn | uniq -c | sort -rn ; \
echo "DNS Queries:" ; jq 'select(.event_type=="dns" )' logs/eve.json | jq .dns.rrname | sort -rn | uniq -c | sort -rn ; \
echo "Filetransfer protocols:" ; jq 'select(.event_type=="fileinfo" )' logs/eve.json | jq .app_proto | sort -rn | uniq -c | sort -rn ; \
echo "Filenames:" ; jq 'select(.event_type=="fileinfo")' logs/eve.json | jq .fileinfo.filename | sort -rn | uniq -c | sort -rn ; \
echo "File magic:" ; jq 'select(.event_type=="fileinfo")' logs/eve.json | jq .fileinfo.magic | sort -rn | uniq -c | sort -rn ; \
echo "Mail from:" ; jq 'select(.event_type=="smtp")' logs/eve.json | jq .email.from | sort -rn | uniq -c | sort -rn ; \
echo "Mail to:" ; jq 'select(.event_type=="smtp")' logs/eve.json | jq .email.to[] | sort -rn | uniq -c | sort -rn ; \
echo "Mail helo:" ; jq 'select(.event_type=="smtp")' logs/eve.json | jq .smtp.helo | sort -rn | uniq -c | sort -rn ;
This is the output. Below you will find a breakdown of the findings with explanations. We will skip over the ones already covered in the previous blog. In other words, we will only review the FTP protocol findings from Suricata
Let’s cover the basics. Below is a list of different unique event fields that could give us a hunting idea during an initial review.
These different unique event fields could give us some hunting ideas during our initial review. These unique protocol and other log event types were all produced by Suricata after reading the PCAP. Any of these Suricata event types can be expanded and reviewed in full detail, however we have them summed up here for the purpose of brevity and for the sake of getting a “bird’s eye” view.
Suricata event types:
62 flow records - “event_type”:”flow”
6 alerts - “event_type”:”alert”
4 DNS protocol logs - “event_type”:”dns”
1 stats entry log - “event_type”:”stats”
1 SMTP protocol log - “event_type”:”smtp”
1 anomaly log - “event_type”:”anomaly”
The six alerts are:
Alerts:
1 "ETPRO MALWARE Win32/AgentTesla/OriginLogger Data Exfil via SMTP M2"
1 "ETPRO MALWARE Agent Tesla Exfil via SMTP"
1 "ET MALWARE AgentTesla Exfil Via SMTP"
1 "ET INFO TLS Handshake Failure"
1 "ET INFO External IP Lookup Domain (ipify .org) in DNS Lookup"
1 "ET INFO External IP Address Lookup Domain (ipify .org) in TLS SNI"
The SMTP findings are the most interesting we see here. Based on the information, we can quickly see that there was SMTP mail to and from involved in the exfiltration. Those fields are displayed in the “event_type:smtp” protocol log produced by Suricata.
Mail from:
1 "asia@asiaparadisehotel.com"
Mail to:
1 "europe@asiaparadisehotel.com"
We can also see the SMTP Helo showing the hostname. This field is displayed from the “event_type:smtp” protocol log produced by Suricata.
Mail helo:
1 "User-PC"
Below you can see the full SMTP protocol logs as part of that PCAP, including SMTP application layer data and MAC (ether) addresses:
jq 'select(.event_type=="smtp")' logs/eve.json
One of the SMTP-based AgentTesla exfiltration alerts gives interesting payload results. This includes flow and application protocol SMTP metadata. Here is the JQ command to display an alert based on its message:
jq 'select(.event_type=="alert")' logs/eve.json | jq 'select(.alert.signature=="ET MALWARE AgentTesla Exfil Via SMTP")'
There we have it: a very basic, although very useful, quick analysis of a PCAP trace from a live malware sandbox detonation to give us an idea of what is happening in this example. Keep in mind that the PCAPs from these examples have a short timespan, so the idea is to see and explore the details in that specific timeframe of the malware process. However, this gives us plenty of information and ideas for structured and unstructured hunting recipes or queries, which in turn can be automated. For example, we could hunt for clear text SMTP, FTP transactions, the SMB/KRB5 user and hostnames that are present during malware proliferation and investigation, and other similar results.
The commands showcased above and from the previous blog post can also be put in a script to give us a quick overview of the major pieces of information that are essential for a fast initial review of specific malware: communication points, usernames, hostnames, email, TLS certificates, OS levels, file magic/names, and file transactions and flows.
The logs reviewed are standard JSON. There is also, naturally, a possibility to review full and summarized details of the logs in any graphic interfaces as Grafana (free dashboards for Suricata), Kibana (free dashboards for Suricata ), Splunk (free Stamus app for Suricata) and others. The purpose of these blog posts however is to do these reviews fast and easy on the command line to get an initial view (in less than 1 minute) of the network communication.
Conclusion
Some people might still consider Suricata a “legacy” intrusion detection system (IDS), but at Stamus Networks, we don’t see it that way. It is not only a highly capable IDS but also an impressive tool for gathering NSM data and full protocol , file transaction, flow and anomaly logging. And Suricata is, in fact, a powerful foundation on which to build a full-featured network detection and response (NDR) system.
Many Suricata users remain unaware of how they can use and optimize Suricata beyond the basic alerts and signatures. For more JQ tips and tricks you could also review this video recording of a webinar by Open Information Security Foundation (OISF). Additionally, further reading on this topic can be found in our free book, “Suricata for Analysts.”” the world’s first practical guide to threat detection and hunting using Suricata.
Make sure to subscribe to the Stamus Networks blog. You can also receive updates by following us on Twitter, LinkedIn, and Facebook, or by joining our Discord.
